Tag Archives: c#

Handling nullables efficiently

I earlier wrote about Handling null checks efficiently by using extension methods to make our code more terse, not cluttered by null check blocks. When we use nullables (Nullable<T>), we come across a similar construction in our code. We have to check if the nullable has a value before accessing the value. If we try to reference the value of a nullable, and it has no value, we get an InvalidOperationException. This is quite similar to getting an NullReferenceException if trying to access a reference which is null.

Let’s take the example code from my previous post, and add an enum for Nationality. Then, we add a Citizenship property with type Nullable<Nationality> to the Person class. Like so:

enum Nationality
{
    No,Se,Dk,Uk,Us
}
internal class Person
{
    public string Name { get; set; }
    public Address Address { get; set; }
    public Nationality? Citizenship { get; set; }
}

If we then create a Person object that has the Citizenship property not set, we get an exception if we try to dereference it:

var person = new Person {Name = "Jackie Chiles"};
Console.Write("Citizenship:");
Console.Write(person.Citizenship.Value);

Handling nullables efficiently

What we can do here, is that we can create an .IfHasValue and .DoIfHasValue extension methods that are analogous to the .IfNotNull and .DoIfNotNull extension methods in the previous blog post:

public static class FlowControlExtensions
{
    [DebuggerStepThrough]
    public static void DoIfHasValue<T>(this T? obj, 
        Action<T> action, bool doContinue = true) where T : struct
    {
        if (obj.HasValue)
        {
            action(obj.Value);
        }
        if (doContinue)
            return;
        ThrowInvalidOperationException<T, T>();
    }
    [DebuggerStepThrough]
    public static TResult IfHasValue<T, TResult>(this T? obj, 
        Func<T, TResult> func, bool doContinue = true, 
        TResult defaultValue = default(TResult)) where T : struct
    {
        if (obj.HasValue)
        {
            return func(obj.Value);
        }
        return doContinue ? defaultValue : 
            ThrowInvalidOperationException<T, TResult>();
    }
    [DebuggerStepThrough]
    private static TResult ThrowInvalidOperationException<T, TResult>()
    {
        throw new InvalidOperationException(string.Format(
            "Tried to access value of nullable of {0}, but it had no value",
                typeof(T).FullName));
    }
}

Now we can use this to prevent our code from throwing an exception, and return a default value instead:

 var person = new Person {Name = "Jackie Chiles" };
 Console.Write("Citizenship:");
 Console.Write(person.Citizenship.IfHasValue(c => c.ToString()));

If we wish to give it another default value, we can simply do this like so:

Console.Write(person.Citizenship
    .IfHasValue(c => c.ToString(), defaultValue: "(unknown)"));

You can find the code on GitHub

Handling null checks efficiently

At the QCon Conference in London in 2009, Sir C.A.R. Hoare apologized for inventing the null reference back in 1965 for the Algol W programming language. He called it his billion-dollar mistake.

Checking for null is one of the constructs that we use very often in our C# code. If we have a lot of them, they tend to clutter our code and make it harder to read and understand the business logic that our code implements. In this post I will use extension methods to make a bit more terse null checks to handle a few different code cases. It is by no means a novel idea, you can find various references on the web. Anyway, let’s give it a try.

Let’s consider this simple object model:

    +--------+    +-----------------+
    | Person |    | Address         |
    |--------|+-->|-----------------|
    | Name   |    | City            |
    +--------+    | PostalCode      |
                  | Street          |
                  | WriteToConsole()|
                  | ToString()      |
                  +-----------------+

We have a Person class which contains a reference to an Address class. When our program are trying to reference properies of the Address objects via the Person object, we have to consider that the reference to the Address object might be null. Here’s an example of what we would start out with:

var person = new Person {Name = "Cosmo Kramer"};
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
person.Address.WriteToConsole();

Of course, if person.Address is null, this will give us a NullReferenceExecption. So, one way to handle this, is to check for null before calling WriteToConsole, like so:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
if (person.Address != null)
{
    person.Address.WriteToConsole();
}

This is good, but if we have a lot of such null checks like these, our code will be quite cluttered. One simple solution would be to create an extension method that let us give a code to be called if the object called on is not null. Consider the following method:

public static class FlowControlExtensions
{
    public static void DoIfNotNull<T>(this T obj,
        Action<T> action) where T : class
    {
        if (obj != null)
        {
            action(obj);
        }
    }
}

If we use this new method, our code now looks like this:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
person.Address.DoIfNotNull(a => a.WriteToConsole());

Then we have a little more terse code. In this example, the execution will continue if the Address property of the person object is null. In some situations, nulls are not acceptable and we would like to throw an exception. For example:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
if (person.Address != null)
{
    person.Address.WriteToConsole();
}
else
{
    throw new NullReferenceException();
}

One of the annoying things about the standard .NET usage of the NullReferenceException is that it’s error message does not contain any hints about which object was null. Let’s take the opportunity to make it little better. Let’s change our DoIfNotNull method implementation:

public static class FlowControlExtensions
{
    [DebuggerStepThrough]
    public static void DoIfNotNull<T>(this T obj,
        Action<T> action, bool doContinue = true) where T : class
    {
        if (obj != null)
        {
            action(obj);
        }
        if (doContinue) return;
        throw new NullReferenceException(
            string.Format("Reference of type {0} was null.",
                typeof (T).FullName));
    }
}

We have added a default parameter to the method signature. This acts as a flag to set whether or not to continue the execution if null occurs, or throw an exception. Also, we take care to construct the error message so that it gives us the type of the reference being null, which would help us a bit investigating the error. Also note that I added the DebuggerStepThroughAttribute which means that the debugger will not step into this method, so that it will actually stop at the same code line as it would before we introduced our extension method. In the situation that we wish to throw an exception if the reference is null, the example now looks like so:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
person.Address.DoIfNotNull(a => a.WriteToConsole(), doContinue : false);

This code will now throw an exception with the message “Reference of type Address was null.”

So, now we have the possibility to run any code snippet that fits in an Action<T> if the reference is not null. Let’s try to handle the situation where we wish to evaluate an expression instead. Consider the following example:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
Console.WriteLine(person.Address.ToString());

The last line in this example will throw a NullReferenceException if the Address property is not set. Again, we will try to handle this situation using an extension method:

public static class FlowControlExtensions
{
    [DebuggerStepThrough]
    public static TResult IfNotNull<T,TResult>(this T obj, 
        Func<T, TResult> func) where T : class
    {
        if (obj != null)
        {
            return func(obj);
        }
        return default(TResult);
    }
}

Notice that if the object reference in fact is null, we return default(TResult) from the extension method. Now, we achieve to make our code run even though we have a null reference:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
Console.WriteLine(person.Address.IfNotNull(a => a.ToString()));

One improvement we might want to do here, is to make our little program make it explicit if the address is null. Right now, it will only print a blank line. Let’s make a small change to our extension method:

public static class FlowControlExtensions
{
    [DebuggerStepThrough]
    public static TResult IfNotNull<T,TResult>(this T obj,
        Func<T, TResult> func, 
        TResult defaultValue = default(TResult)) where T : class
    {
        if (obj != null)
        {
            return func(obj);
        }
        return defaultValue;
    }
}        

Here we introduced a named parameter which by default is default(TResult). If we wish, we can now override this to give the value returned by the extension method in case of null:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
Console.WriteLine(person.Address.IfNotNull(a => a.ToString(), defaultValue : "(unknown)"));

Before we call it a day, let’s handle one final scenario. Say that we actually want to sustain the default behavior in case of null, throw a NullReferenceException. This is trivial to do, but let’s take the opportunity to give a more helpful error message than the one we get by default. For this, we will put expression trees to work.

Instead of having our extension method take a Func<T,TResult> parameter, we will make it an Expression<Func<T,Result>>. What we achieve with this is to be able to traverese the expression tree to look for the property or method on the reference being null that we were trying to call. This will give us helpful hints about which object reference was actually null. Also, let’s introduce a parameter that enables us to say that we wish to throw an exception in case of an error. A simple implementation looks like this:

public static class FlowControlExtensions
{
    [DebuggerStepThrough]
    public static TResult IfNotNull<T,TResult>(this T obj, 
        Expression<Func<T, TResult>> func, 
        TResult defaultValue = default(TResult), 
        bool doContinue = true) where T : class
    {
        if (obj != null)
        {
            return func.Compile()(obj);
        }
        if (doContinue) return defaultValue;
        throw new NullReferenceException(
            string.Format("Reference of type {0} was null.",
                typeof (T).FullName));
    }
}

We can now use this method to throw an exception in case of null:

var person = new Person { Name = "Cosmo Kramer" };
Console.WriteLine("Person: {0}", person.Name);
Console.WriteLine("Address:");
Console.WriteLine(person.Address.IfNotNull(a => a.ToString(), doContinue: false));

If the Address property is null, it will throw a NullReferenceException with the message ‘Reference of type Address was null.’.

Handling null checks efficiently-2

Let’s try to see if we can improve the error message to give us a little more information. First, let’s introduce a private class which will traverse the expression tree to look for the method or property referenced:

public static class FlowControlExtensions
{
    private class MemberNameCollector : ExpressionVisitor
    {
        public string MemberName = "Unknown";
        private readonly string _parameterName;
        public MemberNameCollector(string parameterName)
        {
            _parameterName = parameterName;
        }
        protected override Expression VisitMethodCall(MethodCallExpression node)
        {
            if (node.Object != null 
                && node.Object.NodeType == ExpressionType.Parameter
                && _parameterName == node.Object.ToString())
            {
                MemberName = node.Method.Name + "(...)";
            }
            return base.VisitMethodCall(node);
        }
        protected override Expression VisitMember(MemberExpression node)
        {
            if (node.NodeType == ExpressionType.MemberAccess 
                && node.Expression.ToString() == _parameterName)
            {
                MemberName = node.Member.Name;
            }
            return base.VisitMember(node);
        }
    }
}

We can now use this class to alter the implementation of our IfNotNull extension method:

public static class FlowControlExtensions
{
    [DebuggerStepThrough]
    public static TResult IfNotNull<T,TResult>(this T obj,
        Expression<Func<T, TResult>> func, 
        TResult defaultValue = default(TResult), 
        bool doContinue = true) where T : class
    {
        if (obj != null)
        {
            return func.Compile()(obj);
        }
        if (doContinue) return defaultValue;
        var parameterName = func.Parameters.First().Name;
        var parameterType = typeof (T).FullName;
        var visitor = new MemberNameCollector(parameterName);
        visitor.Visit(func.Body);
        throw new NullReferenceException(
            string.Format("Tried to reference .{0} on parameter with name '{1}' of type {2}, but it was  null",
            visitor.MemberName, parameterName, parameterType));
    } 
}

We now get a little more information in our NullReferenceException message stating “Tried to reference .ToString(…) on parameter with name ‘a’ of type Address, but it was null”. A little more helpful…

Handling null checks efficiently-3

Summary

We have used extension methods to handle several null handling situations in a terse way. Here’s what we have covered:

person.Address.DoIfNotNull(a => a.WriteToConsole()); Runs a statement. Continue execution in case of null.
person.Address.DoIfNotNull(a => a.WriteToConsole(), doContinue : false); Runs a statement. Throws a NullReferenceException in case of null. Somewhat improved exception message compared to OOTB.
person.Address.IfNotNull(a => a.ToString()); Evaluates an expression. Returns default(T) in case of null.
person.Address.IfNotNull(a => a.ToString(), defaultValue : "(unknown)"); Evaluates an expression. Returns "(unknown)" in case of null.
person.Address.IfNotNull(a => a.ToString(), doContinue: false); Evaluates an expression. Throws a NullReferenceException in case of null. Improved exception message compared to OOTB.

If you want to try this out yourself, it is quite straightforward to copy and paste the code. You can also find the example code on GitHub.

NLog: writing log entries to Azure Table Storage

In August last year, I blogged about how to get Log4Net log entries written to Azure Table Storage. In this article, I will show how the same thing can be easily achieved using NLog.

The concepts in NLog is very similar to Log4Net. More or less, replace the word “appender” in Log4Net lingo with “target”, and you’re game.

First, let’s create a class for log entries:

public class LogEntry : TableServiceEntity
{
    public LogEntry()
    {
        var now = DateTime.UtcNow;
        PartitionKey = string.Format("{0:yyyy-MM}", now);
        RowKey = string.Format("{0:dd HH:mm:ss.fff}-{1}", now, Guid.NewGuid());
    }
    #region Table columns
    public string Message { get; set; }
    public string Level { get; set; }
    public string LoggerName { get; set; }
    public string RoleInstance { get; set; }
    public string DeploymentId { get; set; }
    public string StackTrace { get; set; }
    #endregion
}

Next, we need to do is to create a class that represents the table storage service. It needs to inherit from TableServiceContext:

public class LogServiceContext : TableServiceContext
{
    public LogServiceContext(string baseAddress, StorageCredentials credentials) : base(baseAddress, credentials) { }
    internal void Log(LogEntry logEntry)
    {
        AddObject("LogEntries", logEntry);
        SaveChanges();
    }
    public IQueryable<LogEntry> LogEntries
    {
        get
        {
            return CreateQuery<LogEntry>("LogEntries");
        }
    }
}

Finally, as far as code is concerned, a class that is a custom NLog target that gets called when the NLog framework needs to log something:

[Target("AzureStorage")]
public class AzureStorageTarget : Target
{
    private LogServiceContext _ctx;
    private string _tableEndpoint;
    [Required]
    public string TableStorageConnectionStringName { get; set; }
    protected override void InitializeTarget()
    {
        base.InitializeTarget();
        var cloudStorageAccount =
            CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue(TableStorageConnectionStringName));
        _tableEndpoint = cloudStorageAccount.TableEndpoint.AbsoluteUri;
        CloudTableClient.CreateTablesFromModel(typeof(LogServiceContext), _tableEndpoint, cloudStorageAccount.Credentials);
        _ctx = new LogServiceContext(cloudStorageAccount.TableEndpoint.AbsoluteUri, cloudStorageAccount.Credentials);
    }
    protected override void Write(LogEventInfo loggingEvent)
    {
        Action doWriteToLog = () =>
        {
            try
            {
                _ctx.Log(new LogEntry
                {
                    RoleInstance = RoleEnvironment.CurrentRoleInstance.Id,
                    DeploymentId = RoleEnvironment.DeploymentId,
                    Timestamp = loggingEvent.TimeStamp,
                    Message = loggingEvent.FormattedMessage,
                    Level = loggingEvent.Level.Name,
                    LoggerName = loggingEvent.LoggerName,
                    StackTrace = loggingEvent.StackTrace != null ? loggingEvent.StackTrace.ToString() : null
                });
            }
            catch (DataServiceRequestException e)
            {
                InternalLogger.Error(string.Format("{0}: Could not write log entry to {1}: {2}",
                    GetType().AssemblyQualifiedName, _tableEndpoint, e.Message), e);
            }
        };
        doWriteToLog.BeginInvoke(null, null);
    }
}

So, to make it work, we need to register the target with the NLog framework. This is done in the NLog.config file:

<?xml version="1.0"?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <extensions>
    <add assembly="Demo.NLog.Azure" />
  </extensions>
  <targets>
    <target name="azure" type="AzureStorage" tableStorageConnectionStringName="Log4Net.ConenctionString" />
  </targets>
  <rules>
    <logger name="*" minlevel="Info" writeTo="azure" />
  </rules>
</nlog>

For information about how to set your ServiceDefinition.csdef and ServiceConfiguration.cscfg files, see my previous post.You can find the code for this example on GitHub. Suggestions for improvement are very welcome.

Getting user information programmatically from Windows Identity Foundation

You can use the Windows Identity Foundation SDK to replace the authentication scheme of your ASP.NET web application. Most notably, this is useful for making your application claims-aware, which allows it to seamlessly play together with solutions like Active Directory Federation Services or the Windows Azure AppFabric Access Service. This is useful for a various number of SSO and federated authentication scenarios.

Basically, what you do is that you switch off the built-in authentication in ASP.NET like forms-based authentication, and let WIF act as a proxy in front of your application. WIF uses the authorization settings in web.config (/configuration/system.web/authorization and /configuration/location/system.web/authorization elements) and authenticates the user before the ASP.NET application receives the request. See WS-Federated Authentication Module Overview for details

So, when the application receives the request, the user is already authenticated, which is fine. However, there are times when the application needs to know who the user is, or getting access to the other claims that were provided from the identity service. Luckily, this is available on the HTTP context. Say, for instance, if you wish to find the email address of the logged-in user, you can do it like so:

protected void Page_Load(object sender, EventArgs e)
{
    var claimsIdentity = Context.User.Identity as IClaimsIdentity;
    foreach (var claim in claimsIdentity.Claims)
    {
        if (claim.ClaimType == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress")
        {
            var email = claim.Value;
        }
    }
}

If the user was authenticated using claims, Context.User.Identity will be an IClaimsIdentity which will contain a number of claims about the user. We can then iterate over these claims to find the one that we want.

The claim types are denoted using XML namespaces, which are a little bit cumbersome to work with. So, to make it easier to access, we iterate over the list of claims, and make all the claims available on a common object:

public static class IdentityExtension
{
    public static DynamicUser GetUserFromClaims(this HttpContext context)
    {
        var claimsIdentity = context.User.Identity as IClaimsIdentity;
        if (claimsIdentity == null) throw new FailedAuthenticationFaultException();
        return new DynamicUser(claimsIdentity.Claims);
    }
}
public class DynamicUser : DynamicObject
{
    private readonly ClaimCollection _claims;
    public DynamicUser(ClaimCollection claims)
    {
        _claims = claims;
        Id = ClaimsValue("nameidentifier")[0];
    }
    public string Id { get; private set; }
    public bool IsInRole(string role)
    {
        return ClaimsValue("role").Any(c => c.Equals(role, StringComparison.OrdinalIgnoreCase));
    }
    private string[] ClaimsValue(string claimSuffix)
    {
        return _claims.Where(c => c.ClaimType.EndsWith(claimSuffix)).Select(c => c.Value).ToArray();
    }
    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        var claimValue = ClaimsValue(binder.Name.ToLowerInvariant());
        if (claimValue.Length == 0)
        {
            return base.TryGetMember(binder, out result);
        }
        if (claimValue.Length == 1)
        {
            result = claimValue[0];
        }
        else
        {
            result = claimValue;
        }
        return true;
    }
}

Then, we can access the claim values using a prefix of the actual XML namespace of the claim type:

protected void Page_Load(object sender, EventArgs e)
{
    var user = Context.GetUserFromClaims();
    var id = user.Id;
    var username = user.Name; // "John Doe"
}

When hooking up the application to Windows Azure AppFabric Access Control Service (ACS), which claims that the application will receive depends on your rule group configuration. Here are some examples:

Claim type Shorthand Example value
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/nameidentifier user.Id NHsTR1UXFI9xYk1xIRUNfucZ4/a5OWiILHlNyNEXozP=
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name user.Name John Doe
http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress user.Emailaddress john.doe@hotmail.com

If you set up role claim rules in ACS, you can even get information about the user’s roles so that you can do authorization in the code:

protected void Page_Load(object sender, EventArgs e)
{
    var user = Context.GetUserFromClaims();
    var isAdmin = user.IsInRole("administrator"):
}

Log4Net: writing log entries to Azure Table Storage

Earlier, I blogged about how one can leverage Azure Diagnostics to easily set up Log4Net for your application running in Azure, and how to customize the log entries for the Azure environment.

An alternative to doing this two-step process of first writing to the local disk and then transfer the logs to Azure blob storage, is to write the log entries directly to Azure table storage (or in principal, to Azure blob storage for that matter). This is what I will do here.

Each log entry that the application writes will be a single row in a table in the Azure Table Storage. The log message itself and various meta data about it will be inserted into separate columns in the table. In order to achieve this, we first create a class that represents each entry in the table:

public class LogEntry : TableServiceEntity
{
    public LogEntry()
    {
        var now = DateTime.UtcNow;
        PartitionKey = string.Format("{0:yyyy-MM}", now);
        RowKey = string.Format("{0:dd HH:mm:ss.fff}-{1}", now, Guid.NewGuid());
    }
    #region Table columns
    public string Message { get; set; }
    public string Level { get; set; }
    public string LoggerName { get; set; }
    public string Domain { get; set; }
    public string ThreadName { get; set; }
    public string Identity { get; set; }
    public string RoleInstance { get; set; }
    public string DeploymentId { get; set; }
    #endregion
}

Note that the PartitionKey is the current year and month, and the RowKey is a combination of the date, time and a GUID. This is done to make the querying of the log entries efficient for our purpose. So, the next thing we need to do is to create a class that represents the table storage service. It needs to inherit from TableServiceContext:

internal class LogServiceContext : TableServiceContext
{
    public LogServiceContext(string baseAddress, StorageCredentials credentials) : base(baseAddress, credentials) {}
    internal void Log(LogEntry logEntry)
    {
        AddObject("LogEntries", logEntry);
        SaveChanges();
    }
    public IQueryable<LogEntry> LogEntries
    {
        get
        {
            return CreateQuery<LogEntry>("LogEntries");
        }
    }
}

The method that we will actually use in our code is the Log method which takes a LogEntry instance and persists it to table storage. What we need next, is to create a new Appender for Log4Net which interacts with the table store to store the log entries:

public class AzureTableStorageAppender : AppenderSkeleton
{
    public string TableStorageConnectionStringName { get; set; }
    private LogServiceContext _ctx;
    private string _tableEndpoint;
    public override void ActivateOptions()
    {
        base.ActivateOptions();
        var cloudStorageAccount =
            CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue(TableStorageConnectionStringName));
        _tableEndpoint = cloudStorageAccount.TableEndpoint.AbsoluteUri;
        CloudTableClient.CreateTablesFromModel(typeof(LogServiceContext), _tableEndpoint, cloudStorageAccount.Credentials);
        _ctx = new LogServiceContext(cloudStorageAccount.TableEndpoint.AbsoluteUri, cloudStorageAccount.Credentials);
    }
    protected override void Append(LoggingEvent loggingEvent)
    {
        Action doWriteToLog = () => {
            try
            {
                _ctx.Log(new LogEntry
                {
                    RoleInstance = RoleEnvironment.CurrentRoleInstance.Id,
                    DeploymentId = RoleEnvironment.DeploymentId,
                    Timestamp = loggingEvent.TimeStamp,
                    Message = loggingEvent.RenderedMessage,
                    Level = loggingEvent.Level.Name,
                    LoggerName = loggingEvent.LoggerName,
                    Domain = loggingEvent.Domain,
                    ThreadName = loggingEvent.ThreadName,
                    Identity = loggingEvent.Identity
                });
            }
            catch (DataServiceRequestException e)
            {
                ErrorHandler.Error(string.Format("{0}: Could not write log entry to {1}: {2}",
                    GetType().AssemblyQualifiedName, _tableEndpoint, e.Message));
            }
        };
        doWriteToLog.BeginInvoke(null, null);
    }
}

In the code above, the actually writing to the log is done asynchronically in order to prevent the log write to slow down the request handling. We are now done with all the coding. What is left is to use our new AzureTableStorageAppender. Here is the log4net.config:

<log4net>
  <appender name="AzureTableStoreAppender" type="Demo.Log4Net.Azure.AzureTableStorageAppender, Demo.Log4Net.Azure">
    <param name="TableStorageConnectionStringName" value="Log4Net.ConenctionString" />
  </appender>
  <root>
    <level value="DEBUG" />
    <appender-ref ref="AzureTableStoreAppender" />
  </root>
</log4net>

Notice the TableSTorageConnectionStringName attribute of the param element in the configuration. It corresponds to the property of the same name in the AzureTableStorageAppender. Furthermore, take take notice that it’s value is 'Log4Net.ConnectionString', which corresponds to a custom configuration setting that we will add to ServiceDefinition.csdef file:

<ServiceDefinition ...>
  <WebRole ...>
    <ConfigurationSettings>
      <Setting name="Log4Net.ConenctionString"/>
    </ConfigurationSettings>
    ...
  </WebRole>
</ServiceDefinition>

We also need to give the Log4Net.ConfigurationString setting a value in the ServiceConfiguration.cscfg file. It should be a connection string that points to the storage account to use for storing the log entries. In this example, let’s use the development storage:

<ServiceConfiguration ...>
  <Role ...>
    <ConfigurationSettings>
      <Setting name="Log4Net.ConenctionString" value="UseDevelopmentStorage=true"/>
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

…and that’s it. You should now find the log entries in the table storage:

You can find the code for this example on GitHub. Suggestions for improvement are very welcome.

Customizing Log4net log entries on Azure

I have earlier blogged about how to use Log4Net on Azure compute. With this solution the log files from the various running instances gets transferred to a common container on Azure blob store. When I mine the log data, I usually merge all the log files together an run text utils like grep or sed on them.

One challenge when merging the log files together is that we then lose the information about which instance the different log entries came from. In order to fix this, we can customize the log entries to that we keep this information.

The first we need to do, is to create a new layout class that inherits from PatternLayout:

using log4net.Layout;
namespace Demo.Log4Net.Azure
{
    public class AzurePatternLayout : PatternLayout
    {
        public AzurePatternLayout()
        {
             // TODO: add converters
        }
    }
}

Next, we need to register this class in the log4net configuration:

<log4net>
  <appender ...>
    <layout type="Demo.Log4Net.Azure.AzurePatternLayout, Demo.Log4Net.Azure">
      <conversionPattern ... />
    </layout>
  </appender>
  ...
</log4net>

So now that we have a new PatternLayout class, we can add some logic into it for adding Azure-specific log information into the entries. To do so, we first need a new Converter class:

using System.IO;
using log4net.Util;
using Microsoft.WindowsAzure.ServiceRuntime;
namespace Demo.Log4Net.Azure
{
    internal class AzureInstanceIdPatternConverter : PatternConverter
    {
        protected override void Convert(TextWriter writer, object state)
        {
            writer.Write(RoleEnvironment.CurrentRoleInstance.Id);
        }
    }
}

Now, we register the new AzureInstanceIdPatternConverter in the constructor of AzurePatternLayout:

public class AzurePatternLayout : PatternLayout
{
    public AzurePatternLayout()
    {
        AddConverter("roleinstance", typeof(AzureInstanceIdPatternConverter));
    }
}

Then we can change the conversionPattern element of the Log4Net configuration to use the new Azure environment information:

<layout type="Demo.Log4Net.Azure.AzurePatternLayout, Demo.Log4Net.Azure">
  <conversionPattern value="%date [%roleinstance] [%thread] %-5level %logger [%appdomain] - %message%newline" />
</layout>

….which will make the log entries look something like this:

Custom log entries

(The screenshot shows instance ids generated by DevFabric, not an instance in the cloud)

Using Log4Net in Azure Compute

Log4Net is a popular logging framework, and if you have an existing application that you wish to move to Azure compute, you probably want to avoid rewriting your application to use another logging framework. Luckily, keeping Log4Net as your logging tool in Azure is certainly possible, but there are a few hoops you have to jump through to get there.

There are several ways to achieve this goal. I decided to rely as much as possible on a feature provided in Azure Compute that allows for automatically synchronizing certain directories on the instance’s local file system to Azure blob storage. Using this approach, there are only very few changes that need to be done in the application, and indeed none of the existing code needs to be altered.

Baseline: an existing application which uses Log4Net

In order for this example to work, we need an application that we what to move to the cloud:

  1. Start off with File->New->Project... in Visual Studio and use the “ASP.NET Web Application” template
  2. Add Log4Net capabilities to the application. This can be done by adding a reference to Log4Net using Nuget, and then configuring it like Phil Haack has described here.

Then, run the web application locally in Visual Studio to assert that the logging works.

Enabling the application for Azure

Starting off with the simple ASP.NET web application we created in the previous section, do the following:

  1. Right-click on the solution in Visual Studio to select Add->New Project. Use the Windows Azure Project template, and do not add any roles in the dialog box initially.
  2. Set the newly created cloud project as the startup project in the solution.
  3. Right-click on the Roles-folder of the newly created Azure project and select Add->Web role project in solution... to add the web application project as an Azure Web role.
Adding an Azure Compute web role to an existing ASP.NET solution

Now, press F5 to run the application in the local Azure development environment (DevFabric) to see that it works (functionally, not logging-wise)

So, we are done with the prerequisites. Now to the interesting parts!

Setting log directory for Azure

The first issue we will grapple with is the fact that in Azure Compute, the application effectively runs in a sandbox with limited access to the file system, which means that the “standard” approach logging to a file does not work. Basically, the Azure compute role has only access to a certain subdirectory of the file system and the exact location needs to be retrieved by the application at runtime.

In order to retain the existing logging in the application, locating the path to the role’s designated area on the disk can be solved by subclassing one of the appenders that Log4Net provides out of the box. I chose the RollingFileAppender because it provides the ability to split the log into several files. This is beneficial from an operations perspective. Here’s what the custom appender looks like:

using System.Diagnostics;
using System.IO;
using log4net.Appender;
using Microsoft.WindowsAzure.ServiceRuntime;
namespace Demo.Log4Net.Azure
{
    public class AzureAppender : RollingFileAppender
    {
        public override string File
        {
            set
            {
                base.File = RoleEnvironment.GetLocalResource("Log4Net").RootPath + @"" 
                    + new FileInfo(value).Name + "_"
                    + Process.GetCurrentProcess().ProcessName;
            }
        }
    }
}

What happens here, is that when the configuration is read when the logging framework initializes, it calls our method to set the log file name. This corresponds to the file element in the XML configuration for the appender:

<log4net>
  <appender>
    <param name="File" value="app.log" />
    ...
  </appender>
  ...
</log4net>

What happens here, is that the application asks the role environment for the whereabouts of the local resource called “Log4Net”. This resource is a directory that we specify designated for containing our logs and needs to be specified in the ServiceDefinition.csdef file:

<ServiceDefinition>
  <WebRole name="WebRole1">
    <LocalResources>
        <LocalStorage name="Log4Net" sizeInMB="2048" cleanOnRoleRecycle="true"/> 
    </LocalResources>
  </WebRole>
</ServiceDefinition>

When we have the path of the local resource, it is used to construct an absolute path for the log file. Also note that the current process name is appended to the filename. This is done because if you run the application as a WebRole in “Full IIS” mode in Azure, the web application and the RoleEntryPoint code run in different processes. (If you look at blog entries on the Internet for Azure information, you should have in mind that the “Full IIS” mode was introduced with the Azure SDK version 1.3 in late 2010, and information that predates this might not be valid for the current Azure version.) This means that if there are log entries in the RoleEntryPoint as well as in the rest of your application, two processes would potentially try to keep a write lock on the file at the same time. Therefore, we use one log file for each process. Note that this is not a relevant topic for Worker roles. For more on the execution model, take a look here.

So, now that the new custom appender is ready, we need to change the Log4Net configuration to use it. Basically, we change the assembly type in the appender configuration section so that the configuration looks like this:

<log4net>
  <appender name="AzureRollingLogFileAppender" type="Demo.Log4Net.Azure.AzureAppender, Demo.Log4Net.Azure">
    <param name="File" value="app.log" />
    <param name="AppendToFile" value="true" />
    <param name="RollingStyle" value="Date" />
    <param name="StaticLogFileName" value="false" />
    <param name="DatePattern" value=".yyyy-MM-dd.log" />
    <layout type="log4net.Layout.PatternLayout">
      <conversionPattern value="%date [%thread] %-5level %logger [%appdomain] - %message%newline" />
    </layout>
  </appender>
  <root>
    <level value="DEBUG" />
    <appender-ref ref="AzureRollingLogFileAppender" />
  </root>
</log4net>

Now it’s time to run the application to see if the logging works. First, deploy to devfabric, and then open the Windows Azure Compute Emulator. Right-click on the running instance, and click on Open local store....

Open the local store for a role instance running in DevFabric

Then navigate to the ‘directoryLog4Net‘ to find the log files:

Log files in local store in DevFabric

Persisting logs to Azure blob storage

The next issue we need to handle, is the fact that the local file system in an Azure role instance is not persistent. Local data will be lost when the application is redeployed (and also when the Role recycles, if you have chosen to do so). Furthermore, the only way to access the local file system is using a Remote Desktop Connection. In theory, you could probably also make the directory a shared drive accessible over the Internet, but you probably would not want to do that. Besides, it will be a headache if you have a lot of instances.

So, the solution that Azure offers to this, is to have a scheduled synchronization of certain of the local resources (directories) to the Azure blob store. What we need to do, is to add the following code to the descendant of RoleEntryPoint:

public class WebRole : RoleEntryPoint
{
    public override bool OnStart()
    {
        var diagnosticsConfig = DiagnosticMonitor.GetDefaultInitialConfiguration();
        diagnosticsConfig.Directories.ScheduledTransferPeriod = TimeSpan.FromMinutes(5);
        diagnosticsConfig.Directories.DataSources.Add(
                new DirectoryConfiguration
                {
                    Path = RoleEnvironment.GetLocalResource("Log4Net").RootPath,
                    DirectoryQuotaInMB = 2048,
                    Container = "wad-log4net"
                }
        );
        DiagnosticMonitor.Start("Microsoft.WindowsAzure.Plugins.Diagnostics.ConnectionString", diagnosticsConfig);
        return base.OnStart();
    }
}

…and that’s it. Now you can try to run the application and observe a container called ‘wad-log4net’ will be created in your blob service account that will contain the logs:

Logs in Azure blob store

(I use the AzureXplorer extension for Visual Studio)

The solution shown here targeted an ASP.NET application running as a WebRole, but the setup works equally well for Worker roles.

ASP.NET 3.5: improving testability with System.Web.Abstractions

The testability of ASP.NET code has long been a challenge; creating unit tests for your ASP.NET code has been difficult. One of the main points of the new ASP.NET MVC framework has been to make code written for it easily testable. However, not many people know that in ASP.NET 3.5, Microsoft has added a few features to make any ASP.NET applications, not only ASP.NET MVC applications, more easy to test. The System.Web.Abstractions assembly adds a few classes to the System.Web namespace that will help the situation. For instance, looking at the documentation for System.Web.HttpRequestBase, it states that

The HttpRequestBase class is an abstract class that contains the same members as the HttpRequest class. The HttpRequestBase class enables you to create derived classes that are like the HttpRequest class, but that you can customize and that work outside the ASP.NET pipeline. When you perform unit testing, you typically use a derived class to implement members that have customized behavior that fulfills the scenario that you are testing.

Very well. Looking at the documentation for HttpRequest, we see that HttpRequest is not a decendent of HttpRequestBase as one might expect from the name. The reason for this is probably that that would break backwards compatability with older versions of ASP.NET. So, how can we exploit the HttpRequestBase then? The answer is the HttpRequestWrapper class which is a decendant of HttpRequestBase and has a constructor that takes an HttpRequest object as a parameter. Then, we can take the HttpRequest object passed to our code from the framework, wrap it inside an HttpRequestWrapper object and pass it on to our code as a HttpRequestBase object. As I will show you in the examples below, this will enable us to create unit tests of our code by creating fake implementations of ASP.NET framework clases (using Rhino.Mocks).

Example #1: Testing a page codebehind file

Take, for instance, this simple page codebehind code that we would like to test:

using System;
using System.Web;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        Response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

The first step to take here, is to extract a method which take the request object as a parameter instead of fetching it from a method in a superclass. In general, this is a variation of the dependency injection pattern which in many situations will help us make our code testable (also, see my earlier related post). Like so:

using System.Web;
using System;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        SetCacheablityOfResponse(Response);
    }

    public void SetCacheablityOfResponse(HttpResponse response)
    {
        response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

So, then having extracted our code in a separate method, the next step is to change the parameter type of this method from HttpRequest to HttpRequestBase. Furthermore, when calling this method, we need to wrap the HttpRequest object by creating an instance of HttpRequestWrapper. The code, then, looks like this:

<

pre class=”c-sharp” name=”code””>using System.Web;
using System;

public partial class _Default : System.Web.UI.Page
{
protected void Page_Init(object sender, EventArgs e)
{
SetCacheablityOfResponse(new HttpResponseWrapper(Response));
}

public void SetCacheablityOfResponse(HttpResponseBase response)
{
    response.Cache.SetCacheability(HttpCacheability.NoCache);
}

}
Having now prepared our code for testing, we can create a unit test where we test the SetCacheabilityOfResponse method:

[TestMethod]
public void ShouldSetNoCacheabilityOnDefaultPage()
{
    _Default page = new _Default();
    MockRepository mocks = new MockRepository();
    HttpResponseBase responseStub = mocks.Stub<HttpResponseBase>();
    HttpCachePolicyBase cachePolicyMock = mocks.CreateMock<HttpCachePolicyBase>();
    With.Mocks(mocks).Expecting(delegate
    {
        SetupResult.For(responseStub.Cache).Return(cachePolicyMock);
        cachePolicyMock.SetCacheability(HttpCacheability.NoCache);
        LastCall.On(cachePolicyMock).Repeat.AtLeastOnce();
    }).Verify(delegate
    {
        page.SetCacheablityOfResponse(responseStub);
    });
}

If you are not familiar with Rhino.Mocks or any other mocking framework, there appears to be a lot going on in that test. The basic idea is that we create derivatives of the -Base classes and pass these to the code that we are going to test, mimicking the behavior of the “real” objects that the framework would pass our code at runtime. Also note that in this particular test we test the side effect of our code, namely that the code should call a the SetCacheability method with a specific parameter. This is achieved using a mock object.

Example #2: Testing an HTTP Handler

Given the following HTTP handler code:

using System;
using System.Web;

public class RedirectAuthenticatedUsersHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        if (context.Request.IsAuthenticated)
        {
            context.Server.TransferRequest("/farfaraway");
        }
    }
}

Again, we extract the code we want to test into a separate method, passing it a -Base object and wrap the object passed to us from the framework in a -Wrapper object:

using System;
using System.Web;

public class RedirectAuthenticatedUsersHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        TransferUserIfAuthenticated(new HttpContextWrapper(context));
    }

    public void TransferUserIfAuthenticated(HttpContextBase context)
    {
        if (context.Request.IsAuthenticated)
        {
            context.Server.TransferRequest("/farfaraway");
        }
    }
}

This allows us to create unit tests for the TransferUserIfAuthenticated method, for instance:

[TestMethod]
public void ShouldRedirectAuthenticatedUser()
{
    RedirectAuthenticatedUsersHandler handler = new RedirectAuthenticatedUsersHandler();
    MockRepository mocks = new MockRepository();
    HttpContextBase httpContextStub = mocks.Stub<HttpContextBase>();
    HttpRequestBase httpRequestBaseStub = mocks.Stub<HttpRequestBase>();
    HttpServerUtilityBase httpServerUtilityMock = mocks.CreateMock<HttpServerUtilityBase>();
    With.Mocks(mocks).Expecting(delegate
    {
        SetupResult.For(httpContextStub.Request).Return(httpRequestBaseStub);
        SetupResult.For(httpContextStub.Server).Return(httpServerUtilityMock);
        SetupResult.For(httpRequestBaseStub.IsAuthenticated).Return(true);
        httpServerUtilityMock.TransferRequest("/farfaraway");
        LastCall.On(httpServerUtilityMock).Repeat.AtLeastOnce();
    }).Verify(delegate
    {
        handler.TransferUserIfAuthenticated(httpContextStub);
    });
}

Summary

I have shown two very simple examples on how some of the classes in the System.Web.Abstractions assembly can help us test our ASP.NET code. I have used HttpResponseBase, HttpServerUtilityBase, HttpContextBase, HttpRequestBase, and HttpCachePolicyBase. Note that there are a number of classes available, so if you are faced with not being able to test your ASP.NET code because of dependencies to classes in the framework, take a look in the System.Web namespace and see if there are -Base classes that can help you out.

Testability is a large topic, and there is much to be said about it. I have shown a couple of very simple examples on how to improve testability. Testability has a lot to do with code design as well; in a real world I would write the test before writing the code and I would move my code out of codebehinds. Those topics are discussed a lot elsewere, hopefully this post will bring you a small step further in writing testable code.

Spring.NET: programatically add objects to the existing (XML) application context

My experience is that Spring.NET configuration files tend to grow very large. As far as I can figure, there are two principal problems that arise from this:

  1. The configuration files get difficult to read and maintain
  2. It gets easier to introduce errors in the configuration because of its size

In general, I am in favour of keeping configuration files as small as possible. I often work with web applications that can (quite) easily be redeployed to the production environment, hence I always ask the question “will this value ever change between environments or deployments” when considering introducing a new configuration part.

Now, the Spring XML configuration usually serves two main purposes; to wire together the application, and to provide values that should be possible to change between deployments of the application or for different environments. The first purpose, I would argue does not necessarily need to be in the XML configuration. Rather, if this is done in code, we get the benefit that the compiler will tell us right away if there are typos or missing references. If this wiring is in the XML configuration file, such errors will not surface until the application starts.

So, the question that I had, was how Spring context wiring could be combined in code and in XML. I found one way of doing it, but it is only applicable to singleton objects.

Say, for instance that we have an object “something” that we wish to have configured in XML:

  <object id="something" type="SpringTest.Something, SpringTest" singleton="false"/>

Then, we have a class that we want to initialize in code:

class Foo
{
    public Foo() { }
    private Something _s;
    Something S
    {
        set { _s = value; }
        get { return _s; }
    }
}

Now, we see that Foo has a dependency on Something; it needs an instance of Something to be injected. We can use the Spring context to do this after we have created the instance of Foo:

IApplicationContext context = ContextRegistry.GetContext();
Foo f = new Foo();
context.ConfigureObject(f, "fooPrototype");

But Spring does not yet know that the Foo instance needs to be injected Something. Hence, we need to tell Spring that by creating what I would call a “prototype” or “template” object configuration:

<object id="fooPrototype" type="ContextTestProject.Foo, ContextTestProject">
   <property name="S" ref="something"></property>
</object>

The final step is then to register our newly created object in the Spring context:

XmlApplicationContext xmlContext = context as XmlApplicationContext;
xmlContext.ObjectFactory.RegisterSingleton("foo", f);

After this, the Foo instance is available for the application in the Spring context.

Internet Explorer automation/Watin: catching navigation error codes

One question have troubled me for some time when automating Internet Explorer (actually, I am doing web testing with Watin): how to test for HTTP status codes. Finally, I figured out how to do this. The lies in an event that the InternetExplorer object raises when navigation is unsuccessful.

I ended up with writing a C# helper class:

using System.Net;
using WatiN.Core;
using SHDocVw;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Globalization;
namespace Test
{
    public class NavigationObserver
    {
        private HttpStatusCode _statusCode;
        public NavigationObserver(IE ie)
        {
            InternetExplorer internetExplorer = (InternetExplorer)ie.InternetExplorer;
            internetExplorer.NavigateError += new DWebBrowserEvents2_NavigateErrorEventHandler(IeNavigateError);
        }
        public void ShouldHave(HttpStatusCode expectedStatusCode)
        {
            if (!_statusCode.Equals(expectedStatusCode))
            {
                Assert.Fail(string.Format(CultureInfo.InvariantCulture, "Wrong status code. Expected {0}, but was {1}",
                    expectedStatusCode, _statusCode));
            }
        }
        private void IeNavigateError(object pDisp, ref object URL, ref object Frame, ref object StatusCode, ref bool Cancel)
        {
            _statusCode = (HttpStatusCode)StatusCode;
        }
    }
}

Note that I use Visual Studio test runner to run my web tests. Then, I can use this in my test:

using (IE ie = new IE())
{
    NavigationObserver observer = new NavigationObserver(ie);
    ie.GoTo("http://some.where.com");
    observer.ShouldHave(HttpStatusCode.NotFound);
}