Vidar’s Musings

In my previous post, I talked about how we could use System.Xml.Serialization.XmlSerializer and System.Runtime.Serialization.DataContractSerializer to parse XML into an object tree. I pointed out that DataContractSerializer in my mind has some advantages in that types, properties, and members to use with the deserialization do not need to be public. On the downside, DataContractSerializer puts a quite limiting constraint on the XML format in that it cannot parse XML attributes. This is as far as I know, an absolute constraint that cannot easily be circumvented. Thus, regrettably, if we are not in control of the XML format, DataContractSerializer is sometimes useless.

In those situations, we can still use XmlSerializer. In order to achieve the same encapsulation with XmlSerializer, we have to adjust our model a bit. Here’s one suggestion on how to do this:

My approach to this is inspired by Josh Bloch’s Builder pattern. The idea is changing the classes used for deserialization from being domain objects to being builder objects that build domain objects. This has another advantage in that our domain objects are not “polluted” with attributes and interfaces related to deserialization and are 100% plain old CLR objects (POCOs). So, lets first do a change our “deserialization” class (formerly ‘Country’) to a builder object, like so:

public class CountryBuilder
{
   [XmlElement(ElementName = "name")]
   public string Name;

   [XmlElement(ElementName = "iso-3166-alpha-2-code")]
   public string Code;

   public Country Build()
   {
      return new Country(Name, Code);
   }
}

Note here that our builder object has a Build() method which returns the domain object ‘Country’. This is the object that we will pass on to our clients. The Country class now represents our domain object:

public class Country
{
   private readonly string _name, _code;

   internal Country(string name, string code)
   {
      _name = name;
      _code = code;
   }

   public string Name { get { return _name; } }
   public string Code { get { return _code; } }
}

We have now restricted access to the creation of Country objects in that the constructor is internal, and it is also immutable (cannot change state once created) though making its fields readonly. It only exposes getters for its internal state. We can then do the same to our list of countries. The builder object for countries would look like this:

[XmlRoot("countries")]
public class Countries
{
   [XmlElement(ElementName="country")]
   public CountryBuilder[] countries;

   public IEnumerable<Country> Build()
   {
      return countries.Select<CountryBuilder , Country>(x => x.Build());
   }
}

The code for doing the serialization will then look like this:

string xml = ...;
XmlSerializer xmlSerializer = new XmlSerializer(typeof(CountriesBuilder));
var builder = xmlSerializer.Deserialize(new StringReader(inputXml)) as CountriesBuilder;
IEnumerable<Country> cs = builder.Build();

What we have achieved now is that we now can control the accessibility and encapsulation of our domain model. The builder objects are still public to anyone, but that really does not matter much in my mind. Thus, a relatively swift parsing of XML of various formats into a well designed object model.

If you want to parse XML in .NET, you have a lot of options to choose from. You can use XmlDocument to parse the XML into a DOM tree, you can use the XmlReader to write an efficient “pull” parser, or you can leverage some of the features provided with various serialization APIs.

Given the case where you have a fairly straightforward XML document (not too deep document tree, not too complex set of attributes and elements) that maps pretty well to your domain model, the serialization options is in my mind a good choice that requires little coding. Compared with this approach, using XmlDocument seems to be a bit of an overkill if you don’t need advanced traversal of the document, and writing a parser by hand using XmlReader seems to require quite a bit of coding.

So, given the following sample XML document, I will investigate the serialization options:

<countries>
   <country>
      <iso-3166-alpha-2-code>AF</iso-3166-alpha-2-code>
      <name>Afghanistan</name>
   </country>
   <country>
      <iso-3166-alpha-2-code>AX</iso-3166-alpha-2-code>
      <name>Åland Islands</name>
   </country>
   <country>
      <iso-3166-alpha-2-code>AL</iso-3166-alpha-2-code>
      <name>Albania</name>
   </country>
</countries>

Using System.Xml.XmlSerializer

The first option that came to mind, was to use the XmlSerializer object to deserialize the XML into C# (or VB for that matter) objects. It first requires that I annotate my object model in order to tell the serializer how to deserialize the XML:

[XmlRoot("countries")]
public class Countries
{
   [XmlElement(ElementName="country")]
   public Country[] countries;
}

public class Country
{
   [XmlElement(ElementName = "name")]
   public string Name;

   [XmlElement(ElementName = "iso-3166-alpha-2-code")]
   public string Code;
}

Then, I can use the serializer to deserialize the code:

string xml = ...;

XmlSerializer xmlSerializer = new XmlSerializer(typeof(Countries));
Countries c = xmlSerializer.Deserialize(new StringReader(xml)) as Countries;

Pretty sweet,  heh? Definitely. However, this has some drawbacks. If I want my Country class to be a well designed domain object that follows good OO design principles, I probably would like to encapsulate my data. Furthermore, I might want to restrict the creation of such objects from other parts of the code. In order for XmlSerializer to create my object, it requires that my types are public and that all properties or fields to set are public as well. What to do if I want to enforce my objects to be immutable once handed off to other parts of the code?

Using System.Runtime.Serialization.DataContractSerializer

Luckily, the serialization API that come with Windows Communication Framework has some neat features that fit like a glove. When defining my data model, it does not require that the types, neither the properties nor fields to set are public. Actually, I can restrict access to the type, its default constructor, and any of the properties or fields that I want to be deserialized! w00t!

So, this is what the Country class will looks like:

[DataContract(Name="country", Namespace="")]
internal class Country : IExtensibleDataObject
{
   private Country() { }

   [DataMember(Name="name")]
   public string Name { get; private set; }

   [DataMember(Name = "iso-3166-alpha-2-code")]
   public string Code { get; private set; }

   public ExtensionDataObject ExtensionData { get; set; }
}

The XML file contains a list of countries, and luckily, we have the CollectionDataContractAttribute to denote an element that is a list of elements. It supports generics, so that we can define our class as a strongly typed list:

[CollectionDataContract(Name="countries", Namespace="")]
internal class Countries : List<Country>, IExtensibleDataObject
{
   public ExtensionDataObject ExtensionData { get; set; }
}

And that’s it. Now we can deserialize:

string xml = ...;

DataContractSerializer ser = new DataContractSerializer(typeof(Countries));
using (StringReader stringReader = new StringReader(xml))
{
   using (XmlReader xmlReader = XmlReader.Create(stringReader))
   {
      Countries countries = (Countries)ser.ReadObject(xmlReader);
   }
}

Alternatively, our result could be typed as a list of countries:

IList<Country> countries = (IList<Country>)ser.ReadObject(xmlReader);

Note that there is a limitation in the latter method in that deserializing XML attributes is not supported. Thus, an XML document like the following would not work:

   <country iso-3166-alpha-2-code="AF">
      <name>Afghanistan</name>
   </country>
   <country iso-3166-alpha-2-code="AX">
      <name>Åland Islands</name>
   </country>
   <country iso-3166-alpha-2-code="AL">
      <name>Albania</name>
   </country>
</countries>

This will, however, work using the XmlSerializer.

As a follow-up to my previous post ASP.NET 3.5: improving testability with System.Web.Abstractions, I would like to show how the same testability can be achieved without using any mock framework like Rhino.Mocks. The C# 3.0 featuires ‘object initializers’ and ‘automatic properties’ makes our code sufficiently non-verbose to make it easy and readable.

So, given the same examples as in my previous post, here is what the test code will look like:

Example #1: Testing a page codebehind file

[TestMethod]
public void ShouldSetNoCacheabilityOnDefaultPage()
{
    _Default page = new _Default();
    HttpCachePolicyMock httpCachePolicyMock = new HttpCachePolicyMock();
    page.SetCacheablityOfResponse(new HttpResponseStub
    {
        TheCache = httpCachePolicyMock
    });
    httpCachePolicyMock.ShouldHaveSetCacheabilityTo(HttpCacheability.NoCache);
}

class HttpResponseStub : HttpResponseBase
{
    public override HttpCachePolicyBase Cache { get { return TheCache; } }
    public HttpCachePolicyBase TheCache { get; set; }
}

class HttpCachePolicyMock : HttpCachePolicyBase
{
    private HttpCacheability _cacheability;
    public override void SetCacheability(HttpCacheability cacheability)
    {
        _cacheability = cacheability;
    }
    public void ShouldHaveSetCacheabilityTo(HttpCacheability expectedCacheability)
    {
        Assert.AreEqual(expectedCacheability, _cacheability);
    }
}

I have created two helper classes, one with the suffix -Stub and one with the suffix -Mock. The convention here is that a stub is a type of class used to provide a context to the class under test. Mocks also do that, but additionally a mock can make expectation about what should happen to it during the test.

Example #2: Testing an HTTP handler

[TestMethod]
public void ShouldRedirectAuthenticatedUser()
{
    HttpServerUtilityMock httpServerUtilityMock = new HttpServerUtilityMock();
    HttpContextStub httpContextStub = new HttpContextStub
    {
        TheRequest = new HttpRequestStub { IsItAuthenticated = true },
        TheServer = httpServerUtilityMock
    };
    new RedirectAuthenticatedUsersHandler().TransferUserIfAuthenticated(httpContextStub);
    httpServerUtilityMock.ShouldHaveTransferredTo("/farfaraway");
}

class HttpContextStub : HttpContextBase
{
    public override HttpRequestBase Request { get { return TheRequest; } }
    public override HttpServerUtilityBase Server { get { return TheServer; } }
    public HttpRequestBase TheRequest { get; set; }
    public HttpServerUtilityBase TheServer { get; set; }
}

class HttpRequestStub : HttpRequestBase
{
    public override bool IsAuthenticated { get { return IsItAuthenticated; } }
    public bool IsItAuthenticated { get; set; }
}

class HttpServerUtilityMock : HttpServerUtilityBase
{
    private string _path;
    public override void TransferRequest(string path)
    {
        _path = path;
    }

    public void ShouldHaveTransferredTo(string expectedPath)
    {
        Assert.AreEqual(expectedPath, _path);
    }
}

The testability of ASP.NET code has long been a challenge; creating unit tests for your ASP.NET code has been difficult. One of the main points of the new ASP.NET MVC framework has been to make code written for it easily testable. However, not many people know that in ASP.NET 3.5, Microsoft has added a few features to make any ASP.NET applications, not only ASP.NET MVC applications, more easy to test. The System.Web.Abstractions assembly adds a few classes to the System.Web namespace that will help the situation. For instance, looking at the documentation for System.Web.HttpRequestBase, it states that

The HttpRequestBase class is an abstract class that contains the same members as the HttpRequest class. The HttpRequestBase class enables you to create derived classes that are like the HttpRequest class, but that you can customize and that work outside the ASP.NET pipeline. When you perform unit testing, you typically use a derived class to implement members that have customized behavior that fulfills the scenario that you are testing.

Very well. Looking at the documentation for HttpRequest, we see that HttpRequest is not a decendent of HttpRequestBase as one might expect from the name. The reason for this is probably that that would break backwards compatability with older versions of ASP.NET. So, how can we exploit the HttpRequestBase then? The answer is the HttpRequestWrapper class which is a decendant of HttpRequestBase and has a constructor that takes an HttpRequest object as a parameter. Then, we can take the HttpRequest object passed to our code from the framework, wrap it inside an HttpRequestWrapper object and pass it on to our code as a HttpRequestBase object. As I will show you in the examples below, this will enable us to create unit tests of our code by creating fake implementations of ASP.NET framework clases (using Rhino.Mocks).

Example #1: Testing a page codebehind file

Take, for instance, this simple page codebehind code that we would like to test:

using System;
using System.Web;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        Response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

The first step to take here, is to extract a method which take the request object as a parameter instead of fetching it from a method in a superclass. In general, this is a variation of the dependency injection pattern which in many situations will help us make our code testable (also, see my earlier related post). Like so:

using System.Web;
using System;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        SetCacheablityOfResponse(Response);
    }

    public void SetCacheablityOfResponse(HttpResponse response)
    {
        response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

So, then having extracted our code in a separate method, the next step is to change the parameter type of this method from HttpRequest to HttpRequestBase. Furthermore, when calling this method, we need to wrap the HttpRequest object by creating an instance of HttpRequestWrapper. The code, then, looks like this:

using System.Web;
using System;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        SetCacheablityOfResponse(new HttpResponseWrapper(Response));
    }

    public void SetCacheablityOfResponse(HttpResponseBase response)
    {
        response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

Having now prepared our code for testing, we can create a unit test where we test the SetCacheabilityOfResponse method:

[TestMethod]
public void ShouldSetNoCacheabilityOnDefaultPage()
{
    _Default page = new _Default();
    MockRepository mocks = new MockRepository();
    HttpResponseBase responseStub = mocks.Stub<HttpResponseBase>();
    HttpCachePolicyBase cachePolicyMock = mocks.CreateMock<HttpCachePolicyBase>();
    With.Mocks(mocks).Expecting(delegate
    {
        SetupResult.For(responseStub.Cache).Return(cachePolicyMock);
        cachePolicyMock.SetCacheability(HttpCacheability.NoCache);
        LastCall.On(cachePolicyMock).Repeat.AtLeastOnce();
    }).Verify(delegate
    {
        page.SetCacheablityOfResponse(responseStub);
    });
}

If you are not familiar with Rhino.Mocks or any other mocking framework, there appears to be a lot going on in that test. The basic idea is that we create derivatives of the -Base classes and pass these to the code that we are going to test, mimicking the behavior of the “real” objects that the framework would pass our code at runtime. Also note that in this particular test we test the side effect of our code, namely that the code should call a the SetCacheability method with a specific parameter. This is achieved using a mock object.

Example #2: Testing an HTTP Handler

Given the following HTTP handler code:

using System;
using System.Web;

public class RedirectAuthenticatedUsersHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        if (context.Request.IsAuthenticated)
        {
            context.Server.TransferRequest("/farfaraway");
        }
    }
}

Again, we extract the code we want to test into a separate method, passing it a -Base object and wrap the object passed to us from the framework in a -Wrapper object:

using System;
using System.Web;

public class RedirectAuthenticatedUsersHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        TransferUserIfAuthenticated(new HttpContextWrapper(context));
    }

    public void TransferUserIfAuthenticated(HttpContextBase context)
    {
        if (context.Request.IsAuthenticated)
        {
            context.Server.TransferRequest("/farfaraway");
        }
    }
}

This allows us to create unit tests for the TransferUserIfAuthenticated method, for instance:

[TestMethod]
public void ShouldRedirectAuthenticatedUser()
{
    RedirectAuthenticatedUsersHandler handler = new RedirectAuthenticatedUsersHandler();
    MockRepository mocks = new MockRepository();
    HttpContextBase httpContextStub = mocks.Stub<HttpContextBase>();
    HttpRequestBase httpRequestBaseStub = mocks.Stub<HttpRequestBase>();
    HttpServerUtilityBase httpServerUtilityMock = mocks.CreateMock<HttpServerUtilityBase>();
    With.Mocks(mocks).Expecting(delegate
    {
        SetupResult.For(httpContextStub.Request).Return(httpRequestBaseStub);
        SetupResult.For(httpContextStub.Server).Return(httpServerUtilityMock);
        SetupResult.For(httpRequestBaseStub.IsAuthenticated).Return(true);
        httpServerUtilityMock.TransferRequest("/farfaraway");
        LastCall.On(httpServerUtilityMock).Repeat.AtLeastOnce();
    }).Verify(delegate
    {
        handler.TransferUserIfAuthenticated(httpContextStub);
    });
}

Summary

I have shown two very simple examples on how some of the classes in the System.Web.Abstractions assembly can help us test our ASP.NET code. I have used HttpResponseBase, HttpServerUtilityBase, HttpContextBase, HttpRequestBase, and HttpCachePolicyBase. Note that there are a number of classes available, so if you are faced with not being able to test your ASP.NET code because of dependencies to classes in the framework, take a look in the System.Web namespace and see if there are -Base classes that can help you out.

Testability is a large topic, and there is much to be said about it. I have shown a couple of very simple examples on how to improve testability. Testability has a lot to do with code design as well; in a real world I would write the test before writing the code and I would move my code out of codebehinds. Those topics are discussed a lot elsewere, hopefully this post will bring you a small step further in writing testable code.

Five years ago there was a lot of hype around portal technologies. (You know of which I speak; portlets, JSR-168, WSRP, etc.).  In 2004, Gartner listed technologies such as JSR-168(Portlet Specification), JSR-170 (Content Repository for Java™ Technology API) and WSRP (Web Services for Remote Portlets) at the peak of their hype cycle in their “Hype Cycle for the Portal Ecosystem“. My impression is that since then those technologies have kind of faded away.

In my personal experience, I have not  seen any really successful portal technology deployments. (Of course, I am not implying that there are none such in existence, only I haven’t seen them). I have participated in my share of portal technology implementation projects, but none of them, I my opinion, did live up to their promise. Now seeing that there is less and less talk about portals (except from some software vendors that still are eager to sell their products that they have invested a lot in), I am questioning the future of portal frameworks as we know them.

What value do portals add?

In a discussion around web 2.0 technologies, a corporate communications person in a large international telecommunications company stated something like (I don’t remember his exact words) “portals have not really caught on – after all people prefer to read their email in Outlook instead of settling with the inferior experience in a portal”. This is a good point – many of the corporate portals focus on bringing a subset of functionality of other applications into a portal or workspace. But what is really the value added? After all, Firefox gave us tabbed browsing that has been adopted by all major browser vendors. Switching between your webmail and your (web-based) CRM system is only a Ctrl-tab away… In most scenarios, integration of applications using portal products did not deliver more than that.

A portal will add value when it is a common entry point where you can start when wanting to access information or a certain functionality. A typical example of this would be your corporate intranet. But it does not mean that the functionality necessarily needs to be delivered through that portal. Rather, it should guide or direct you to that application which delivers that functionality – only a click away.

Another area where portals could add value, is if data/functionality from several applications can be combined to add value. This has been a promise from portal frameworks that in my opinion has only to a very limited degree has been delivered. Inter-portlet communication did not happen to any large extent.

What is the cost of portals?

Compared to a plain web server technologies like Java Servlets, ASP.Net and the like, portal frameworks not only typically represent an extra licensing expense, they add quite a bit of complexity. In addition to configuration complexity, there is added complexity associated with customization and development. For instance, development models for portals are heavier and so are the development environments. Furthermore, portal frameworks constrain you in several ways, for instance when changing the look-and-feel or user interface experience. What is relatively easy to change in a standard web platform is much harder to change in a portal.

With the emergence of agile methodologies, disciplines like test-driven development and automated testing has become something that we have learned to appreciate. This has shown to be another area where portal frameworks get in your way as a developer- they are inherently hard to test.

The challengers

With the emergence of web 2.0-technologies, backed up by large Internet companies like Google and Yahoo! that do not sell software but services, the focus seems to be on lightweight technologies like Ajax, mashups, RSS, and REST-based services. Widgets/gadgets like the ones offered by services like Netvibes, iGoogle, Yahoo! and live.com are the soupe du jour. They offer a light-weight development model with a very low startup cost; no specific server-side technology knowledge is required, no specialized IDE or dev tools  required. The big question in my mind is whether these will challenge “traditional” portal technologies as the leading enabling technologies for intranet content aggregation? Quite recently Netvibes chose to open source their JavaScript widget engine. Furthermore, open source projects like Shindig are popping up that aims to offer developers frameworks for widget development.

Portal frameworks also continue to evolve. JSR-168 has been succeeded by JSR-286 and there is a version 2 of WSRP. The question is if this is enough to to make portlets prosper. Personally, I don’t think so as the problems are at a more fundamental, conceptual level than API specifications. But time will show…

In conclusion

“Traditional” portal frameworks have not lived up to their expectations. The use cases for portal technology represent niche functionality, at most. Furthermore, portal technology represent a heavy development model that in many cases will slow you down, compared to other technologies. If you are considering implementing portal technology in your company, you should very carefully investigate cost/benefit. Do not exclusively listen to portal vendors; talk to peer companies that have implemented portals, inquire them about their experiences. Talk to devs. Keep a close eye on emerging technologies. Keywords include widgets, Ajax, RSS, mashups.

Hva gjør en teknolog som forvalter SOA-laget? Hvor kommer dette SOA-laget fra? Er de i førstedivisjon?

Programvareselgere har klart å få kunder til å tro at SOA handler om lagdeling. Har du noe som kalles ‘SOA-laget’ i arkitekturen din, er det noe fundamentalt galt.

21apps.com has written an article about how to use Typemock Isolator in order to unit test SharePoint applications. According to the article, “Typemock has become the tool of choice for unit testing SharePoint as it supports faking everything including the sealed classes found throughout the object model”. Problem is that the root cause that prevents unit testing in SharePoint is an invasive API that creates dependencies between your code and the API. Using Typemock Isolator is treating the concequences without attacking the cause of the problem. But of course, having unit test is an advantage compared to having none.

Reading up on theory about mashups I found an article on Wikipedia that states the following about “oldfashioned” portals’ aggregation style: “Salad bar” style – Aggregated content is presented ’side-by-side’ without overlaps”. I think this is a nice analogy, and I would like to take it a step further, saying that:

• Oldfashioned portals (”portlet” technology) are salad bars, while
• mashups are salads

Portlets:

(Image: Wikipedia commons)

Mashups:

(Image: Wikipedia Commons)

No measurement beats WTFs/min!

When trying to introduce unit tests into an existing project, code dependencies is often a problem. During a teeth-grinding session grappling with this problem I came to think about pure functions.

Pure functions are functions that (according to Wikipedia’s definition):

• always evaluates the same result value given the same argument value(s). The function result value cannot depend on any hidden information or state that may change as program execution proceeds, nor can it depend on any external input from I/O devices, and
• evaluation of the result does not cause any semantically observable side effect or output, such as mutation of mutable objects or output to I/O devices.

Seems to me that pure functions would be a wet dream for unit testing. Imagine, you could pass in whatever the function needed to execute, and then check its return value. No mocking about with mock objects with side effect expectations needed!

But alas, we live in a world with non-pure functions. Even in functional programming, which does explicitly keep any state, you find them. In an object oriented language a primary idea is that objects should carry state. An object has a state. Period. However, if we can be inspired by the notion of pure functions, and make methods that have less dependencies, we will be much better off when it comes to unit testing.

I my experience, the ASP.NET programming model with ASPX code behind files is a model that promotes writing methods that are problematic to create unit tests for.

Take, for example, the following (theoretical, I admit) code behind snippet example:

protected void Page_Load(object sender, EventArgs e)

    {

        ShowOrHideEditButton(Page.User.Identity.Name);

    }

    void ShowOrHideEditButton(string currentUser)

    {

        if (Request.QueryString["user"].Equals(currentUser))

        {

            EditButton.Enabled = true;

        }

        else

        {

            EditButton.Enabled = false;

        }

    }

So, what is the problem here? Actually, there are two problems.

1. The ShowOrHideMethod is not only dependant on its input parameters, but also on another member variable. When looking at the method signature, you do not see this, and might be misleaded about what the outcome of the methods is dependent on.
2. The method does not return any value, but has a side-effect on its environment (setting the edit button visibility).

These two issues, makes it troublesome to test the method in a unit test. You have spend time to set up the methods context, and you have to measure the method’s side effects in order to make assesments about its result.

A much better approach would be something like this:

    protected void Page_Load(object sender, EventArgs e)

    {

        EditButton.Enabled = ShowOrHideEditButton(Page.User.Identity.Name, Request.QueryString["user"]);

    }

    bool ShowOrHideEditButton(string currentUserName, string profilePerson)

    {

        return (currentUserName.Equals(profilePerson));

    }

This way, we can easily test the ShowOrHideEditButton method in a unit test without having to use a great deal test code to set up the test context.

For increased testability, I would suggest the following for better method purity:

• A method should evaluate the same result, given the same argument value(s) and object state.
• A method execution result should be represented as, in order of preference: 1) a return value, 2) changed object state (mutation), or 3) a side-effect on an I/O device