Ni av ti nordmenn ønsker å kommunisere med fastlegen sin over internett.
– Helsevesenet bør få én felles inngangsportal på nett, sier Teknologiråds-direktør Tore Tennøe.

Denne påstanden er basert på en undersøkelse foretatt av Response Analyse Oslo for Teknologirådet. Hvis man føler lenke til denne artikkelen, fremgår noe mer informasjon om metodikken bak undersøkelsen:

Et landsrepresentativt utvalg på 1098 personer fra 17 til 84 år har svart på den elektroniske spørreundersøkelsen…

Spørsmålet er da: er det et landsrepresentativt utvalg hvis man kun baserer seg på dem som svarer på elektroniske undersøkelser? En ting som er helt sikkert, er at man ikke kan si at “9 av 10 nordmenn ønsker…”. Siden undersøkelsen gjelder bruk av elektroniske hjelpemidler er det en åpenbar svakhet at (sannsynligvis) kun de som bruker elektroniske hjelpemidler har mulighet til å svare. Når det er sagt er det veldig mangelfull informasjon om metodikken brukt i undersøkelsen, så man kan ikke være bastant her. Viktige momenter er hvordan man har valgt ut respondenter, hvordan man har kontaktet dem, og ikke minst hvor mange som har unnlatt å svare.

Metodikken i undersøkelsen er det vanskelig å si noe konkret om siden veldig lite informasjon om den er tilgjengelig. En ting er dog sikkert, artikkelen som har dedusert at “9 av 10 nordmenn…” burde si noe om datagrunnlaget for påstanden. At undersøkelsen var foretatt elektronisk er essensielt. Kanskje burde det heller stå “9 av 10 nordmenn som svarer på elektroniske undersøkelser ønsker…”

One thing is that the perpetrators could use this information to get details about my relationship with Atlassian, among other things my license keys for Atlassian products. Even worse is that they could try and use the passwords to get into my accounts at other sites. That would be successful if I used the very usual and very baaad practice of reusing the same (user name and) password on several sites and applications. Luckily I don’t.

Here is what Atlassian states about why this could happen:

During July 2008, we migrated our customer database into Atlassian Crowd, our identity management product, and all customer passwords were encrypted. However, the old database table was not taken offline or deleted, and it is this database table that we believe could have been exposed during the breach

Trying to act as a responsible company, Atlassian goes on to list what they have learned from the incident. Among other things, they state that

The legacy customer database, with passwords stored in plain text, was a liability. Even though it wasn’t active, it should have been deleted. There’s no logical explanation for why it wasn’t, other than as we moved off one project, and on to the next one, we dropped the ball and screwed up.

I am sorry, but I find it hard to believe that this is the entire truth. Yesterday afternoon (European time) I went to their site and on the login screen I used their “Forgot my password” functionality. Can you guess what happened? They sent me an email with my password in clear text! So, I would indeed say that this “legacy database” is indeed active…

Later on the day yesterday, I also got an email from the Apache Software Foundation that their JIRA instance also have been hacked. See their blog entry about the issue. According to the blog entry, the situation is a bit better than it is at Atlassian. They state that

If you are a user of the Apache hosted JIRA, Bugzilla, or Confluence, a hashed copy of your password has been compromised.

JIRA and Confluence both use a SHA-512 hash, but without a random salt. We believe the risk to simple passwords based on dictionary words is quite high, and most users should rotate their passwords.

At least, the password was not stored in clear text, which is of course much better than having it in clear text. However, the compromised passwords could still be useful for an attacher because they are not salted. It allows an hacker to compare hashes of other accounts with hashes of a known password which would allow a dictionary attack (as is stated).

This is not security for the crowds (pun intended).

The application in question has a scheduled task that once every day sends out an email which has a list of recipients. This list of recipients reached the aforementioned limit and the email sending failed. The log stated:

OverQuotaError: The API call mail.Send() required more quota than is available.

The solution to this was to change the email sending so that one email was generated per recipient, and the sending of the email was queued using the brilliant (however experimental) task queue functionality that GAE provides. The code for queuing looks like this:

queue = Queue('mail-queue')
for recipient in to:
    queue.add(Task(url='/task/mail', params= { 'to' : recipient, 'subject' : subject, 'body' : body }))

Here’s the actual task code that sends the email:

class MailSender(webapp.RequestHandler):
    def post(self):
        to = self.request.get('to')
        subject = self.request.get('subject')
        body = self.request.get('body')
        logging.info("Sending '%s' to %s" % (subject, to))
        mail.send_mail("not.the.real.sender@not.a.real.domain.com", to, subject, body)

Finally, I defined the ‘mail-queue’ in ‘queue.yaml’:

queue:
- name: mail-queue
  rate: 8/m

Works like a charm! For more information about task queuing, see http://code.google.com/intl/no/appengine/docs/python/taskqueue/overview.html

In those situations, we can still use XmlSerializer. In order to achieve the same encapsulation with XmlSerializer, we have to adjust our model a bit. Here’s one suggestion on how to do this:

My approach to this is inspired by Josh Bloch‘s Builder pattern. The idea is changing the classes used for deserialization from being domain objects to being builder objects that build domain objects. This has another advantage in that our domain objects are not “polluted” with attributes and interfaces related to deserialization and are 100% plain old CLR objects (POCOs). So, lets first do a change our “deserialization” class (formerly ‘Country’) to a builder object, like so:

public class CountryBuilder
{
   [XmlElement(ElementName = "name")]
   public string Name;

   [XmlElement(ElementName = "iso-3166-alpha-2-code")]
   public string Code;

   public Country Build()
   {
      return new Country(Name, Code);
   }
}

Note here that our builder object has a Build() method which returns the domain object ‘Country’. This is the object that we will pass on to our clients. The Country class now represents our domain object:

public class Country
{
   private readonly string _name, _code;

   internal Country(string name, string code)
   {
      _name = name;
      _code = code;
   }

   public string Name { get { return _name; } }
   public string Code { get { return _code; } }
}

We have now restricted access to the creation of Country objects in that the constructor is internal, and it is also immutable (cannot change state once created) though making its fields readonly. It only exposes getters for its internal state. We can then do the same to our list of countries. The builder object for countries would look like this:

[XmlRoot("countries")]
public class Countries
{
   [XmlElement(ElementName="country")]
   public CountryBuilder[] countries;

   public IEnumerable<Country> Build()
   {
      return countries.Select<CountryBuilder , Country>(x => x.Build());
   }
}

The code for doing the serialization will then look like this:

string xml = ...;
XmlSerializer xmlSerializer = new XmlSerializer(typeof(CountriesBuilder));
var builder = xmlSerializer.Deserialize(new StringReader(inputXml)) as CountriesBuilder;
IEnumerable<Country> cs = builder.Build();

What we have achieved now is that we now can control the accessibility and encapsulation of our domain model. The builder objects are still public to anyone, but that really does not matter much in my mind. Thus, a relatively swift parsing of XML of various formats into a well designed object model.

Given the case where you have a fairly straightforward XML document (not too deep document tree, not too complex set of attributes and elements) that maps pretty well to your domain model, the serialization options is in my mind a good choice that requires little coding. Compared with this approach, using XmlDocument seems to be a bit of an overkill if you don’t need advanced traversal of the document, and writing a parser by hand using XmlReader seems to require quite a bit of coding.

So, given the following sample XML document, I will investigate the serialization options:

<countries>
   <country>
      <iso-3166-alpha-2-code>AF</iso-3166-alpha-2-code>
      <name>Afghanistan</name>
   </country>
   <country>
      <iso-3166-alpha-2-code>AX</iso-3166-alpha-2-code>
      <name>Åland Islands</name>
   </country>
   <country>
      <iso-3166-alpha-2-code>AL</iso-3166-alpha-2-code>
      <name>Albania</name>
   </country>
</countries>

Using System.Xml.XmlSerializer

The first option that came to mind, was to use the XmlSerializer object to deserialize the XML into C# (or VB for that matter) objects. It first requires that I annotate my object model in order to tell the serializer how to deserialize the XML:

[XmlRoot("countries")]
public class Countries
{
   [XmlElement(ElementName="country")]
   public Country[] countries;
}

public class Country
{
   [XmlElement(ElementName = "name")]
   public string Name;

   [XmlElement(ElementName = "iso-3166-alpha-2-code")]
   public string Code;
}

Then, I can use the serializer to deserialize the code:

string xml = ...;

XmlSerializer xmlSerializer = new XmlSerializer(typeof(Countries));
Countries c = xmlSerializer.Deserialize(new StringReader(xml)) as Countries;

Pretty sweet,  heh? Definitely. However, this has some drawbacks. If I want my Country class to be a well designed domain object that follows good OO design principles, I probably would like to encapsulate my data. Furthermore, I might want to restrict the creation of such objects from other parts of the code. In order for XmlSerializer to create my object, it requires that my types are public and that all properties or fields to set are public as well. What to do if I want to enforce my objects to be immutable once handed off to other parts of the code?

Using System.Runtime.Serialization.DataContractSerializer

Luckily, the serialization API that come with Windows Communication Framework has some neat features that fit like a glove. When defining my data model, it does not require that the types, neither the properties nor fields to set are public. Actually, I can restrict access to the type, its default constructor, and any of the properties or fields that I want to be deserialized! w00t!

So, this is what the Country class will looks like:

[DataContract(Name="country", Namespace="")]
internal class Country : IExtensibleDataObject
{
   private Country() { }

   [DataMember(Name="name")]
   public string Name { get; private set; }

   [DataMember(Name = "iso-3166-alpha-2-code")]
   public string Code { get; private set; }

   public ExtensionDataObject ExtensionData { get; set; }
}

The XML file contains a list of countries, and luckily, we have the CollectionDataContractAttribute to denote an element that is a list of elements. It supports generics, so that we can define our class as a strongly typed list:

[CollectionDataContract(Name="countries", Namespace="")]
internal class Countries : List<Country>, IExtensibleDataObject
{
   public ExtensionDataObject ExtensionData { get; set; }
}

And that’s it. Now we can deserialize:

string xml = ...;

DataContractSerializer ser = new DataContractSerializer(typeof(Countries));
using (StringReader stringReader = new StringReader(xml))
{
   using (XmlReader xmlReader = XmlReader.Create(stringReader))
   {
      Countries countries = (Countries)ser.ReadObject(xmlReader);
   }
}

Alternatively, our result could be typed as a list of countries:

IList<Country> countries = (IList<Country>)ser.ReadObject(xmlReader);

Note that there is a limitation in the latter method in that deserializing XML attributes is not supported. Thus, an XML document like the following would not work:

   <country iso-3166-alpha-2-code="AF">
      <name>Afghanistan</name>
   </country>
   <country iso-3166-alpha-2-code="AX">
      <name>Åland Islands</name>
   </country>
   <country iso-3166-alpha-2-code="AL">
      <name>Albania</name>
   </country>
</countries>

This will, however, work using the XmlSerializer.

So, given the same examples as in my previous post, here is what the test code will look like:

Example #1: Testing a page codebehind file

[TestMethod]
public void ShouldSetNoCacheabilityOnDefaultPage()
{
    _Default page = new _Default();
    HttpCachePolicyMock httpCachePolicyMock = new HttpCachePolicyMock();
    page.SetCacheablityOfResponse(new HttpResponseStub
    {
        TheCache = httpCachePolicyMock
    });
    httpCachePolicyMock.ShouldHaveSetCacheabilityTo(HttpCacheability.NoCache);
}

class HttpResponseStub : HttpResponseBase
{
    public override HttpCachePolicyBase Cache { get { return TheCache; } }
    public HttpCachePolicyBase TheCache { get; set; }
}

class HttpCachePolicyMock : HttpCachePolicyBase
{
    private HttpCacheability _cacheability;
    public override void SetCacheability(HttpCacheability cacheability)
    {
        _cacheability = cacheability;
    }
    public void ShouldHaveSetCacheabilityTo(HttpCacheability expectedCacheability)
    {
        Assert.AreEqual(expectedCacheability, _cacheability);
    }
}

I have created two helper classes, one with the suffix -Stub and one with the suffix -Mock. The convention here is that a stub is a type of class used to provide a context to the class under test. Mocks also do that, but additionally a mock can make expectation about what should happen to it during the test.

Example #2: Testing an HTTP handler

[TestMethod]
public void ShouldRedirectAuthenticatedUser()
{
    HttpServerUtilityMock httpServerUtilityMock = new HttpServerUtilityMock();
    HttpContextStub httpContextStub = new HttpContextStub
    {
        TheRequest = new HttpRequestStub { IsItAuthenticated = true },
        TheServer = httpServerUtilityMock
    };
    new RedirectAuthenticatedUsersHandler().TransferUserIfAuthenticated(httpContextStub);
    httpServerUtilityMock.ShouldHaveTransferredTo("/farfaraway");
}

class HttpContextStub : HttpContextBase
{
    public override HttpRequestBase Request { get { return TheRequest; } }
    public override HttpServerUtilityBase Server { get { return TheServer; } }
    public HttpRequestBase TheRequest { get; set; }
    public HttpServerUtilityBase TheServer { get; set; }
}

class HttpRequestStub : HttpRequestBase
{
    public override bool IsAuthenticated { get { return IsItAuthenticated; } }
    public bool IsItAuthenticated { get; set; }
}

class HttpServerUtilityMock : HttpServerUtilityBase
{
    private string _path;
    public override void TransferRequest(string path)
    {
        _path = path;
    }

    public void ShouldHaveTransferredTo(string expectedPath)
    {
        Assert.AreEqual(expectedPath, _path);
    }
}

WTF? I am quite sure that replacing ToLower() with ToUpperInvariant() will make my test fail…

The HttpRequestBase class is an abstract class that contains the same members as the HttpRequest class. The HttpRequestBase class enables you to create derived classes that are like the HttpRequest class, but that you can customize and that work outside the ASP.NET pipeline. When you perform unit testing, you typically use a derived class to implement members that have customized behavior that fulfills the scenario that you are testing.

Very well. Looking at the documentation for HttpRequest, we see that HttpRequest is not a decendent of HttpRequestBase as one might expect from the name. The reason for this is probably that that would break backwards compatability with older versions of ASP.NET. So, how can we exploit the HttpRequestBase then? The answer is the HttpRequestWrapper class which is a decendant of HttpRequestBase and has a constructor that takes an HttpRequest object as a parameter. Then, we can take the HttpRequest object passed to our code from the framework, wrap it inside an HttpRequestWrapper object and pass it on to our code as a HttpRequestBase object. As I will show you in the examples below, this will enable us to create unit tests of our code by creating fake implementations of ASP.NET framework clases (using Rhino.Mocks).

Example #1: Testing a page codebehind file

Take, for instance, this simple page codebehind code that we would like to test:

using System;
using System.Web;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        Response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

The first step to take here, is to extract a method which take the request object as a parameter instead of fetching it from a method in a superclass. In general, this is a variation of the dependency injection pattern which in many situations will help us make our code testable (also, see my earlier related post). Like so:

using System.Web;
using System;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        SetCacheablityOfResponse(Response);
    }

    public void SetCacheablityOfResponse(HttpResponse response)
    {
        response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

So, then having extracted our code in a separate method, the next step is to change the parameter type of this method from HttpRequest to HttpRequestBase. Furthermore, when calling this method, we need to wrap the HttpRequest object by creating an instance of HttpRequestWrapper. The code, then, looks like this:

using System.Web;
using System;

public partial class _Default : System.Web.UI.Page
{
    protected void Page_Init(object sender, EventArgs e)
    {
        SetCacheablityOfResponse(new HttpResponseWrapper(Response));
    }

    public void SetCacheablityOfResponse(HttpResponseBase response)
    {
        response.Cache.SetCacheability(HttpCacheability.NoCache);
    }
}

Having now prepared our code for testing, we can create a unit test where we test the SetCacheabilityOfResponse method:

[TestMethod]
public void ShouldSetNoCacheabilityOnDefaultPage()
{
    _Default page = new _Default();
    MockRepository mocks = new MockRepository();
    HttpResponseBase responseStub = mocks.Stub<HttpResponseBase>();
    HttpCachePolicyBase cachePolicyMock = mocks.CreateMock<HttpCachePolicyBase>();
    With.Mocks(mocks).Expecting(delegate
    {
        SetupResult.For(responseStub.Cache).Return(cachePolicyMock);
        cachePolicyMock.SetCacheability(HttpCacheability.NoCache);
        LastCall.On(cachePolicyMock).Repeat.AtLeastOnce();
    }).Verify(delegate
    {
        page.SetCacheablityOfResponse(responseStub);
    });
}

If you are not familiar with Rhino.Mocks or any other mocking framework, there appears to be a lot going on in that test. The basic idea is that we create derivatives of the -Base classes and pass these to the code that we are going to test, mimicking the behavior of the “real” objects that the framework would pass our code at runtime. Also note that in this particular test we test the side effect of our code, namely that the code should call a the SetCacheability method with a specific parameter. This is achieved using a mock object.

Example #2: Testing an HTTP Handler

Given the following HTTP handler code:

using System;
using System.Web;

public class RedirectAuthenticatedUsersHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        if (context.Request.IsAuthenticated)
        {
            context.Server.TransferRequest("/farfaraway");
        }
    }
}

Again, we extract the code we want to test into a separate method, passing it a -Base object and wrap the object passed to us from the framework in a -Wrapper object:

using System;
using System.Web;

public class RedirectAuthenticatedUsersHandler : IHttpHandler
{
    public void ProcessRequest(HttpContext context)
    {
        TransferUserIfAuthenticated(new HttpContextWrapper(context));
    }

    public void TransferUserIfAuthenticated(HttpContextBase context)
    {
        if (context.Request.IsAuthenticated)
        {
            context.Server.TransferRequest("/farfaraway");
        }
    }
}

This allows us to create unit tests for the TransferUserIfAuthenticated method, for instance:

[TestMethod]
public void ShouldRedirectAuthenticatedUser()
{
    RedirectAuthenticatedUsersHandler handler = new RedirectAuthenticatedUsersHandler();
    MockRepository mocks = new MockRepository();
    HttpContextBase httpContextStub = mocks.Stub<HttpContextBase>();
    HttpRequestBase httpRequestBaseStub = mocks.Stub<HttpRequestBase>();
    HttpServerUtilityBase httpServerUtilityMock = mocks.CreateMock<HttpServerUtilityBase>();
    With.Mocks(mocks).Expecting(delegate
    {
        SetupResult.For(httpContextStub.Request).Return(httpRequestBaseStub);
        SetupResult.For(httpContextStub.Server).Return(httpServerUtilityMock);
        SetupResult.For(httpRequestBaseStub.IsAuthenticated).Return(true);
        httpServerUtilityMock.TransferRequest("/farfaraway");
        LastCall.On(httpServerUtilityMock).Repeat.AtLeastOnce();
    }).Verify(delegate
    {
        handler.TransferUserIfAuthenticated(httpContextStub);
    });
}

Summary

I have shown two very simple examples on how some of the classes in the System.Web.Abstractions assembly can help us test our ASP.NET code. I have used HttpResponseBase, HttpServerUtilityBase, HttpContextBase, HttpRequestBase, and HttpCachePolicyBase. Note that there are a number of classes available, so if you are faced with not being able to test your ASP.NET code because of dependencies to classes in the framework, take a look in the System.Web namespace and see if there are -Base classes that can help you out.

Testability is a large topic, and there is much to be said about it. I have shown a couple of very simple examples on how to improve testability. Testability has a lot to do with code design as well; in a real world I would write the test before writing the code and I would move my code out of codebehinds. Those topics are discussed a lot elsewere, hopefully this post will bring you a small step further in writing testable code.

• IIS and Kerberos. Part 1 – What is Kerberos and how does it work?
• IIS and Kerberos. Part 2 – Service Principal Names
• IIS and Kerberos. Part 3 – A simple scenario
• IIS and Kerberos. Part 4 – A simple delegation scenario
• IIS and Kerberos Part 5 – Protocol Transition, Constrained Delegation, S4U2S and S4U2P
• IIS and Kerberos Part 6 – New in IIS 7
• IIS and Kerberos Part 7 – A simple cross Forest scenario
• IIS and Kerberos Part 8 – a simple cross Forest/Domain delegation scenario
• IIS and Kerberos Part 9 – Cross Forest Delegation scenario with UPN suffix routing

Simply a great source of information!