# line-gunk.ps1
"123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"

If you run this Powershell file from the Windows command line, and redirects output to a file:

C:\>@powershell -file .\line-gunk.ps1 > foo.txt

Then you will get:

1234567890123456789012345678901234567890123456789012345678901234567890123456789
01234567890

However, if you do the redirection in Powershell, it’s fine:

PS C:\> .\line-gunk.ps1 > bar.txt

A näive fix to this, would be to change your script to use Write-Host to explicity write the text to the console. However, I consider this a bad practice because it would restrict the usage of your script as you could not manipulate the output further if reusing the script. I found this entry on stackoverflow which provides a better solution. Going back to the original script, here is a fixed one that worked for me:

# line-gunk-fixed.ps1
$Host.UI.RawUI.BufferSize = New-Object Management.Automation.Host.Size(500, 25)
"123456789012345678901234567890123456789012345678901234567890123456789012345678901234567890"

Notice that I used the simplifed version from the stackoverflow entry, which might not work in all circumstances.

Backstory

How did I come across this problem? I running Powershell build scripts in TeamCity and I was trying to report the build progress back to TeamCity. However, it somehow did not work. I then discovered that the status messages written by my script were broken by the line breaks, rendering them useless…

The remote server returned an error: (407) Proxy Authentication Required.

Yes, I am behind a firewall that requires authentication. I found several references to this error message on the chocolatey forums, but they all seemed to be fixed in the latest chocolatey version.

Through some googling I was able to track down the problem. Acutally, it is a problem with the .NET 3.5 runtime that causes the System.Net.WebClient to give up on an NTLM authentication challenge from proxies. Chocolatey uses Powershell, and indeed the System.Net.WebClient to download installation packages (all though not the package manifest itself, where it uses NuGet). As you my or may not know, Powershell uses the .NET 3.5 runtime per default.

Hence, the fix was to make Powershell use the .NET 4.0 runtime, where this bug is fixed. I figured out how to do this based on this entry on stackoverflow.

I then wrote this little Powershell snippet which changes Powershell on my machine to use .NET 4.0 instead of .NET 3.5:

if ([Environment]::Version.Major -lt 4) {
    $configFile = Join-Path $PSHOME -ChildPath '.\powershell.exe.config'
    if (-not(Test-Path $configFile)) {
        @'
<?xml version="1.0"?>
<configuration>
  <startup useLegacyV2RuntimeActivationPolicy="true">
    <supportedRuntime version="v4.0.30319" />
    <supportedRuntime version="v2.0.50727" />
  </startup>
</configuration>
'@ | Out-File -FilePath $configFile -Encoding UTF8
        "Created $configFile"
    }
    'Restart powershell in order to make it run in .NET 4'
}

Wrong:

@'1
2
3
'@

Unrecognized token in source text.
At ...ps1:1 char:1
+  <<<< @'1
    + CategoryInfo          : ParserError: (:) [], ParseException
    + FullyQualifiedErrorId : UnrecognizedToken

@'
1
2
3'@

The string starting:
At ...:12 char:6
+ $s =  <<<< @'
is missing the terminator: '@.
At ...ps1:34 char:3
+ #> <<<<
    + CategoryInfo          : ParserError: (1
2
3'@
:String) [], ParseException
    + FullyQualifiedErrorId : TerminatorExpectedAtEndOfString

@'
1
2
3
'@

1
2
3

@'
@echo off
echo Wi nøt trei a høliday in Sweden this ÿer?
'@ > '.\Opening titles (unicode).cmd'
& '.\Opening titles (unicode).cmd'

Output: '■@' is not recognized as an internal or external command,
operable program or batch file.

@'
@echo off
echo Wi nøt trei a høliday in Sweden this ÿer?
'@ | Out-file '.\Opening titles (Ascii).cmd' -Encoding ASCII
& '.\Opening titles (Ascii).cmd'

Output: Wi n?t trei a h?liday in Sweden this ?er?

@'
@echo off
echo Wi nøt trei a høliday in Sweden this ÿer?
'@ | Out-file '.\Opening titles (OEM).cmd' -Encoding OEM
& '.\Opening titles (OEM).cmd'

Output: Wi nøt trei a høliday in Sweden this ÿer?

var index = (a == "a") ? 1 : 2;

In Powershell, such an operator is (to my knowledge) not available. One solution is described here. As an alternative without having to create a function and alias, I suggest:

$index = @{$true=1;$false=2}[$a -eq 'a']

Mmmmm, one-liner FTW

var ascii7BitEncoding = Encoding.ASCII; // ASCII
var ascii8BitEncodingIsh = Encoding.GetEncoding("iso-8859-1"); // Extended ASCII for almost all practical purposes

Where Extended ASCII differs from ISO-8859-*, according to Wikipedia:

One notable way in which ISO character sets differ from code pages is that the character positions 128 to 159, corresponding to ASCII control characters with the high-order bit set, are specifically unused and undefined in the ISO standards, though they had often been used for printable characters in proprietary code pages, a breaking of ISO standards that was almost universal.

However, to my mind there is a shortcoming. Since the transformation happens during packaging or publishing of the application, transformation rules do not take effect when debugging a web application (typically by hitting F5 in Visual Studio) because the application is served from the project directory. Here’s how to solve this: essentially, we can change the the project setup so that the transformation happens right after the project build step. Here’s how:

Step 1: Unload the project in Visual Studio

In the Solution Explorer in Visual Studio, right-click on the project and select Unload project from the context menu:

Step 2: Rename Web.config on the file system

In the Windows Explorer, browse to the project directory, and rename Web.config to Web.template.config:

Step 3: Rename Web.config in the project file

What we will do here, is to rename the existing Web.config to Web.template.config in the project file (.csproj or .vbproj). This will be the file that we keep in the version control system, and the final Web.config will instead be generated for us when building the application.

In the Solutuion Explorer in Visual Studio, right-click on the project and select Edit <project name>:

Search for “web.config” in the project file to locate where the Web.config and transformation files definitions. It looks something like so:

<Content Include="Web.config" />
<Content Include="Web.Debug.config">
   <DependentUpon>Web.config</DependentUpon>
</Content>
<Content Include="Web.Release.config">
   <DependentUpon>Web.config</DependentUpon>
</Content>

In the project file, for instance on the line after the one with Web.config, add the following:

<None Include="Web.template.config" />

Then, change the text of the DependentUpon element to read Web.template.config instead of Web.config. Finally, change the Content elements for for Web.Debug.config and Web.Release.config to None elements. This prevents these files from being included in the deployment package. They don’t need to be, as the transformations have already been run. All in all, the snippet from above should now read:

<Content Include="Web.config" />
<None Include="Web.template.config" />
<None Include="Web.Debug.config">
   <DependentUpon>Web.template.config</DependentUpon>
</None>
<None Include="Web.Release.config">
   <DependentUpon>Web.template.config</DependentUpon>
</None>

Step 4: Running transformation when building

Now that we have a reference to Web.template.config in the project file, the next step is to run the transformation during the build step. Add a custom target that runs the transformation and then include it at the begining of the build task using the {{BuildDependsOn}} property. The resulting XML snippet looks like so:

<PropertyGroup>
  <BuildDependsOn>
    CustomWebConfigTransform;
    $(BuildDependsOn);
  </BuildDependsOn>
</PropertyGroup>
<Target Name="CustomWebConfigTransform">
   <TransformXml> source="Web.template.config"
      transform="Web.$(Configuration).config"
      destination="Web.config" />
</Target>

Save the file and reload the project into the Visual Studio solution. Then, in the Solution Explorer, observe that the Web.config file now is denoted as missing, and that the Web.template.config is present with the two transformation files beneath it in the file tree:

Step 5: Test running transformations during build

In this step, we will test that the transformations will be run when launching the application by hitting F5 in Visual Studio. First, we need to create a configuration example in Web.Debug.config. Remove all the gunk that Microsoft so helpfully has filled the file with during project creation, and insert one simple appSettings entry, making the Web.Debug.config look like this:

<configuration xmlns:xdt="http://schemas.microsoft.com/XML-Document-Transform">
  <appSettings>
    <add key="Configuration" value="Debug" xdt:Transform="Insert"/>
  </appSettings>
</configuration>

Take care to include the xdt XML namespace definition. Without it, the transformations will not be run.

Open up an ASPX page, for instance Default.aspx, and enter a label that will display the value of our confiugration setting:

<p>
   Configuration: <asp:Label ID="Configuration" runat="server" />
</p>

Next, enter this in the code behind:

protected void Page_Load(object sender, EventArgs e)
{
   Configuration.Text = ConfigurationManager.AppSettings["Configuration"];
}

Hit F5 to run the application and verify that it in fact is able to read from the transformed Web.config:

Step 6: Disable transformations during packaging and publishing

Since msbuild by default runs the transformations, we need to disable this to prevent the transformations to be applied twice. To do this, unload the project from the solution as we did in step 1, and then edit the project file in Visual Studio. For each of the configurations that you have in your solution, disable the built-in transformation runs by adding the TransformWebConfigEnabled element and setting it to false. Like so:

<PropertyGroup Condition=" '$(Configuration)|$(Platform)' == 'Debug|AnyCPU' ">
   <TransformWebConfigEnabled>False</TransformWebConfigEnabled>
   ...
</PropertyGroup>

Summary / other tweaks

What we have achieved, is that the transformations are applied when building the application, making the transformed file available when running the application locally from Visual Studio. Another tip with regards to Web Transformations is to use the SlowCheetah Visual Studio Add-in that Scott Hanselman has described very well here. It will give you a nice transformation preview on the context menu in Visual Studio which is extremely helpful.

The concepts in NLog is very similar to Log4Net. More or less, replace the word “appender” in Log4Net lingo with “target”, and you’re game.

First, let’s create a class for log entries:

public class LogEntry : TableServiceEntity
{
    public LogEntry()
    {
        var now = DateTime.UtcNow;
        PartitionKey = string.Format("{0:yyyy-MM}", now);
        RowKey = string.Format("{0:dd HH:mm:ss.fff}-{1}", now, Guid.NewGuid());
    }
    #region Table columns
    public string Message { get; set; }
    public string Level { get; set; }
    public string LoggerName { get; set; }
    public string RoleInstance { get; set; }
    public string DeploymentId { get; set; }
    public string StackTrace { get; set; }
    #endregion
}

Next, we need to do is to create a class that represents the table storage service. It needs to inherit from TableServiceContext:

public class LogServiceContext : TableServiceContext
{
    public LogServiceContext(string baseAddress, StorageCredentials credentials) : base(baseAddress, credentials) { }
    internal void Log(LogEntry logEntry)
    {
        AddObject("LogEntries", logEntry);
        SaveChanges();
    }
    public IQueryable<LogEntry> LogEntries
    {
        get
        {
            return CreateQuery<LogEntry>("LogEntries");
        }
    }
}

Finally, as far as code is concerned, a class that is a custom NLog target that gets called when the NLog framework needs to log something:

[Target("AzureStorage")]
public class AzureStorageTarget : Target
{
    private LogServiceContext _ctx;
    private string _tableEndpoint;
    [Required]
    public string TableStorageConnectionStringName { get; set; }
    protected override void InitializeTarget()
    {
        base.InitializeTarget();
        var cloudStorageAccount =
            CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue(TableStorageConnectionStringName));
        _tableEndpoint = cloudStorageAccount.TableEndpoint.AbsoluteUri;
        CloudTableClient.CreateTablesFromModel(typeof(LogServiceContext), _tableEndpoint, cloudStorageAccount.Credentials);
        _ctx = new LogServiceContext(cloudStorageAccount.TableEndpoint.AbsoluteUri, cloudStorageAccount.Credentials);
    }
    protected override void Write(LogEventInfo loggingEvent)
    {
        Action doWriteToLog = () =>
        {
            try
            {
                _ctx.Log(new LogEntry
                {
                    RoleInstance = RoleEnvironment.CurrentRoleInstance.Id,
                    DeploymentId = RoleEnvironment.DeploymentId,
                    Timestamp = loggingEvent.TimeStamp,
                    Message = loggingEvent.FormattedMessage,
                    Level = loggingEvent.Level.Name,
                    LoggerName = loggingEvent.LoggerName,
                    StackTrace = loggingEvent.StackTrace != null ? loggingEvent.StackTrace.ToString() : null
                });
            }
            catch (DataServiceRequestException e)
            {
                InternalLogger.Error(string.Format("{0}: Could not write log entry to {1}: {2}",
                    GetType().AssemblyQualifiedName, _tableEndpoint, e.Message), e);
            }
        };
        doWriteToLog.BeginInvoke(null, null);
    }
}

So, to make it work, we need to register the target with the NLog framework. This is done in the NLog.config file:

<?xml version="1.0"?>
<nlog xmlns="http://www.nlog-project.org/schemas/NLog.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <extensions>
    <add assembly="Demo.NLog.Azure" />
  </extensions>
  <targets>
    <target name="azure" type="AzureStorage" tableStorageConnectionStringName="Log4Net.ConenctionString" />
  </targets>
  <rules>
    <logger name="*" minlevel="Info" writeTo="azure" />
  </rules>
</nlog>

For information about how to set your ServiceDefinition.csdef and ServiceConfiguration.cscfg files, see my previous post.You can find the code for this example on GitHub. Suggestions for improvement are very welcome.

Basically, what you do is that you switch off the built-in authentication in ASP.NET like forms-based authentication, and let WIF act as a proxy in front of your application. WIF uses the authorization settings in web.config (/configuration/system.web/authorization and /configuration/location/system.web/authorization elements) and authenticates the user before the ASP.NET application receives the request. See WS-Federated Authentication Module Overview for details

So, when the application receives the request, the user is already authenticated, which is fine. However, there are times when the application needs to know who the user is, or getting access to the other claims that were provided from the identity service. Luckily, this is available on the HTTP context. Say, for instance, if you wish to find the email address of the logged-in user, you can do it like so:

protected void Page_Load(object sender, EventArgs e)
{
    var claimsIdentity = Context.User.Identity as IClaimsIdentity;
    foreach (var claim in claimsIdentity.Claims)
    {
        if (claim.ClaimType == "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/emailaddress")
        {
            var email = claim.Value;
        }
    }
}

If the user was authenticated using claims, Context.User.Identity will be an IClaimsIdentity which will contain a number of claims about the user. We can then iterate over these claims to find the one that we want.

The claim types are denoted using XML namespaces, which are a little bit cumbersome to work with. So, to make it easier to access, we iterate over the list of claims, and make all the claims available on a common object:

public static class IdentityExtension
{
    public static DynamicUser GetUserFromClaims(this HttpContext context)
    {
        var claimsIdentity = context.User.Identity as IClaimsIdentity;
        if (claimsIdentity == null) throw new FailedAuthenticationFaultException();
        return new DynamicUser(claimsIdentity.Claims);
    }
}
public class DynamicUser : DynamicObject
{
    private readonly ClaimCollection _claims;
    public DynamicUser(ClaimCollection claims)
    {
        _claims = claims;
        Id = ClaimsValue("nameidentifier")[0];
    }
    public string Id { get; private set; }
    public bool IsInRole(string role)
    {
        return ClaimsValue("role").Any(c => c.Equals(role, StringComparison.OrdinalIgnoreCase));
    }
    private string[] ClaimsValue(string claimSuffix)
    {
        return _claims.Where(c => c.ClaimType.EndsWith(claimSuffix)).Select(c => c.Value).ToArray();
    }
    public override bool TryGetMember(GetMemberBinder binder, out object result)
    {
        var claimValue = ClaimsValue(binder.Name.ToLowerInvariant());
        if (claimValue.Length == 0)
        {
            return base.TryGetMember(binder, out result);
        }
        if (claimValue.Length == 1)
        {
            result = claimValue[0];
        }
        else
        {
            result = claimValue;
        }
        return true;
    }
}

Then, we can access the claim values using a prefix of the actual XML namespace of the claim type:

protected void Page_Load(object sender, EventArgs e)
{
    var user = Context.GetUserFromClaims();
    var id = user.Id;
    var username = user.Name; // "John Doe"
}

When hooking up the application to Windows Azure AppFabric Access Control Service (ACS), which claims that the application will receive depends on your rule group configuration. Here are some examples:

If you set up role claim rules in ACS, you can even get information about the user’s roles so that you can do authorization in the code:

protected void Page_Load(object sender, EventArgs e)
{
    var user = Context.GetUserFromClaims();
    var isAdmin = user.IsInRole("administrator"):
}

An alternative to doing this two-step process of first writing to the local disk and then transfer the logs to Azure blob storage, is to write the log entries directly to Azure table storage (or in principal, to Azure blob storage for that matter). This is what I will do here.

Each log entry that the application writes will be a single row in a table in the Azure Table Storage. The log message itself and various meta data about it will be inserted into separate columns in the table. In order to achieve this, we first create a class that represents each entry in the table:

public class LogEntry : TableServiceEntity
{
    public LogEntry()
    {
        var now = DateTime.UtcNow;
        PartitionKey = string.Format("{0:yyyy-MM}", now);
        RowKey = string.Format("{0:dd HH:mm:ss.fff}-{1}", now, Guid.NewGuid());
    }
    #region Table columns
    public string Message { get; set; }
    public string Level { get; set; }
    public string LoggerName { get; set; }
    public string Domain { get; set; }
    public string ThreadName { get; set; }
    public string Identity { get; set; }
    public string RoleInstance { get; set; }
    public string DeploymentId { get; set; }
    #endregion
}

Note that the PartitionKey is the current year and month, and the RowKey is a combination of the date, time and a GUID. This is done to make the querying of the log entries efficient for our purpose. So, the next thing we need to do is to create a class that represents the table storage service. It needs to inherit from TableServiceContext:

internal class LogServiceContext : TableServiceContext
{
    public LogServiceContext(string baseAddress, StorageCredentials credentials) : base(baseAddress, credentials) {}
    internal void Log(LogEntry logEntry)
    {
        AddObject("LogEntries", logEntry);
        SaveChanges();
    }
    public IQueryable<LogEntry> LogEntries
    {
        get
        {
            return CreateQuery<LogEntry>("LogEntries");
        }
    }
}

The method that we will actually use in our code is the Log method which takes a LogEntry instance and persists it to table storage. What we need next, is to create a new Appender for Log4Net which interacts with the table store to store the log entries:

public class AzureTableStorageAppender : AppenderSkeleton
{
    public string TableStorageConnectionStringName { get; set; }
    private LogServiceContext _ctx;
    private string _tableEndpoint;
    public override void ActivateOptions()
    {
        base.ActivateOptions();
        var cloudStorageAccount =
            CloudStorageAccount.Parse(RoleEnvironment.GetConfigurationSettingValue(TableStorageConnectionStringName));
        _tableEndpoint = cloudStorageAccount.TableEndpoint.AbsoluteUri;
        CloudTableClient.CreateTablesFromModel(typeof(LogServiceContext), _tableEndpoint, cloudStorageAccount.Credentials);
        _ctx = new LogServiceContext(cloudStorageAccount.TableEndpoint.AbsoluteUri, cloudStorageAccount.Credentials);
    }
    protected override void Append(LoggingEvent loggingEvent)
    {
        Action doWriteToLog = () => {
            try
            {
                _ctx.Log(new LogEntry
                {
                    RoleInstance = RoleEnvironment.CurrentRoleInstance.Id,
                    DeploymentId = RoleEnvironment.DeploymentId,
                    Timestamp = loggingEvent.TimeStamp,
                    Message = loggingEvent.RenderedMessage,
                    Level = loggingEvent.Level.Name,
                    LoggerName = loggingEvent.LoggerName,
                    Domain = loggingEvent.Domain,
                    ThreadName = loggingEvent.ThreadName,
                    Identity = loggingEvent.Identity
                });
            }
            catch (DataServiceRequestException e)
            {
                ErrorHandler.Error(string.Format("{0}: Could not write log entry to {1}: {2}",
                    GetType().AssemblyQualifiedName, _tableEndpoint, e.Message));
            }
        };
        doWriteToLog.BeginInvoke(null, null);
    }
}

In the code above, the actually writing to the log is done asynchronically in order to prevent the log write to slow down the request handling. We are now done with all the coding. What is left is to use our new AzureTableStorageAppender. Here is the log4net.config:

<log4net>
  <appender name="AzureTableStoreAppender" type="Demo.Log4Net.Azure.AzureTableStorageAppender, Demo.Log4Net.Azure">
    <param name="TableStorageConnectionStringName" value="Log4Net.ConenctionString" />
  </appender>
  <root>
    <level value="DEBUG" />
    <appender-ref ref="AzureTableStoreAppender" />
  </root>
</log4net>

Notice the TableSTorageConnectionStringName attribute of the param element in the configuration. It corresponds to the property of the same name in the AzureTableStorageAppender. Furthermore, take take notice that it’s value is 'Log4Net.ConnectionString', which corresponds to a custom configuration setting that we will add to ServiceDefinition.csdef file:

<ServiceDefinition ...>
  <WebRole ...>
    <ConfigurationSettings>
      <Setting name="Log4Net.ConenctionString"/>
    </ConfigurationSettings>
    ...
  </WebRole>
</ServiceDefinition>

We also need to give the Log4Net.ConfigurationString setting a value in the ServiceConfiguration.cscfg file. It should be a connection string that points to the storage account to use for storing the log entries. In this example, let’s use the development storage:

<ServiceConfiguration ...>
  <Role ...>
    <ConfigurationSettings>
      <Setting name="Log4Net.ConenctionString" value="UseDevelopmentStorage=true"/>
    </ConfigurationSettings>
  </Role>
</ServiceConfiguration>

…and that’s it. You should now find the log entries in the table storage:

You can find the code for this example on GitHub. Suggestions for improvement are very welcome.