Tuesday 30 September 2008

Getting Started With MSBuild

I tried using MSBuild for the first time this week, and took a couple of wrong turnings along the way. Here's a brief summary of how I got started and how to perform a few basic tasks...

What is it?

MSBuild is the build system used by Visual Studio to compile your .NET projects. Check out this nice introduction to MSBuild on CodeProject. The .csproj files you may be familiar with from Visual Studio are in fact nothing more than MSBuild project files.

Why use it?

There are a few reasons why you might want to use MSBuild directly in preference to simply compiling your solution in Visual Studio:

  • You can use it on a dedicated build machine without the need to install Visual Studio.
  • You can use it to batch build several Visual Studio solutions.
  • You can use it to perform additional tasks such as creating installers, archiving, retrieving from source control etc.

It is the third reason that appeals to me, as I wanted to automate the creation of zip files for source code and runtime binaries, as well as run some unit tests.

Creating a project file

MSBuild project files are simply XML files. I created master.proj and put it in the root of my source code folder, alongside my .sln file.

Build Target

The first code I put in to my project file was an example I found on the web that performed a Clean followed by a Build. Here is a slightly modified version of the project file:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="3.5" DefaultTargets="Build"  
    xmlns="http://schemas.microsoft.com/developer/msbuild/2003">

  <PropertyGroup>
    <Configuration Condition=" '$(Configuration)' == '' ">Debug</Configuration>
    <DeploymentProject>MyApplication</DeploymentProject>
    <OutputDirectory>$(DeploymentProject)\bin\$(Configuration)</OutputDirectory>
  </PropertyGroup>

  <Target Name="Clean">    
    <RemoveDir Directories="$(OutputDirectory)" 
            Condition="Exists($(OutputDirectory))"></RemoveDir>
  </Target>
  <Target Name="Build">
    <MSBuild 
      Projects="$(DeploymentProject)\MyApplication.csproj"
      Properties="Configuration=$(Configuration)" >      
    </MSBuild>
  </Target>
</Project>

I ran the build script using the following command line: MSBuild master.proj

It deleted all my source code! Aarrghh!! Fortunately I had a recent backup available.

The reason was that I had not put a default value for Configuration in the PropertyGroup, and my OutputDirectory property was missing the bin from its folder path. Note to self: always backup before running any script that can delete files for the first time.

Zip Target

After my unfortunate experience with clean, I was more determined than ever to create a target that would perform a source-code backup for me! To use the Zip task in MSBuild you first need to download and install MSBuild Community Tasks.

Then you create an ItemGroup defining which files you want to be included (and excluded) before using the Zip task to actually create the archive. Here's the two targets I created - one to make a release package, and one to make a source code archive:

<Target Name="Package" DependsOnTargets="Build">
    <ItemGroup>
      <!-- All files from build -->
      <ZipFiles Include="$(DeploymentProject)\bin\$(Configuration)\**\*.*"
         Exclude="**\*.zip;**\*.pdb;**\*.vshost.*" />
    </ItemGroup>
    <Zip Files="@(ZipFiles)"
         WorkingDirectory="$(DeploymentProject)\bin\$(Configuration)\"
         ZipFileName="$(ApplicationName)-$(Configuration)-$(Version).zip"
         Flatten="True" />
    </Target>

    <Target Name="Backup">
    <ItemGroup>
      <!-- All source code -->
      <SourceFiles Include="**\*.*" 
        Exclude="**\bin\**\*.*;**\obj\**\*.*;*.zip" />
    </ItemGroup>
    <Zip Files="@(SourceFiles)"
         WorkingDirectory=""
         ZipFileName="$(ApplicationName)-SourceCode-$(Version).zip" />
</Target>

See Ben Hall's blog for another example.

Test Task

My final task was to get some NUnit tests running. Again the MSBuild Community Tasks are required for this. It took me a while to get this working correctly, particularly because it seemed to have trouble detecting where my copy of NUnit was installed. Here's the XML:

<Target Name="Test" DependsOnTargets="Build">
    <NUnit Assemblies="@(TestAssembly)"
           WorkingDirectory="MyApplication.UnitTests\bin\$(Configuration)"
           ToolPath="C:\\Program Files\\NUnit 2.4.7\\bin"
           />
</Target>

Conclusion

It took me longer than I wanted to work out how to do these basic tasks with MSBuild, but even so, I am sure the time will be very quickly recouped as I will be able to reuse most of these tasks on future projects. The jury is still out on whether it is preferable to use MSBuild to NAnt though.

Friday 26 September 2008

Don't Repeat Your Threading Code

Often when writing Windows Forms applications I will need to perform a long-running action on a background thread. So I write some code that can kick a delegate off in another thread (probably using the ThreadPool) and then some more code that handles when the action has completed. Then there is the code that handles any exceptions that were raised during that background action. Then there is the code that marshals back to the GUI thread so I can actually update controls on the form. Then I often end up adding a framework for reporting progress, and for aborting the action.

The end result is that the actual code that implements the task at hand has been obscured by a huge pile of threading related lines of code. And very often this code gets replicated every time a new action that needs to run on a background thread is written.

So I decided to see how far I could go with a DRY approach - ("Don't Repeat Yourself"), and decided to create a base BackgroundAction class that would hide a lot of the heavy lifting.

The first step was to create an interface, IBackgroundAction. The key components are the Begin method, that starts the action, and the Finished event, which indicates that the action has finished (successfully or not). I also put in a basic Progress reporting mechanism, as well as a way to request a Cancel of the action.

public interface IBackgroundAction
{
    void Begin();
    event EventHandler<ActionFinishedEventArgs> Finished;
    event EventHandler<ProgressEventArgs> Progress;
    void Cancel();
    ActionState State { get; }
}

public class ActionFinishedEventArgs : EventArgs
{
    public ActionState State { get; set; }
    public Exception Exception { get; set; }
}

public class ProgressEventArgs : EventArgs
{
    public string Message { get; set; }
}

public enum ActionState
{
    None,
    InProgress,
    Success,
    Cancelled,
    Error
}

The next step was to create an abstract base class, BackgroundAction that implements the interface, and does as much of the work as is possible:

abstract class BackgroundAction : IBackgroundAction
{
    protected volatile bool cancel;
    ActionState state = ActionState.None;
    public event EventHandler<ActionFinishedEventArgs> Finished;
    public event EventHandler<ProgressEventArgs> Progress;

    public void Begin()
    {
        if (state != ActionState.None)
        {
            throw new InvalidOperationException("Begin should only be called once");
        }
        state = ActionState.InProgress;
        ThreadPool.QueueUserWorkItem(new WaitCallback(ThreadProc));
    }

    void ThreadProc(object state)
    {
        try
        {
            PerformBackgroundAction();
            state = cancel ? ActionState.Cancelled : ActionState.Success;
            ReportFinished();
        }
        catch (Exception e)
        {
            state = ActionState.Error;
            if (Finished != null)
            {
                Finished(this, new ActionFinishedEventArgs() { State = State, Exception = e });
            }
        }
    }

    protected abstract void PerformBackgroundAction();

    public void Cancel()
    {
        cancel = true;
    }

    public ActionState State
    {
        get { return state; }
    }

    void ReportFinished()
    {
        if (Finished != null)
        {
            Finished(this, new ActionFinishedEventArgs() { State = state });
        }
    }

    protected void ReportProgress(string message, params object[] args)
    {
        if (Progress != null)
        {
            Progress(this, new ProgressEventArgs() { Message = String.Format(message, args) });
        }
    }
}

This class manages the launching of the background action and the reporting of its completion (whether successful or not). It also includes a ReportProgress function as a helper for the concrete class to use.

Moving all this threading related code into a base class means that whenever we create a new action, the code we write is much cleaner, and only has the additional concerns of cancel checking and reporting progress. Here's an example from the project I was trying this out on. All I had to do was override the PerformBackgroundAction function to do the task at hand...

class DeserializeAllAction : BackgroundAction
{
    IEnumerable<BinarySearchResult> searchResults;

    public DeserializeAllAction(IEnumerable<BinarySearchResult> 
                                searchResults)
    {
        this.searchResults = searchResults;
    }

    protected override void PerformBackgroundAction()
    {
        ReportProgress("Preparing to deserialize");
        int count = 0;
        foreach (BinarySearchResult result in searchResults)
        {
            if (cancel)
            {
                break;
            }
            if (++count % 10 == 0)
            {
                ReportProgress("Deserialized {0}", count);
            }
            result.Deserialize();
        }
    }
}

The next step was to create a Progress Form that could take any IBackgroundAction, and run it, while showing progress updates and allowing the user to cancel. Here's a simple implementation:

public partial class ProgressForm : Form
{
    IBackgroundAction backgroundAction;
    bool finished;
    
    public ProgressForm(string title, IBackgroundAction 
                        backgroundAction)
    {
        this.backgroundAction = backgroundAction;
        backgroundAction.Finished += new 
            EventHandler<ActionFinishedEventArgs>
               (backgroundAction_Finished);
        backgroundAction.Progress += new 
            EventHandler<ProgressEventArgs>
                (backgroundAction_Progress);
        InitializeComponent();
        this.FormClosing += new 
            FormClosingEventHandler(ProgressForm_FormClosing);
        this.labelProgressMessage.Text = String.Empty;
        this.Text = title;
    }

    void ProgressForm_FormClosing(object sender, 
       FormClosingEventArgs e)
    {
        if (!finished)
        {
            backgroundAction.Cancel();
            buttonCancel.Enabled = false;
            e.Cancel = true;
        }
    }

    void backgroundAction_Progress(object sender, 
       ProgressEventArgs e)
    {
        if (this.InvokeRequired)
        {
            this.BeginInvoke(new 
               EventHandler<ProgressEventArgs>
                  (backgroundAction_Progress), sender, e);
            return;
        }
        labelProgressMessage.Text = e.Message;
    }

    void backgroundAction_Finished(object sender, 
        ActionFinishedEventArgs e)
    {
        if (this.InvokeRequired)
        {
            this.BeginInvoke(new  
               EventHandler<ActionFinishedEventArgs>
                  (backgroundAction_Finished), sender, e);
            return;
        }
        if(e.State == ActionState.Error)
        {
            MessageBox.Show(this, e.Exception.Message,
                this.Title,
                MessageBoxButtons.OK, MessageBoxIcon.Warning);
        }
        finished = true;
        this.Close();
    }

    private void ProgressForm_Load(object sender, EventArgs e)
    {
        backgroundAction.Begin();
    }
}

Now the code to create and run my background action becomes trivial:

private void buttonDeserializeAll_Click(object sender, EventArgs e)
{
    DeserializeAllAction deserializer = new DeserializeAllAction(searchResults);
    ProgressForm progressForm = new ProgressForm("Deserializing all results", deserializer);
    progressForm.ShowDialog(this);
}

So I was quite pleased with what I had achieved, but I started wondering whether I could take things one step further. In our PerformBackgroundAction function, we still have to perform periodic checks for cancellation and progress updates. These can't be moved down into the base class because it doesn't know when it can check and what it can report.

But then I considered the fact that most long-running background operations are in fact working their way through some kind of list. If I could insert the progress reporting and cancel checking into the enumerator, the PerformBackgroundAction function could focus entirely on doing the action at hand.

Here's my ProgressEnumerator class that injects progress reporting and cancel checking into any IEnumerable type. It calls back once every time round the loop, allowing a progress message to be emitted (if required), and allowing cancellation to be signalled (by returning true from the callback):

delegate bool ProgressCallback<T>(T value, int count);

class ProgressEnumerator<T>
{
    public static IEnumerable<T> Create(IEnumerable<T> items, ProgressCallback<T> callback)
    {
        ProgressEnumerator<T> enumerator = new ProgressEnumerator<T>(items, callback);
        return enumerator.Items;
    }

    IEnumerable<T> items;
    ProgressCallback<T> callback;
    int count;

    public ProgressEnumerator(IEnumerable<T> items, ProgressCallback<T> callback)
    {
        this.items = items;
        this.callback = callback;
        count = 0;
    }

    public IEnumerable<T> Items
    {
        get
        {
            foreach (T item in items)
            {
                if(callback(item,++count))
                    break;
                yield return item;
            }
        }
    }
}

Now we can revisit our PerformBackgroundAction function and remove the progress reporting and cancellation checking from inside the foreach loop, and moving it out into a separate function.

protected override void PerformBackgroundAction()
{
    ReportProgress("Preparing to deserialize");
    foreach (BinarySearchResult result in 
        ProgressEnumerator<BinarySearchResult>.Create(searchResults,ProgressCallback))
    {
        result.Deserialize();
    }
}

bool ProgressCallback(BinarySearchResult result, int count)
{
    if (count % 10 == 0)
    {
        ReportProgress("Deserialized {0}", count);
    }
    return cancel;
}

Finally I have achieved what I wanted. The GUI code that launches the background action is simple, and the code that performs the background action - the real 'business logic' - is unencumbered with threading related noise.

I'm sure that both my BackgroundAction and ProgressEnumerator classes have lots of ways in which they can be improved or streamlined further. Feel free to critique them in the comments.