Sunday, 18 December 2011

NAudio 1.5 Released

I have finally got round to releasing the long overdue version 1.5 of NAudio. There are loads of bugfixes and improvements, so I recommend upgrading if possible. Here’s the highlights.

  • Now available on NuGet!
  • Numerous bugfixes mean we are now working fully in x64 as well as x86, so NAudio.dll is now marked as AnyCPU. (You can still force x86 by marking your own executable as x86 only.)
  • WaveOutEvent – a new WaveOut mode with event callback, highly recommended instead of WaveOut with function callbacks
  • 24 bit ASIO driver mode (LSB)
  • Float LSB ASIO driver mode
  • WaveFileWriter has had a general code review and API cleanup
  • Preview of new ISampleProvider interface making it much easier to write custom 32 bit IEEE (float) audio pipeline components, without the need to convert to byte[]. Lots of examples in NAudioDemo of using this and more documentation will follow in future.
  • Several ISampleProvider implementations to get you started. Expect plenty more in future NAudio versions:
    • PanningSampleProvider
    • MixingSampleProvider
    • MeteringSampleProvider
    • MonoToStereoSampleProvider
    • NotifyingSampleProvider
    • Pcm16BitToSampleProvider
    • Pcm8BitToSampleProvider
    • Pcm24BitToSampleProvider
    • SampleChannel
    • SampleToWaveProvider
    • VolumeSampleProvider
    • WaveToSampleProvider
  • Added AiffFileReader courtesy of Giawa
  • AudioFileReader to simplify opening any supported file, easy volume control, read/reposition locking
  • BufferedWaveProvider uses CircularBuffer instead of queue (less memory allocations)
  • CircularBuffer is now thread-safe
  • MP3Frame code cleanup
  • MP3FileReader throws less exceptions
  • ASIOOut bugfixes for direct 16 bit playback
  • Some Demos added to NAudioDemo to give simple examples of how to use the library
    • NAudioDemo has an ASIO Direct out form, mainly for testing the AsioOut class at different bit depths (still recommended to convert to float before you get there).
    • NAudioDemo has simple MP3 streaming form (play MP3s while they download)
    • NAudioDemo has simple network streaming chat application
    • NAudioDemo playback form uses MEF to make it much more modular and extensible (new output drivers, new file formats etc)
    • NAudioDemo can play aiff
  • GSM 6.10 ACM codec support
  • DSP Group TrueSpeech ACM codec support
  • Fully managed G.711 a-law and mu-law codecs (encode & decode)
  • Fully managed G.722 codec (encode & decode)
  • Example of integration with NSpeex
  • Fix to PlaybackStopped using SyncContext for thread safety
  • Obsoleted IWavePlayer.Volume (can still set volume on WaveOut directly if you want)
  • Improved FFT display in WPF demo
  • WaveFileReader - tolerate junk after data chunk
  • WaveOut constructor detects if no sync context & choose func callbacks
  • WaveOut function mode callbacks hopefully chased out the last of the hanging bugs (if in a WaveOutWrite at same time as WaveOutReset, bad things happen - so need locks, but if WaveOutReset called during a previous func callback that is about to call waveOutWrite we deadlock)
  • Now has an msbuild script allowing me to more easily create releases, run tests etc
  • Now using Mercurial for source control, hopefully making bug fixing old releases and accepting user patches easier. n.b. this unfortunately means all old submitted patches are no longer available for download on the CodePlex page.
  • WPF Demo enhancements:
    • WPF Demo is now .NET 4, allowing us to use MEF, and will be updated hopefully with more examples of using NAudio.
    • WPF Demo uses windowing before FFT for a more accurate spectrum plot
    • WPF Demo has visualization plugins, allowing me to trial different drawing mechanisms
    • WPF Demo has a (very basic) drum machine example

I also intend that this will be the last NAudio that targets .NET 2.0 (1.6 will be .NET 3.5). Let me know if you have any objections.

Hope you have fun using it, and do send me the links to any cool stuff you make with it.

Tuesday, 29 November 2011

Creating Zip Files with IronPython

I’ve been working on automating some of the parts of the release process for NAudio, as I always seem to forget something or make a mistake. One of the tasks was to create a “demo” zip file containing the demo applications together with their supporting files.

I initially attempted to do this with MSBuild and the Zip task from MSBuild Community Extensions, but I ended up double-adding a number of files, as well as struggling to get exactly the right folder structure.

This is exactly the sort of task that Python excels at, and Python comes with the zipfile module built in, meaning that the script I wrote is not IronPython specific. Here’s what I came up with:

import zipfile
import os

folders = ['AudioFileInspector','NAudioDemo','NAudioWpfDemo']
files = {}

def exclude(filename):
    return filename.endswith('.pdb') or ('nunit' in filename)

for folder in folders:
    fullpath = folder + "\\bin\\debug\\"
    for filename in os.listdir(fullpath):
        if not exclude(filename):
            files[filename] = fullpath + filename

zip = zipfile.ZipFile("BuildArtefacts\\test.zip", "w")

for filename, fullpath in files.iteritems():
    if os.path.isdir(fullpath):
        for subfile in os.listdir(fullpath):
            zip.write(fullpath + "\\" + subfile, filename + "\\" + subfile)
    else:
        zip.write(fullpath, filename)

zip.close()

There's not a lot to it really. I first build up a dictionary containing the files I want in my zip, using the filename to exclude duplicates. Then I use the write method on zipfile to specify the file I want to add, and the folder it belongs in.

My Python skills are a bit rusty, so the code above would probably benefit from being refactored a little, but as you can see, it is very easy, and much simpler than fighting MSBuild to make it do what I want.

Monday, 11 July 2011

Screencast: Modular WPF with MEF & MVVM Tutorial Part 2

In Part 1, I showed how to create a very simple modular WPF application, and introduced MEF to allow us to easily add modules without changing any existing code.

In part 2, I introduce the MVVM pattern to show how we can make our applications more easily unit testable, by separating the view from the business logic. I also create some unit tests using the Microsoft unit testing framework and moq, and show how to perform code coverage analysis.

Incidentally, I normally use NUnit and TestDriven.NET instead of the Microsoft unit testing framework, which I find much simpler to use, and also supports code coverage analysis using NCover or the Visual Studio code coverage.

I do plan to follow this up with a third video, in which I replace the modules ListBox with some buttons to switch modules and show some of the typical problems you might run into when using MVVM.

Saturday, 9 July 2011

10 C# keywords you should be using

Most developers who learn C# pick up the basic keywords quite quickly. Within a few weeks of working with a typical codebase you’ll have come across around a third of the C# keywords, and understand roughly what they do. You should have no trouble explaining what the following keywords mean:

public, private, protected, internal, class, namespace, interface, get, set, for, foreach .. in, while, do, if, else, switch, break, continue, new, null, var, void, int, bool, double, string, true, false, try, catch

However, while doing code reviews I have noticed that some developers get stuck with a limited vocabulary of keywords and never really get to grips with some of the less common ones, and so miss out on their benefits. So here’s a list, in no particular order, of some keywords that you should not just understand, but be using on a semi-regular basis in your own code.

is & as

Sometimes I come across a variation of the following code, where we want to cast a variable to a different type but would like to check first if that cast is valid:

if (sender.GetType() == typeof(TextBox))
{
   TextBox t = (TextBox)sender;
   ...
}

While this works fine, the is keyword could be used to simplify the if clause:

if (sender is TextBox)
{
   TextBox t = (TextBox)sender;
   ...
}

We can improve things further by using the as keyword, which is like a cast, but doesn’t throw an exception if the conversion is not valid – it just returns null instead. This means we can write code in a way that doesn’t require the .NET framework to check the type of our sender variable twice:

TextBox t = sender as TextBox;
if (t != null)
{
   ...
}

I feel obliged to add that if your code contains a lot of casts, you are probably doing something wrong, but that is a discussion for another day.

using

Most developers are familiar with the using keyword for importing namespaces, but a surprising number do not make regular use of it for dealing with objects that implement IDisposable. For example, consider the following code:

var writer = new StreamWriter("test.txt");
writer.WriteLine("Hello World");
writer.Dispose();

What we have here is a potential resource leak if there is an exception thrown between opening the file and closing it. The using keyword ensures that Dispose will always be called if the writer object was successfully created.

using (var writer = new StreamWriter("test.txt"))
{
   writer.WriteLine("Hello World");
}

Make it a habit to check whether the classes you create implement IDisposable, and if so, make use of the using keyword.

finally

Which brings us onto our next keyword, finally. Even developers who know and use using often miss appropriate scenarios for using a finally block. Here’s a classic example:

public void Update()
{   
    if (this.updateInProgress)
    {    
        log.WriteWarning("Already updating");    
        return;
    } 
    this.updateInProgress = true;
    ...
    DoUpdate();
    ...
    this.updateInProgress = false;

}

The code is trying to protect us from some kind of re-entrant or multithreaded scenario where an Update can be called while one is still in progress (please ignore the potential race condition for the purposes of this example). But what happens if there is an exception thrown within DoUpdate? Now we are never able to call Update again because our updateInProgress flag never got unset. A finally block ensures we can’t get into this invalid state:

public void Update()
{   
    if (this.updateInProgress)
    {    
        log.WriteWarning("Already updating");    
        return;
    } 
    try
    {
        this.updateInProgress = true;
        ...
        DoUpdate();
        ...
    }
    finally
    {
        this.updateInProgress = false;
    }
}

readonly

OK, this one is a fairly simple one, and you could argue that code works just fine without it. The readonly keyword says that a field can only be written to from within the constructor. It’s handy from a code readability point of view, since you can immediately see that this is a field whose value will never change during the lifetime of the class. It also becomes a more important keyword as you begin to appreciate the benefits of immutable classes. Consider the following class:

public class Person
{
    public string FirstName { get; private set; }
    public string Surname { get; private set; }

    public Person(string firstName, string surname)
    { 
        this.FirstName = firstName;
        this.Surname = surname;
    }
}

Person is certainly immutable from the outside – no one can change the FirstName or Surname properties. But nothing stops me from modifying those properties within the class. In other words, my code doesn’t advertise that I intend this to be an immutable class. Using the readonly keyword, we can express our intent better:

public class Person
{
    private readonly string firstName;
    private readonly string surname;

    public string FirstName { get { return firstName; } }
    public string Surname { get { return surname; } }

    public Person(string firstName, string surname)
    { 
        this.firstName = firstName;
        this.surname = surname;
    }
}

Yes, it’s a shame that this second version is a little more verbose than the first, but it makes it more explicit that we don’t want firstName or surname to be modified during the lifetime of the class. (Sadly C# doesn’t allow the readonly keyword on properties).

yield

This is a very powerful and yet rarely used keyword. Suppose we have a class that searches our hard disk for all MP3 files and returns their paths. Often we might see it written like this:

public List<string> FindAllMp3s()
{
   var mp3Paths = List<string>();
   ... 
   // fill the list
   return mp3Paths;
}

Now we might use that method to help us search for a particular MP3 file we had lost:

foreach(string mp3File in FindAllMp3s())
{
   if (mp3File.Contains("elvis"))
   {
       Console.WriteLine("Found it at: {0}", mp3File);
       break;
   }  
}

Although this code seems to work just fine, it’s performance is sub-optimal, since we first find every MP3 file on the disk, and then search through that list. We could save ourselves a lot of time if we checked after each file we found and aborted the search at that point.

The yield keyword allows us to fix this without changing our calling code at all. We modify FindAllMp3s to return an IEnumerable<string> instead of a List. And now every time it finds a file, we return it using the yield keyword. So with some rather contrived example helper functions (.NET 4 has already added a method that does exactly this) our FindAllMp3s method looks like this:

public IEnumerable<string> FindAllMp3s()
{
   var mp3Paths = List<string>();
   
   for (var dir in GetDirs())
   {
       for (var file in GetFiles(dir))
       {
           if (file.EndsWith(".mp3")
           {
               yield return file;
           }
       }
   }      
}

This not only saves us time, but it saves memory too, since we now don’t need to store the entire collection of mp3 files in a List.

It can take a little while to get used to debugging this type of code since you jump in and out of a function that uses yield repeatedly as you walk through the sequence, but it has the power to greatly improve the design and performance of the code you write and is worth mastering.

select

OK, this one is cheating since this is a whole family of related keywords. I won’t attempt to explain LINQ here, but it is one of the best features of the C# language, and you owe it to yourself to learn it. It will revolutionise the way you write code. Download LINQPad and start working through the tutorials it provides.

interface

So you already know about this keyword. But you probably aren’t using it nearly enough. The more you write code that is testable and adheres to the Dependency Inversion Principle, the more you will need it. In fact at some point you will grow to hate how much you are using it and wish you were using a dynamic language instead. (dynamic is itself another very interesting new C# keyword, but I feel that the C# community is only just beginning to discover how we can best put it to use).

throw

You do know you are allowed to throw as well as catch exceptions right? Some developers seem to think that a function should never let any exceptions get away, and so contain a generic catch block which writes an error to the log and returns giving the caller no indication that things went wrong.

This is almost always wrong. Most of the time your methods should simply allow exceptions to propagate up to the caller. If you are using the using keyword correctly, you are probably already doing all the cleanup you need to.

But you can and should sometimes throw exceptions. An exception thrown at the point you realise something is wrong with a good error message can save hours of debugging time.

Oh, and if you really do need to catch an exception and re-throw it, make sure you use do it the correct way.

goto

Only joking, pretend you didn’t see this one. Just because a keyword is in the language, doesn’t mean it is a good idea to use it. out and ref usually fall into this category too – there are better ways to write your code.

Friday, 8 July 2011

Screencast: Modular WPF with MEF & MVVM Tutorial Part 1

A while ago I began to write a blog tutorial about how to create a modular WPF application using MVVM and MEF. I have decided to present it as a screencast instead, so here is the first part. It uses WPF, but a lot of the techniques I show apply equally well to Silverlight.

My approach is not to start off using MVVM and MEF (or any of the more fully featured WPF frameworks), but to introduce these libraries and design patterns at the point at which they make sense. Hopefully this will make it clearer why we might actually want to use them in the first place.

In this first video I create a very simple application with a menu allowing you to switch between two modules, and show how MEF enables us to work around violations of the “Open Closed Principle”.

I’ve got at least one more episode planned which hopefully will appear shortly and introduce MVVM to make our application more easily testable. If there is demand, I may also post my notes and upload my mercurial repository to bitbucket.

Another reason for presenting this as a screencast is that I am planning to create some NAudio screencasts in the future, and so I need to get some practice in. This one is a little raw as I didn’t do a practice beforehand, but I have tried to keep things flowing along at a reasonable pace. I also know the sound quality isn’t brilliant. Let me know if you think the resolution is acceptable.

Wednesday, 6 July 2011

One Language to Rule them All

You may have heard the story of the “tower of Babel”. The story goes that one day, the people of the earth decided to build a great tower. It would be a monument to the greatness of humanity. But God objected to their pride and intervened to thwart their building program. His technique was not to strike the tower down with an earthquake, or strike the builders down with illness. Instead, he opted for a simple yet effective solution: he “confused the language” and the building project was soon abandoned. Inability to communicate doomed the project to failure.

This ancient tale is a remarkably fitting parable for the state of modern programming. Even the smallest of miscommunications, such as requirements not properly understood, or two components in the system using different units of measure can cause the premature demise of entire software projects.

The analogy works on a wider scale too. Imagine what we as a software development community could build if we shared a common programming language. All the wastage of ‘porting’ libraries from one framework to another would be eliminated. Instead of reinventing the same development tools over and over for every new language, we could focus on genuine innovation.

The Last Programming Language

And this is the point made by Robert C Martin at the NDC 2011 conference, in a fascinating talk entitled “Clojure – The Last Programming Language”.

He begins by arguing that we’ve already fully explored the domain of programming languages. In other words, we have experimented with all the types of languages that there are. Or to put it another way, the new languages we keep inventing are just refinements or rearrangements of ideas from existing languages. It’s a slightly depressing thought in some ways – expect no new revolutionary paradigms – we’ve tried it all already.

Whether or not he is right about his first claim, his proposal for the development community to standardise on a single language certainly grabbed my attention. Before dismissing it out of hand as a pipe dream, it is worth pondering the many benefits it would bring.

  • You would not need to hire a C#/Java/Ruby/C programmer. You’d just need to hire a programmer.
  • All platforms and frameworks would be built in that language. No need for all this NUnit, JUnit, PyUnit, Runit business. We’d build on top of what had gone before instead of continually having to reinvent it for every new language.
  • Articles with example code would be immediately understandable to all developers

Can we agree on anything?

But could developers really agree on one programming language? The idea seems almost preposterous. Uncle Bob counters that other industries have gone through the same transition (e.g. biologists, chemists, medicine, mathematics). Eventually the benefits of a lingua franca win through despite the attachment people inevitably feel to their native tongue.

He also points out that this already happened once with C, which is one of those few languages that can genuinely be used to develop on almost every platform. This for me is the big sticking point in seeing Uncle Bob’s vision become a reality. The one language must be able to develop for every platform, server, client, embedded, handheld. It must be usable for every type of development – desktop apps, websites, games, services, device drivers, nuclear power stations. I’m not sure such a language exists.

Features of the final language

But let’s suspend disbelief for a moment and ask what features the final language should have. Uncle Bob’s wishlist included features such as:

  • Not controlled by a corporation.
  • Garbage collected. (I have mixed feelings on this one given .NET’s inability to do low latency audio well)
  • Polymorphic
  • Runs on a virtual machine
  • Able to modify itself at runtime like Ruby (homoiconic)
  • Pared down syntax
  • Functional
  • Hybrid, i.e. Multi-paradigm (support, don’t enforce paradigms)
  • Simple
  • Provide access to existing frameworks
  • Structured (no goto)
  • Fast
  • Textual
  • Dynamically typed (the talk includes an interesting discussion on this).

His proposed language was Clojure. It seems an interesting enough language, although I can’t say I warm to all the parentheses.

A step in the right direction…

In any case, I think that if his vision were to become a reality, we would first have to pick a virtual machine. The two most obvious candidates are the JVM and the CLR, both of which now support a whole host of languages (Clojure can run on either). The CLR may even have the upper hand due to products like Mono which makes it a genuine open source option and available on a very broad range of platforms, although I suspect its ties with Microsoft would generate a lot of resistance.

Libraries that are compiled for a virtual machine like the CLR effectively appear as native libraries for all languages that compile to the same byte code, meaning that there is at least some reduction in the amount of reinvention and relearning required when you switch languages – the libraries can come with you even if the language syntax doesn’t. Another benefit is that often the byte code for a VM can be translated into other languages by a tool, allowing at least some level of automated translation between languages. Maybe someone could invent a browser plugin that automatically converts any code you view in a web-page into the language of your choice – a kind of Google translate for programming languages.

What do you think? Could we agree on a virtual machine, and then get to work on picking a smaller subset of languages to program it with? Or is Uncle Bob’s idea a pipedream? Will we ever get to one common language or will God step in and mix it all up again, just to keep us humble.

Tuesday, 5 July 2011

Test Resistant Code #5–Threading

I want to wrap up my test-resistant code series with one final type of code that proves hard to test, and that is multi-threaded code. We’ll just consider two scenarios, one that proves easy to test, another that proves very complex.

Separate out thread creation

A common mistake is to include the code that creates a thread (or queues a background worker) in the same class that contains the code for the actual work to be performed by the thread. This is a violation of “separation of concerns” In the following trivial example, the function we really need to unit test, DoStuff, is private, and the public interface is not helpful for unit testing.

public void BeginDoStuff()
{
    ThreadPool.QueueUserWorkItem((o) => DoStuff("hello world"));
}

private void DoStuff(string message)
{
    Console.WriteLine(message);
}

Fixing this is not hard. We separate the concerns by making the DoStuff method a public member of a different class, leaving the original class simply to manage the asynchronous calling and reporting of results (which you may find can be refactored into a more generic threading helper class).

Locks and race conditions

But what about locking? Consider a very simple circular buffer class I wrote for NAudio. In the normal use case, one thread writes bytes to it while another reads from it. Here’s the current code for the Read and Write methods (which I’m sure could be refactored down to something much shorter):

/// <summary>
/// Write data to the buffer
/// </summary>
/// <param name="data">Data to write</param>
/// <param name="offset">Offset into data</param>
/// <param name="count">Number of bytes to write</param>
/// <returns>number of bytes written</returns>
public int Write(byte[] data, int offset, int count)
{
    lock (lockObject)
    {
        int bytesWritten = 0;
        if (count > buffer.Length - this.byteCount)
        {
            count = buffer.Length - this.byteCount;
        }
        // write to end
        int writeToEnd = Math.Min(buffer.Length - writePosition, count);
        Array.Copy(data, offset, buffer, writePosition, writeToEnd);
        writePosition += writeToEnd;
        writePosition %= buffer.Length;
        bytesWritten += writeToEnd;
        if (bytesWritten < count)
        {
            // must have wrapped round. Write to start
            Array.Copy(data, offset + bytesWritten, buffer, writePosition, count - bytesWritten);
            writePosition += (count - bytesWritten);
            bytesWritten = count;
        }
        this.byteCount += bytesWritten;
        return bytesWritten;
    }
}

/// <summary>
/// Read from the buffer
/// </summary>
/// <param name="data">Buffer to read into</param>
/// <param name="offset">Offset into read buffer</param>
/// <param name="count">Bytes to read</param>
/// <returns>Number of bytes actually read</returns>
public int Read(byte[] data, int offset, int count)
{
    lock (lockObject)
    {
        if (count > byteCount)
        {
            count = byteCount;
        }
        int bytesRead = 0;
        int readToEnd = Math.Min(buffer.Length - readPosition, count);
        Array.Copy(buffer, readPosition, data, offset, readToEnd);
        bytesRead += readToEnd;
        readPosition += readToEnd;
        readPosition %= buffer.Length;

        if (bytesRead < count)
        {
            // must have wrapped round. Read from start
            Array.Copy(buffer, readPosition, data, offset + bytesRead, count - bytesRead);
            readPosition += (count - bytesRead);
            bytesRead = count;
        }

        byteCount -= bytesRead;
        return bytesRead;
    }
}

The fact that I take a lock for the entirety of both methods makes me confident that the internal state will not get corrupted by one thread calling Write while another calls Read; the threads simply have to take it in turns. But what if I had a clever idea for optimising this code that only involved me locking for part of the time. Maybe I want to do the Array.Copy’s outside the lock since they potentially take the longest. How could I write a unit test that ensured my code remained thread-safe?

Short of firing up two threads reading and writing with random sleep times inserted here and there, I’m not sure I know how best to prove the correctness of this type of code. Locking issues and race conditions can be some of the hardest to track down bugs. I once spent a couple of weeks locating a bug that only manifest itself on a dual processor system (back in the days when those were few and far between). The code had been thoroughly reviewed by all the top developers at the company and yet no one saw the problem.

Here’s another example, based on some code I saw in a product I worked on. A method kicks off two threads to do some long-running tasks and attempts to fire a finished event when both have completed. We want to ensure that the SetupFinished event always fires, and only fires once. You might be able to spot a race condition by examining the code, but how would we write a unit test to prove we had fixed it?

private volatile bool eventHasBeenRaised;

public void Init()
{    
    ThreadPool.QueueUserWorkItem((o) => Setup1());
    ThreadPool.QueueUserWorkItem((o) => Setup2());    
}

private void Setup1()
{
    Thread.Sleep(500);
    RaiseSetupFinishedEvent();
}

private void Setup2()
{
    Thread.Sleep(500);
    RaiseSetupFinishedEvent();
}

private void RaiseSetupFinishedEvent()
{
    if (!eventHasBeenRaised)
    {
        eventHasBeenRaised = true;
        SetupFinished(this, EventArgs.Empty);
    }
}

The only tool for .NET I have heard of that might begin to address this shortcoming is Microsoft CHESS. It seems a very promising tool although it seems to have stalled somewhat – the only integration is with VS2008; VS2010 is not supported. I’d love to hear of other tools or clever techniques for unit testing multi-threaded code effectively. I haven’t delved too much into the C# 5 async stuff yet, but I’d be interested to know how well it plays with unit tests.