Thursday, 19 May 2011

NAudio and the PlaybackStopped Problem

The IWavePlayer interface in NAudio features an event called PlaybackStopped. The original idea behind this event was simple: you start playing a file (e.g. pass a WaveFileReader into WaveOut.Init and then call Play), and when it reaches its end you are informed, allowing you to dispose the input stream and playback device if you wanted to, or maybe start playing something else.

The reality was not quite so simple, and I have ended up trying to discourage users of the NAudio library from making use of the PlaybackStopped event. In this post I will explain some of the reasons behind that and explain how I hope to restore the usefulness of PlaybackStopped in future versions of NAudio.

The Never-Ending Stream Problem

The way that each implementor of IWavePlayer determines whether it should automatically stop playback is when the Read method on the source IWaveProvider returns 0 (In fact Read should always return the count parameter unless the end of the stream has been reached).

However, there are some WaveStreams / IWaveProviders in NAudio that never stop returning audio. This isn’t a bug – it is quite normal behaviour for some scenarios. Perhaps BufferedWaveProvider is the best example – it will return a zero-filled buffer if there is not enough queued data for playback. This is useful for streaming from the network where data might not be arriving fast enough but you don’t want playback to stop. Similarly WaveChannel32 has a property called PadWithZeroes allowing you to turn this behaviour on or off, which can be useful for scenarios where you are feeding into a mixer.

The Threading Problem

This is one of the trickiest problems surrounding the PlaybackStopped event. If you are using WaveOut, the chances are you are using windowed callbacks, which means that the PlaybackStopped event is guaranteed to fire on the GUI thread. Since only one thread makes calls to the waveOut APIs, it is also completely safe for the event handler to make other calls into waveOut, such as calling Dispose, or starting a new playback.

However, with DirectSound and WASAPI, the PlaybackStopped event is fired from a background thread we create. Even more problematic are WaveOut function callbacks and ASIO, where the event is raised from a thread from deep within the OS / soundcard device driver. If you make any calls back into the driver in the handler for the PlaybackStopped event you run the risk of deadlocks or errors. You also don’t want to give the user a chance to do anything that might take a lot of time in that context.

This problem almost caused me to remove PlaybackStopped from the IWavePlayer interface altogether. But I have decided to see if I can give it one last lease of life by using the .NET SynchronizationContext class. The SynchronizationContext class allows us easily in both WPF and WinForms to invoke the PlaybackStopped event on the GUI thread. This greatly reduces the chance of something you do in the handler causing a problem.

The Has it Really Finished Problem

The final problem with PlaybackStopped is another tricky one. How do you know when playback has stopped? You know when you have reached the end of the source file, since the Read method returns 0. And you know when you have given the last block of audio to the soundcard. But the audio may not have finished playing yet, particularly if you are working at high latency. The old WaveOut implementation in particular was guilty of raising PlaybackStopped too early.

One workaround requiring no changes to the existing IWavePlayer implementations would be to create a LeadOutWaveProvider class, deriving from IWaveProvider. This would do nothing more than append a specified amount of silence onto the end of your source stream, ensuring that it plays completely. Here’s a quick example of how that could be implemented:

private int silencePos = 0;
private int silenceBytes = 8000;

public int Read(byte[] buffer, int offset, int count)
{
    int bytesRead = source.Read(buffer,offset,count);
    if (bytesRead < count)
    {
        int silenceToAdd = Math.Min(silenceBytes – silencePos, count – bytesRead);
        Array.Zero(buffer,offset+bytesRead,silenceToAdd);
        bytesRead += silenceToAdd;
        silencePos += silenceToAdd;
    }
    return bytesRead;
}

Goals for NAudio 1.5

Fixing all the problems with PlaybackStopped may not be fully possible in the next version, but my goals are as follows:

  • Every implementor of IWavePlayer should raise PlaybackStopped (this is done)
  • PlaybackStopped should be automatically invoked on the GUI thread if at all possible using SynchronizationContext (this is done)
  • It should be safe to call the Dispose of IWavePlayer in the PlaybackStopped handler (currently testing and bugfixing)
  • PlaybackStopped should not be raised until all audio from the source stream has finished playing (done for WaveOut – we now raise it when there are no queued buffers, will need to code review other classes to decide if this is the case).
  • Keep the number of never-ending streams/providers in NAudio to a minimum and try to make it very clear which ones have this behaviour.

Tuesday, 17 May 2011

How to use the TFS API to retrieve a historical version of your project

I recently had the need to retrieve several old versions of a project from TFS. Fortunately, the TFS API allows you to automate this process.

First you need to connect to the TFS Server:

TfsTeamProjectCollection tpc = new TfsTeamProjectCollection(new Uri("http://tfsserver:8080/"));
tpc.EnsureAuthenticated();

Next we will set up a workspace to use. I check first that the workspace doesn’t exist.

VersionControlServer vcs = tpc.GetService<VersionControlServer>(); //.GetAllTeamProjects(false);
string workspaceName = "temp";
string workspaceOwner = "mheath"; // your user name
// protect ourselves from using a workspace in use for something else\
Assert.Throws<WorkspaceNotFoundException>(() => vcs.GetWorkspace(workspaceName, workspaceOwner), "The workspace already exists");
var workspace = vcs.CreateWorkspace(workspaceName, workspaceOwner);

// use the workspace...

// when we are done with it... 
vcs.DeleteWorkspace(workspaceName, workspaceOwner);

Now we need to map our workspace to a local temporary folder. We can use the server path to select only a subset of the code in the project (for example a single branch):

string localPath = @"D:\TFS\Temp";
string requestPath = "$/MyProject/Dev/Source";

// protect ourselves from accidentally overwriting something important
Assert.IsFalse(Directory.Exists(localPath), "directory should not exist before this test is run");
Directory.CreateDirectory(localPath);

workspace.Map(requestPath, localPath);

Finally we are ready to make a request. We use a VersionSpec to specify which version we want. I am using a DateVersionSpec here, but you can use a ChangeVersionSet or a LabelVersionSet if you prefer. The ItemSpec specifies what to get. We want everything in our request path including subfolders, so recursion is turned on:

var version = new DateVersionSpec(new DateTime(2009, 11, 24));
var item = new ItemSpec(requestPath, RecursionType.Full);
GetRequest getRequest = new GetRequest(item, version);

GetStatus status = workspace.Get(getRequest, GetOptions.GetAll);
Assert.AreEqual(0, status.NumWarnings);
Assert.AreEqual(0, status.NumFailures);

And that is it – it is usually quite quick to retrieve as well (certainly much faster than SourceSafe was!).

Wednesday, 11 May 2011

Announcing Audio Test Files Repository

Today I announce what is quite possibly the most exciting new open source project of the millennium. Or at least moderately interesting. If you like listening to the same bit of sound over and over at different sample rates and bit depths that is.

And that is exactly what is on offer in my brand new Audio Test Files project on CodePlex. For some time I have wanted to store a collection of brief test files in a wide variety of audio formats for the NAudio unit tests to make use of. But since these files would add up to at least 200Mb, I didn’t want to put them in the main NAudio source control.

The repository has only just begun with basic WAV files encoded with PCM 8 bit, 16 bit and 24 bit plus IEEE float all at several sample rates and in mono and stereo. To preview the stunning and original musical composition that is available in multiple encodings, you can have a listen in the embedded player below. Hold onto your hats:

The future will see such thrilling additions as MP3, MID, AIFF plus a whole smorgasbord of telephony codecs. Also, I plan to add useful test signals such as sine wave sweeps and white noise. There will also be files containing unusual or incorrect data structures.

Having these files will facilitate the quick manual testing that NAudio is still able to play the wide variety of audio that it currently supports. I will also run unit tests to check I can open and successfully read to the end of each file. Sadly with something like audio, the manual element of testing is impossible to avoid, but I would like to automate as much as possible.

I welcome submissions to this project. The files must of course not be copyrighted, and I am not looking for large files, just representative samples of each format. Read the submission guidelines first.

Friday, 6 May 2011

How to Convert AIFF files to WAV using NAudio 1.5

Thanks to Giawa, the latest NAudio source code (to be released with NAudio 1.5) has an AIFF File Reader. This makes it very easy to convert from AIFF files to WAV (or simply into PCM format for inputs to other NAudio streams). Here’s how to do it:

public static void ConvertAiffToWav(string aiffFile, string wavFile)
{
    using (AiffFileReader reader = new AiffFileReader(aiffFile))
    {
        using (WaveFileWriter writer = new WaveFileWriter(wavFile, reader.WaveFormat))
        {
            byte[] buffer = new byte[4096];
            int bytesRead = 0;
            do
            {
                bytesRead = reader.Read(buffer, 0, buffer.Length);
                writer.Write(buffer, 0, bytesRead);
            } while (bytesRead > 0);
        }
    }
}

Wednesday, 4 May 2011

How to Play Streaming MP3 Using NAudio

One of the most common support requests I get for NAudio is how it can be used for streaming MP3. Whilst all the requisite bits and pieces are present in NAudio, there has been no good example of how to do this. For NAudio 1.5 I will be rectifying this to include an example in the NAudioDemo application, and have made a few enhancements to the classes in NAudio to make this easier to implement. In this article I will explain how streaming MP3 playback is implemented in the NAudioDemo application.

Design Overview

The basic design for our streaming playback is that one thread will be downloading the audio, while another thread will play it back. One of the issues this raises is which thread should do the decompression. I have chosen that the download thread should do so. This takes work away from the playback thread and so should give additional protection against stuttering playback, but has the down-side that it requires more memory for buffered audio since it is stored in PCM.

Buffering Audio

The key class that enables us to queue up samples on one thread for playback on another is called the BufferedWaveProvider. The BufferedWaveProvider implements IWaveProvider so it can be used as an input to any Wave Output device. When its Read function is called it returns any bytes buffered, or zeroes the playback buffer if none are available. The AddSamples function is called from the downloader thread to queue up samples as they become available.

Under the hood, the BufferedWaveProvider in NAudio 1.4 used a queue of buffers to store the data. For NAudio 1.5 I am switching to a thread safe circular buffer. This means that memory is allocated once up front, and should result in better performance. By default it buffers enough for five seconds of audio, but this can be easily changed using BufferDuration (although once you start reading, the buffer size is fixed). It also makes more sense to track the number of buffered bytes (a helper method, BufferedDuration, turns this into a TimeSpan) instead of the number of queued buffers.

Parsing MP3 Frames

One if the issues that people quickly run into when attempting to play back streaming MP3 is that the MP3Frame class will throw an exception if it can’t read enough bytes to fully load an MP3 frame. Obviously while you are streaming you may well have half an MP3 frame available.

The solution is fairly simple. We wrap our network stream in a ReadFullyStream, which is just a simple class inheriting from System.IO.Stream that will not return from its Read method until it has read the number of bytes requested or the source stream reaches its end. The ReadFullyStream currently resides in the NAudioDemo project, but may be incorporated as a helper utility for NAudio 1.5.

Decompressing Frames

Once you have parsed a complete MP3 frame, it is time to decompress it so we can buffer the PCM audio. NAudio includes a choice of two MP3 frame decompressors, one using the ACM codecs, which should be present on Windows XP onwards, and one for using the DMO (DirectX Media Object) decoder, which is available with Windows Vista onwards. These two classes were internal with NAudio 1.4, but will be made public for NAudio 1.5. You could alternatively use a fully managed MP3 decoder, such as NLayer which is my C# port of the Java MP3 decoder JavaLayer. If I get time I will update NAudioDemo to allow you to select between the available frame decompressors.

Decompression Format

One of the trickiest design issues to work around is the fact that we don’t know the WaveFormat that the BufferedWaveProvider should output until we have read the first MP3 frame. Most MP3s decompress to stereo 44.1kHz, but some are 48kHz or 22.1kHz, and others are mono. We therefore can’t open the soundcard yet because we don’t know what PCM format we will be supplying it with.

There are two ways you can tackle this problem. First you can choose a PCM format (e.g. 44.1kHz stereo) that you will convert the decompressed MP3 frames to even if they are of a different sample rate or channel count. Alternatively, as I have chosen for the demo, you hold back from creating the BufferedWaveProvider until you have received the first MP3 frame and therefore know what format you will convert to.

In the NAudioDemo project this is handled by a timer running on the form periodically checking to see if we have a BufferedWaveProvider yet. Once the BufferedWaveProvider appears, we can initialise our WaveOut player.

Smooth Playback

One of the problems with playing back from a network stream is that we may get stuttering audio if the MP3 file is not downloading fast enough. Although the BufferedWaveProvider will continuously provide data to the soundcard, it could be very choppy if you are experiencing slow download speeds as there would be gaps between each decompressed frame.

For this reason, it is a good idea for the playback thread to go into pause when the buffered audio has run out, but not start playing again until there are at least a couple of seconds of buffered audio. This is accomplished in the NAudioDemo with a simple timer on the GUI thread firing four times a second and deciding if we should go into our come out of pause.

End of Stream

If you are streaming from internet radio, then there is no end of stream, you just keep listening until you have had enough. However, if you are streaming an MP3 file, it will come to an end at some point.

The first consequence is that our MP3Frame constructor will throw an EndOfStream exception because it was trying to read a new MP3Frame and got to the end of the stream. Handling this is straightforward – the downloader thread can capture this exception and exit, knowing we have got everything. I am considering improving the API for the MP3Frame to include a TryParse option so you can deal with this case without the need for catching exceptions.

The other issue is that the playback thread needs to know that the end of the file has been reached, otherwise it will go into pause, waiting for more audio to be buffered that will never arrive. A simple way to deal with this is to set a flag indicating that the whole stream has been read and that we no longer need to go into a buffering state.

Try It Yourself

The code is all checked in and will be part of NAudio 1.5. To try it yourself, go to the NAudio Source Code tab and download the latest code. Here’s a screenshot of the NAudioDemo application playing from an internet radio stream:

NAudio demo playing an internet radio station

Thursday, 21 April 2011

How to Use WaveFileWriter

In this post I will explain how to use the WaveFileWriter class that is part of NAudio. I will discuss how to use it now in NAudio 1.4 and mention some of the changes that will be coming for NAudio 1.5.

The purpose of WaveFileWriter is to allow you to create a standard .WAV file. WAV files are often thought of as containing uncompressed PCM audio data, but actually they can contain any audio compression, and are often used as containers for telephony compression types such as mu-law, ADPCM, G.722 etc.

NAudio provides a one-line method to produce a WAV file if you have an existing WaveStream derived class that can provide the data (in NAudio 1.5 it can be an IWaveProvider).

string tempFile = Path.Combine(Path.GetTempPath(), Guid.NewGuid().ToString() + ".wav");
WaveFormat waveFormat = new WaveFormat(8000, 8, 2);
WaveStream sourceStream = new NullWaveStream(waveFormat, 10000);
WaveFileWriter.CreateWaveFile(tempFile, sourceStream);

In the above example, I am using a simple utility class as my source stream, but in a real application this might be the output of a mixer, or the output from some effects or a synthesizer. The most important thing to note is that the Read method of your source stream MUST eventually return 0, otherwise your file will keep on writing until your disk is full! So beware of classes in NAudio (such as WaveChannel32) that can be configured to always return the number of bytes asked for from the Read method.

For greater control over the data you write, you can simply use the WriteData method (renamed to “Write” in NAudio 1.5, as WaveFileWriter will inherit from Stream). WriteData assumes that you are providing raw data in the correct format and will simply write it directly into the data chunk of the WAV file. This is therefore the most general purpose way of writing to a WaveFileWriter, and can be used for both PCM and compressed formats.

byte[] testSequence = new byte[] { 0x1, 0x2, 0xFF, 0xFE };
using (WaveFileWriter writer = new WaveFileWriter(fileName, waveFormat))
{
    writer.WriteData(testSequence, 0, testSequence.Length);
}

WaveFileWriter has an additional constructor that takes a Stream instead of a filename, allowing you to write to any kind of a stream (for example, a MemoryStream). Be aware though that when you dispose the WaveFileWriter, it disposes the output stream, so use the IgnoreDisposeStream utility class to wrap the output stream if you don’t want that to happen.

One of the most commonly used bit depths for PCM WAV files is 16 bit, and so NAudio provides another WriteData overload (to be called WriteSamples in NAudio 1.5) that allows you to supply data as an array of shorts (Int16s). Obviously, this only really makes sense if you are writing to a 16 bit WAV file, but the current implementation will also try to scale the sample value for different bit depths.

short[] samples = new short[1000];
// TODO: fill sample buffer with data
waveFileWriter.WriteData(samples, 0, samples.Length);

Another consideration is that very often after applying various audio effects (even as simple as changing the volume), the audio samples stored as 32 bit floating point numbers (float or Single). To make writing these to the WAV file as simple as possible, a WriteSample function is provided, allowing you to write one sample at a time. If the underlying PCM format is a different bit depth (e.g. 16 or 24 bits), then the WriteSample function will attempt to convert the sample to that bit depth before writing it to a file. NAudio 1.5 will also feature a WriteSamples function to allow arrays of floating point samples to be written. The following example shows one second of a 1kHz sine wave being written to a WAV file using the WriteSample function:

float amplitude = 0.25f;
float frequency = 1000;

for (int n = 0; n < waveFileWriter.WaveFormat.SampleRate; n++)
{
    float sample = (float)(amplitude * Math.Sin((2 * Math.PI * n * frequency) / waveFileWriter.WaveFormat.SampleRate));
    waveFileWriter.WriteSample(sample);
}

Wednesday, 20 April 2011

NAudio 1.4 Release Notes

It has been far too long coming I know, but I have finally got round to uploading a release candidate for NAudio 1.4. Assuming no glaring bugs are reported in the next week or so, I’ll drop the “RC” from the title.

So what’s new in this latest release?

  • Major interop improvements to support native x64. Please note that I have not in this release changed the dll’s target platform away from x86 only as I don’t personally have an x64 machine to test on. However, we are now in a state where around 95% of the interop should work fine in x64 mode so feel free to recompile for “any CPU”. You should also note that if you do run in native x64 mode, then you probably will find there are no ACM codecs available, so WaveFormatConversionStream might stop working – another reason to stay targetting x86 for now.
  • There have also been major enhancements to MP3 File Reader, which is the main reason for pushing this new release out. Please read this post for more details as this is a breaking change – you no longer need to use a WaveFormatConversionStream or a BlockAlignReductionStream.
  • More examples IWaveProvider implementers have been added, including the particularly useful BufferedWaveProvider which allows you to queue up buffers to be played on demand.
    • BufferedWaveProvider
    • Wave16toFloatProvider
    • WaveFloatTo16Provider
    • WaveInProvider
    • MonoToStereoProvider16
    • StereoToMonoProvider16
    • WaveRecorder
  • The NAudioDemo project has been updated to attempt to show best practices (or at least good practices) of how you should be using these classes.
  • The NAudioDemo project also now demonstrates how to select the output device for WaveOut, DirectSoundOut, WasapiOut and AsioOut.
  • WaveChannel32 can now take inputs of more bit depths – 8, 16, 24 and IEEE float supported. NAudioDemo shows how to play back these files.
  • A general spring clean removed a bunch of obsolete classes from the library.
  • AsioOut more reliable, although I still think there are more issues to be teased out. Please report whether it works on your hardware.
  • WaveFileReader and WaveFileWriter support for 24 and 32 bit samples
  • Allow arbitrary chunks to appear before fmt chunk in a WAV file
  • Reading and writing WAV files with Cues
  • Obsoleted some old WaveFileWriter and WaveFileReader methods
  • Fixed a longstanding issue with WaveOutReset hanging in function callbacks on certain chipsets
  • Added sequencer specific MIDI event
  • RawWaveSourceStream turns a raw audio data stream into a WaveStream with specified WaveFormat
  • A DMO MP3 Frame Decoder as an alternative to the ACM one
  • Easier selection of DirectSound output device
  • WaveOut uses 2 buffers not 3 by default now (a leftover from the original days of NAudio when my PC had a 400MHz Pentium II processor!).
  • Lots more minor bug fixes & patches applied – see the check-in history for full details

Let me take this opportunity to say thank you to those who have offered many feature suggestions, patches and bug reports. It is also very exciting to see NAudio being used for all kinds of cool applications. I am trying to keep the main project page updated with a list of all of them so get in touch and tell me what you are using NAudio for.

I am sorry I have struggled to keep up with all the questions appearing on the NAudio forums and a growing number of stackoverflow NAudio questions. I try to answer everything, but sometimes I get too far behind and some questions have been left unanswered. NAudio is a spare time project for me, and with the recent birth of my fifth child, time is at a premium, so please be patient! Thanks also to those who have helped out by writing tutorials and answering questions (OpenSebJ and Yuval especially).

Should I Upgrade?

If your app works fine then I actually don’t recommend that you update as there are a (small) number of breaking API changes (mostly for those using Mp3FileReader). However, for all new applications, or if you are prepared to make some small code changes, then I recommend you use the latest.

Using NAudio

Using NAudio is as simple as referencing the NAudio DLL and using whatever pieces you need. (I might have a go at creating a NuGet package for NAudio to make things even easier). To find out how to do something, first look at the NAudioDemo application as this showcases a number of core features. There are also plenty of tutorials on this blog, and on the Documentation tab of the CodePlex site. If they don’t give you an answer, feel free to ask a question on the forums or Stack Overflow.

If you feel that NAudio is missing something you need, remember that NAudio is designed to make it as easy as possible to extend its feature set by inheriting from the base classes and interfaces such as WaveStream or IWaveProvider. In many cases you can add the new feature you need in for the outside.

Future

There were of course, lots of things I wanted to get into this release that will have to wait for the next one. Here’s some of my ideas. Please feel free to get in touch and tell me what would be most beneficial to you.

  • Include an effects framework (based on what I prototyped with Skype Voice Changer)
  • Update the MP3 code to more easily support network streaming
  • Make a Silverlight / WP7 DLL containing the parts of NAudio that do not rely on Windows APIs.
  • ASIO recording support.
  • Possibly switch the source control over to Mercurial to make patching easier
  • And I’ll also try to write a few more tutorials here on my blog, as I know many people have complained about the lack of documentation.