NoDriver calling acmFormatSuggest playing MP3s with NAudio

One question that commonly gets asked with NAudio is, “Why do I get a NoDriver calling acmFormatSuggest error when playing MP3s?” The answer is that you don’t have the ACM MP3 decoder installed. If you’re running a regular desktop version of windows from XP all the way up to 8.1, then the Fraunhofer MP3 ACM decoder is installed as standard and everything should just work.

But if you’re running a server version of Windows (or maybe one of the “N” versions of Windows sold in Europe), then you might be missing the codecs you need. There are various workarounds to this:

Option 1: Install Desktop Experience

If you are running Windows Server, make sure you install the “Desktop Experience” component of Windows. This will install all the standard codecs, for audio playback (including Media Foundation support as well). To install on Azure, this script may be helpful.

Option 2: Use the DMO MP3 Frame Decompressor

The NAudio Mp3FileReader class allows you to inject an alternative MP3 frame decompressor, so you don’t need to use ACM if you don’t want to. To use the DMO MP3 frame decompressor for example, use the following code:

new Mp3FileReader(stream,wave=> new DmoMp3FrameDecompressor(wave));

Option 3: Use MediaFoundationReader Instead

With NAudio 1.7, the MediaFoundationReader class was introduced, which can not only play MP3 files, but lots of other formats too. In many ways, this should now be the preferred way of playing MP3s with NAudio. You will still need the Desktop Experience installed if you’re on Windows Server though.

Option 4: Use a fully managed MP3 Decoder

Another option is to switch to fully managed MP3 decoder such as NLayer. This has the benefit of working on all platforms and requiring no codecs.

Option 5: Find and install another ACM MP3 Codec

Finally, you may be here not because you’re playing MP3, but because you tried to play a WAV file containing some codec that your system cannot find an ACM driver for. This also results in the “NoDriver calling acmFormatSuggest error”. What you need to do is search the web for an ACM codec that can decode the format you are using. Unfortunately this is not always a simple process, but unless you have a codec installed that matches the compression type of the file you’re trying to play, you won’t be able to play it.

Announcing Audio Test Files Repository

Today I announce what is quite possibly the most exciting new open source project of the millennium. Or at least moderately interesting. If you like listening to the same bit of sound over and over at different sample rates and bit depths that is.

And that is exactly what is on offer in my brand new Audio Test Files project on CodePlex. For some time I have wanted to store a collection of brief test files in a wide variety of audio formats for the NAudio unit tests to make use of. But since these files would add up to at least 200Mb, I didn’t want to put them in the main NAudio source control.

The repository has only just begun with basic WAV files encoded with PCM 8 bit, 16 bit and 24 bit plus IEEE float all at several sample rates and in mono and stereo. To preview the stunning and original musical composition that is available in multiple encodings, you can have a listen in the embedded player below. Hold onto your hats:

The future will see such thrilling additions as MP3, MID, AIFF plus a whole smorgasbord of telephony codecs. Also, I plan to add useful test signals such as sine wave sweeps and white noise. There will also be files containing unusual or incorrect data structures.

Having these files will facilitate the quick manual testing that NAudio is still able to play the wide variety of audio that it currently supports. I will also run unit tests to check I can open and successfully read to the end of each file. Sadly with something like audio, the manual element of testing is impossible to avoid, but I would like to automate as much as possible.

I welcome submissions to this project. The files must of course not be copyrighted, and I am not looking for large files, just representative samples of each format. Read the submission guidelines first.

How to Play Streaming MP3 Using NAudio

One of the most common support requests I get for NAudio is how it can be used for streaming MP3. Whilst all the requisite bits and pieces are present in NAudio, there has been no good example of how to do this. For NAudio 1.5 I will be rectifying this to include an example in the NAudioDemo application, and have made a few enhancements to the classes in NAudio to make this easier to implement. In this article I will explain how streaming MP3 playback is implemented in the NAudioDemo application.

Design Overview

The basic design for our streaming playback is that one thread will be downloading the audio, while another thread will play it back. One of the issues this raises is which thread should do the decompression. I have chosen that the download thread should do so. This takes work away from the playback thread and so should give additional protection against stuttering playback, but has the down-side that it requires more memory for buffered audio since it is stored in PCM.

Buffering Audio

The key class that enables us to queue up samples on one thread for playback on another is called the BufferedWaveProvider. The BufferedWaveProvider implements IWaveProvider so it can be used as an input to any Wave Output device. When its Read function is called it returns any bytes buffered, or zeroes the playback buffer if none are available. The AddSamples function is called from the downloader thread to queue up samples as they become available.

Under the hood, the BufferedWaveProvider in NAudio 1.4 used a queue of buffers to store the data. For NAudio 1.5 I am switching to a thread safe circular buffer. This means that memory is allocated once up front, and should result in better performance. By default it buffers enough for five seconds of audio, but this can be easily changed using BufferDuration (although once you start reading, the buffer size is fixed). It also makes more sense to track the number of buffered bytes (a helper method, BufferedDuration, turns this into a TimeSpan) instead of the number of queued buffers.

Parsing MP3 Frames

One if the issues that people quickly run into when attempting to play back streaming MP3 is that the MP3Frame class will throw an exception if it can’t read enough bytes to fully load an MP3 frame. Obviously while you are streaming you may well have half an MP3 frame available.

The solution is fairly simple. We wrap our network stream in a ReadFullyStream, which is just a simple class inheriting from System.IO.Stream that will not return from its Read method until it has read the number of bytes requested or the source stream reaches its end. The ReadFullyStream currently resides in the NAudioDemo project, but may be incorporated as a helper utility for NAudio 1.5.

Decompressing Frames

Once you have parsed a complete MP3 frame, it is time to decompress it so we can buffer the PCM audio. NAudio includes a choice of two MP3 frame decompressors, one using the ACM codecs, which should be present on Windows XP onwards, and one for using the DMO (DirectX Media Object) decoder, which is available with Windows Vista onwards. These two classes were internal with NAudio 1.4, but will be made public for NAudio 1.5. You could alternatively use a fully managed MP3 decoder, such as NLayer which is my C# port of the Java MP3 decoder JavaLayer. If I get time I will update NAudioDemo to allow you to select between the available frame decompressors.

Decompression Format

One of the trickiest design issues to work around is the fact that we don’t know the WaveFormat that the BufferedWaveProvider should output until we have read the first MP3 frame. Most MP3s decompress to stereo 44.1kHz, but some are 48kHz or 22.1kHz, and others are mono. We therefore can’t open the soundcard yet because we don’t know what PCM format we will be supplying it with.

There are two ways you can tackle this problem. First you can choose a PCM format (e.g. 44.1kHz stereo) that you will convert the decompressed MP3 frames to even if they are of a different sample rate or channel count. Alternatively, as I have chosen for the demo, you hold back from creating the BufferedWaveProvider until you have received the first MP3 frame and therefore know what format you will convert to.

In the NAudioDemo project this is handled by a timer running on the form periodically checking to see if we have a BufferedWaveProvider yet. Once the BufferedWaveProvider appears, we can initialise our WaveOut player.

Smooth Playback

One of the problems with playing back from a network stream is that we may get stuttering audio if the MP3 file is not downloading fast enough. Although the BufferedWaveProvider will continuously provide data to the soundcard, it could be very choppy if you are experiencing slow download speeds as there would be gaps between each decompressed frame.

For this reason, it is a good idea for the playback thread to go into pause when the buffered audio has run out, but not start playing again until there are at least a couple of seconds of buffered audio. This is accomplished in the NAudioDemo with a simple timer on the GUI thread firing four times a second and deciding if we should go into our come out of pause.

End of Stream

If you are streaming from internet radio, then there is no end of stream, you just keep listening until you have had enough. However, if you are streaming an MP3 file, it will come to an end at some point.

The first consequence is that our MP3Frame constructor will throw an EndOfStream exception because it was trying to read a new MP3Frame and got to the end of the stream. Handling this is straightforward – the downloader thread can capture this exception and exit, knowing we have got everything. I am considering improving the API for the MP3Frame to include a TryParse option so you can deal with this case without the need for catching exceptions.

The other issue is that the playback thread needs to know that the end of the file has been reached, otherwise it will go into pause, waiting for more audio to be buffered that will never arrive. A simple way to deal with this is to set a flag indicating that the whole stream has been read and that we no longer need to go into a buffering state.

Try It Yourself

The code is all checked in and will be part of NAudio 1.5. To try it yourself, go to the NAudio Source Code tab and download the latest code. Here’s a screenshot of the NAudioDemo application playing from an internet radio stream:

Merging MP3 Files with NAudio in C# and IronPython

If you would like to concatenate MP3 files using NAudio, it is quite simple to do. I recommend getting the very latest source code and building your own copy of NAudio, as this will work best with some of the changes that are in preparation for NAudio 1.4.

Here’s the C# code for a function that takes MP3 filenames, and writes a combined MP3 to the output stream:

public static void Combine(string[] inputFiles, Stream output)
{
    foreach (string file in inputFiles)
    {
        Mp3FileReader reader = new Mp3FileReader(file);
        if ((output.Position == 0) && (reader.Id3v2Tag != null))
        {
            output.Write(reader.Id3v2Tag.RawData, 0, reader.Id3v2Tag.RawData.Length);
        }
        Mp3Frame frame;
        while ((frame = reader.ReadNextFrame()) != null)
        {
            output.Write(frame.RawData, 0, frame.RawData.Length);
        }
    }
}

And here’s an IronPython script (just put NAudio.dll in the same folder as the mp3merge.py script):

import clr
clr.AddReference('NAudio.dll')

import sys
from NAudio.Wave import Mp3FileReader
from System.IO import File

def GetAllFrames(reader):
    while True:
        frame = reader.ReadNextFrame()
        if frame:
            yield frame
        else:
            return

def Merge(files, outputStream):
    for file in files:
        with Mp3FileReader(file) as reader:
            if reader.XingHeader:
                print 'discarding a Xing header'
            if not outputStream.Position and reader.Id3v2Tag:
                outputStream.Write(reader.Id3v2Tag.RawData, 0, reader.Id3v2Tag.RawData.Length)                
            for frame in GetAllFrames(reader):
                outputStream.Write(frame.RawData, 0, frame.RawData.Length);
            
if __name__ == '__main__':
    if len(sys.argv) < 3:
        print "Usage: ipy mp3merge.py output.mp3 File1.mp3 File2.mp3"
    else:
        with File.OpenWrite(sys.argv[1]) as outStream:
            Merge(sys.argv[2:],outStream)

Notes:

I simply copy across the ID3v2 tag from the first MP3 file if present. All other ID3v2 tags are discarded (as are ID3v1 tags). Also, I discard the Xing frame from VBR files. It could easily be re-included if desired, although it’s information will not necessarily be valid about the combined MP3 file. One final thing, I wouldn’t recommend merging MP3 files of different sample rates, or mixing mono with stereo, as it could cause various players issues.

State of MP3 Playback Support in NAudio

The MP3 playback support in NAudio was always rather experimental. The ACM conversion code I had written assumed CBR (constant bit rate) and constant block sizes. With MP3s this is not always the case, since there are VBR files with variable block sizes, and even in CBR MP3 files, padding means you can get frames of different sizes. However, despite these issues I did manage to get MP3 more or less playing back, which was cool, but not 100% reliable. People ran into issues like the occasional error while repositioning a stream, or the more irritating fact that Mp3FileReader was not good at calculating the duration of a file.

In this post I will go over some of the key challenges to getting good MP3 playback support, with details of some recent changes I have checked in, along with some ideas for the future.

1. Correctly parsing MP3 frame headers

To work effectively with MP3 files you need to be able to parse frame headers correctly and determine their exact size. If this cannot be done, we have no choice but to pass blocks of MP3 file directly to the decoder without knowing whether we are giving it whole frames or not.

Finding out how to properly parse MP3 frame headers was a much harder challenge than it seemed. Googling for info on MP3 frames reveals some articles that look authoritative but simply failed to correctly parse everything I threw at them. Mono or low sample rate MP3s got their frame size calculation wrong. However, eventually I found this article, whose source code had the final missing piece of information that allowed me to get it reliable.

2. A single frame decoder

Once we can calculate frame sizes reliably, we are able to decode them one by one. The WaveFormatConversionStream assumes everywhere it is working with CBR, so I have removed it from the equation. Now a simple MP3 Frame Decoder class (final name and interface to be decided) is used to decode MP3 frames one at a time using ACM. Alternative frame decoders could easily be plugged in if required in the future (e.g. using DMO or NLayer).

The really big change is that this means that the MP3FileReader no longer returns MP3 data in its Read method, but emits PCM. This makes life so much easier downstream and simplifies the playback graph considerably (no more BlockAlignReductionStream). I’ve made the ReadFrame method public so if you have a pressing need to get the compressed data out instead there is nothing stopping you.

3. Accurate length reporting

Accurate length reporting was never possible before, since it relied on an estimate of the bitrate. But now we can parse MP3 frames, we have accurate knowledge of exactly how many samples each frame will decompress into, and the TotalTime property (and CurrentTime property) of MP3FileReader should be entirely accurate. n.b. I think that it may be possible that the first frame in a VBR MP3 file decompresses to zero samples (although I already exclude the Xing frame so maybe there is another similar meta-data type frame), so we might actually very slightly over-report the length – I’ll need to look into that.

4. Repositioning to frame granularity

When you reposition with the MP3FileReader.Position property, there is of course every possibility that you will ask for it to reposition to a place midway through a frame. We now automatically move you to the start of the frame that contains the position you asked for.

An earlier NAudio contributor had done some cool stuff with a BinarySearch to speed this up. I needed to drop this temporarily as I was making changes to the table of contents generation. However, there is no reason why this could not be reinstated now, to speed up performance further (although repositioning perf doesn’t seem to be a major issue with the tests I have done on 1 hour long MP3s).

5. Repositioning with sample granularity

Obviously, it would be even nicer to support MP3 repositioning with sample granularity. This would involve us decompressing a frame during the seek process, so when the next Read occurs we can read from part-way through that frame. The framework to do that is already in place (we keep track of “leftovers”, so this could be a feature I add in the not too distant future).

6. Forward only

One thing I haven’t got round to doing yet, is making it possible to use MP3FileReader from an input stream that doesn’t support repositioning. Obviously, it would not be able to work out its Length, and there would be no real need for a TOC. Proper support for forward only streams would be useful to people wanting to do network streamed playback, which seems to be one of the most common queries I get.

7. Changes of Sample Rate and Number of Channels

It is theoretically possible within an MP3 file for the sample rate and number of channels to change from frame to frame. However, I have no immediate plans to support this since I’ve never seen an MP3 file that does this. The only scenario in which I could imaging this would be if someone was attempting to concatenate two MP3 files by simply copying frames from one into the other.

8. ID3 Tag Support

I have no plans to introduce ID3 tag reading or writing, since there are other open source libraries out there that do this perfectly well.

Give me feedback

Please grab the latest NAudio code from Codeplex and let me know how you get on with it. As always, the best way to give it a run-through is to use the NAudioDemo app that is included in the solution. Load it up, select WAV playback and try it with whatever MP3s you have lying around on your hard disk. It would be great to have robust MP3 playback as a headline feature for NAudio 1.4.