Wednesday 4 May 2011

How to Play Streaming MP3 Using NAudio

One of the most common support requests I get for NAudio is how it can be used for streaming MP3. Whilst all the requisite bits and pieces are present in NAudio, there has been no good example of how to do this. For NAudio 1.5 I will be rectifying this to include an example in the NAudioDemo application, and have made a few enhancements to the classes in NAudio to make this easier to implement. In this article I will explain how streaming MP3 playback is implemented in the NAudioDemo application.

Design Overview

The basic design for our streaming playback is that one thread will be downloading the audio, while another thread will play it back. One of the issues this raises is which thread should do the decompression. I have chosen that the download thread should do so. This takes work away from the playback thread and so should give additional protection against stuttering playback, but has the down-side that it requires more memory for buffered audio since it is stored in PCM.

Buffering Audio

The key class that enables us to queue up samples on one thread for playback on another is called the BufferedWaveProvider. The BufferedWaveProvider implements IWaveProvider so it can be used as an input to any Wave Output device. When its Read function is called it returns any bytes buffered, or zeroes the playback buffer if none are available. The AddSamples function is called from the downloader thread to queue up samples as they become available.

Under the hood, the BufferedWaveProvider in NAudio 1.4 used a queue of buffers to store the data. For NAudio 1.5 I am switching to a thread safe circular buffer. This means that memory is allocated once up front, and should result in better performance. By default it buffers enough for five seconds of audio, but this can be easily changed using BufferDuration (although once you start reading, the buffer size is fixed). It also makes more sense to track the number of buffered bytes (a helper method, BufferedDuration, turns this into a TimeSpan) instead of the number of queued buffers.

Parsing MP3 Frames

One if the issues that people quickly run into when attempting to play back streaming MP3 is that the MP3Frame class will throw an exception if it can’t read enough bytes to fully load an MP3 frame. Obviously while you are streaming you may well have half an MP3 frame available.

The solution is fairly simple. We wrap our network stream in a ReadFullyStream, which is just a simple class inheriting from System.IO.Stream that will not return from its Read method until it has read the number of bytes requested or the source stream reaches its end. The ReadFullyStream currently resides in the NAudioDemo project, but may be incorporated as a helper utility for NAudio 1.5.

Decompressing Frames

Once you have parsed a complete MP3 frame, it is time to decompress it so we can buffer the PCM audio. NAudio includes a choice of two MP3 frame decompressors, one using the ACM codecs, which should be present on Windows XP onwards, and one for using the DMO (DirectX Media Object) decoder, which is available with Windows Vista onwards. These two classes were internal with NAudio 1.4, but will be made public for NAudio 1.5. You could alternatively use a fully managed MP3 decoder, such as NLayer which is my C# port of the Java MP3 decoder JavaLayer. If I get time I will update NAudioDemo to allow you to select between the available frame decompressors.

Decompression Format

One of the trickiest design issues to work around is the fact that we don’t know the WaveFormat that the BufferedWaveProvider should output until we have read the first MP3 frame. Most MP3s decompress to stereo 44.1kHz, but some are 48kHz or 22.1kHz, and others are mono. We therefore can’t open the soundcard yet because we don’t know what PCM format we will be supplying it with.

There are two ways you can tackle this problem. First you can choose a PCM format (e.g. 44.1kHz stereo) that you will convert the decompressed MP3 frames to even if they are of a different sample rate or channel count. Alternatively, as I have chosen for the demo, you hold back from creating the BufferedWaveProvider until you have received the first MP3 frame and therefore know what format you will convert to.

In the NAudioDemo project this is handled by a timer running on the form periodically checking to see if we have a BufferedWaveProvider yet. Once the BufferedWaveProvider appears, we can initialise our WaveOut player.

Smooth Playback

One of the problems with playing back from a network stream is that we may get stuttering audio if the MP3 file is not downloading fast enough. Although the BufferedWaveProvider will continuously provide data to the soundcard, it could be very choppy if you are experiencing slow download speeds as there would be gaps between each decompressed frame.

For this reason, it is a good idea for the playback thread to go into pause when the buffered audio has run out, but not start playing again until there are at least a couple of seconds of buffered audio. This is accomplished in the NAudioDemo with a simple timer on the GUI thread firing four times a second and deciding if we should go into our come out of pause.

End of Stream

If you are streaming from internet radio, then there is no end of stream, you just keep listening until you have had enough. However, if you are streaming an MP3 file, it will come to an end at some point.

The first consequence is that our MP3Frame constructor will throw an EndOfStream exception because it was trying to read a new MP3Frame and got to the end of the stream. Handling this is straightforward – the downloader thread can capture this exception and exit, knowing we have got everything. I am considering improving the API for the MP3Frame to include a TryParse option so you can deal with this case without the need for catching exceptions.

The other issue is that the playback thread needs to know that the end of the file has been reached, otherwise it will go into pause, waiting for more audio to be buffered that will never arrive. A simple way to deal with this is to set a flag indicating that the whole stream has been read and that we no longer need to go into a buffering state.

Try It Yourself

The code is all checked in and will be part of NAudio 1.5. To try it yourself, go to the NAudio Source Code tab and download the latest code. Here’s a screenshot of the NAudioDemo application playing from an internet radio stream:

NAudio demo playing an internet radio station

25 comments:

Anonymous said...

I have a couple of questions about streaming mp3 audio using naudio. What is the best method of contacting you and asking you those questions.

Unknown said...

@Anonymous, post them on this blog, or on the NAudio Codeplex discussion forum, or StackOverflow.com tagged with [naudio] if they are general questions that other NAudio users may be able to answer

Doan Ngoc Nam said...

Hi Mark.

I'm trying to set position of this streaming to play but I dont know how to do that. Is there anyway to do that?

Thanks

Unknown said...

@1712 when you play streaming audio, you just play audio as it arrives. You could buffer what you receive to reposition backwards, but to reposition forwards you would need to skip over data you were receiving.

Doan Ngoc Nam said...

Can you show me which code I must change to do that? (V1.5)

And one more question, when "fullyDownload", how can I replay without reload data from server?

Thanks :)

Gian said...

Hi Mark,
I'm new in audio management and in NAudio too.
I'm trying to play an AAC raw audio from an IP cam using NAudio.
Audio has a bitrate=32000, mono.
I tried to create a custom format using Mpeg Format without success).
Using NAudioDemo I've seen AAC ACM codec is installed.
I always has "NoDriver calling acmFormatSuggest" exception.
A code snippet:

WaveFormat format = WaveFormat.CreateCustomFormat(WaveFormatEncoding.Mpeg, 32000, 1, 32000, 1, 16);
using (WaveStream blockAlignedStream = new BlockAlignReductionStream(WaveFormatConversionStream.CreatePcmStream(new RawSourceWaveStream(ms, format))))

Thank you in advance for your help.
Gianluca

Unknown said...

@Gianluca, AAC could only be played if an AAC ACM codec was installed on your system. Even then you would also need to know what WaveFormat structure it was expecting to be able to select it correctly. "Mpeg" is probably not right.

Anonymous said...

Hi, is it possible to play a stream that isn't mp3 with this library?? if it is can you point me the right direction to achieve this? Thanks

Unknown said...

you can only stream formats that NAudio can decompress. So this usually means you need an ACM codec installed for it.

Dan Cri said...

Hi Mark. Most internet streams use shoutcast/AAC, is there any chance to include an NAudio Demo module to handle this? Thanks.

Unknown said...

@Dan, NAudio can't play AAC at the moment. That's a feature I'd really like to add, but it is not available using ACM codecs, so I'd need to learn how to wrap the Media Foundation Transforms to do this.

As for Shoutcast, I haven't got any plans at the moment to support that I'm afraid.

Jonx said...

Hi Mark,
Thank you for your hard work over all those years ;)

One question about your comment about playing AAC. You say above that AAC can be played using the ACM codecs but later say the ACM codecs do not permit this. Can you please clarify?
I would like to read from AAC files and transcode them to WAV/PCM. Can this be done with plain Naudio? When existing ACM codecs (where do I get them) or cannot be done?
Thank you for your lights,
John.

Anonymous said...

I have been trying to work out how to stream radio and show a waveform. I have been playing around with your streaming demo but can't get a waveform displaying correctly, the nearest I have got is showing a flat line. Any suggestions on how best to achieve this!

Sajjad Shadow said...

Thanks so much
I have a more question.
can I save or cache the buffered playing stream?

thanks again.

Unknown said...

@sajjad, you could easily also write incoming MP3 frames to a file if you want. The BufferedWaveProvider is effectively an in-memory cache. You can increase its duration if you like.

Anonymous said...

Hi Mark,
Thank you very much for great library and your hard work.
Can you also please provide some video tutorials for Demo programs which you created, it will be helpful for these who is new in .Net and audio programing to understand the examples.

Anonymous said...

Hi,

nice post. Can I play streaming videos using Naduio. Can I get the source code for playing streaming mp3 audios.

Anonymous said...

i have a question:

if i have a file with 50% downloaded,
can i download/stream only the left 50% while playing it?

Unknown said...

you'd need to buffer what you'd downloaded somewhere, and to save space, I'd buffer the compressed audio

Anonymous said...

Hi,

You mean i buffer the exising 50% mp3 to stream and play, at the same time buffer the network stream and then append it to the player?

Anonymous said...

Thank you very much.
Sakis

Unknown said...

please Sir can you give the source code for "How to Play Streaming MP3 Using NAudio" article... atleast reply me SIR :)

Unknown said...

please sir at least help me to play, pause, stop a mp3 file from web address...my email is sufiyan@engineer.com... i have used the last answer code from http://stackoverflow.com/questions/184683/play-audio-from-a-stream-using-c-sharp?lq=1 please help me sir.

Unknown said...

the source code is all on CodePlex, at the NAudio site, inside the NAudioDemo project.

You might actually find that it is much easier if all you want to do is stream from a URL, to simply use the MediaElement control from WPF, and set the source to point to that URL.

Scully said...

@Mark H, can you dynamically update the playlist with NAudio? I would like to add to the playlist dependant on some outside factors as and when it happens