Thursday 30 October 2008

MIDI In for BabySmash

I watched Scott Hanselman's presentation at the PDC on BabySmash this afternoon, and was very impressed both with the quality of his presentation skills as well as the sheer coolness of all the new technologies he was using.

As I watched it, I thought to myself how easy it would be to add MIDI in to this application using the NAudio open source .NET audio toolkit I have written. I downloaded the BabySmash source code from CodePlex and it only took about 10 minutes for me to add MIDI in support.

The first thing to do was to listen for MIDI in events on the default device. Thanks to NAudio, this is just a couple of lines of code, called after InitializeComponent.

private void StartMonitoringMidi()
{
    midiIn = new MidiIn(0); // default device
    midiIn.MessageReceived += midiIn_MessageReceived;
    midiIn.Start();
}

Next we needed to handle the MIDI in messages. I only care about NoteOn messages, and need to switch to the despatcher thread before interacting with the GUI:

void midiIn_MessageReceived(object sender, MidiInMessageEventArgs e)
{
    if (this.Dispatcher.Thread == Thread.CurrentThread)
    {
        NoteOnEvent noteOnEvent = e.MidiEvent as NoteOnEvent;
        if (noteOnEvent != null)
        {
            controller.ProcessMidiNoteOn(this, noteOnEvent);
        }
    }
    else
    {
        Dispatcher.BeginInvoke(new EventHandler<MidiInMessageEventArgs>
            (midiIn_MessageReceived), sender, e);
    }
}

The final step is for the new Controller method to somehow convert the note on event into something to be displayed. Obviously we could have lots of fun here, but for this initial demo I wanted to keep it really simple.

public void ProcessMidiNoteOn(FrameworkElement uie, NoteOnEvent noteOn)
{
    AddFigure(uie, new string((char)('A' + noteOn.NoteNumber - 30), 1));
}

And that's all there is to it. Less than 20 lines of code. Here's a brief video of me demonstrating it:

Some notes:

  • The existing audio playback in BabySmash seems problematic and caused hanging on my laptop. I might see if I can replace it with the audio playback mechanism from NAudio. This would allow me to put some drum / musical sounds in and manage the number of concurrent sounds.
  • I didn't change the graphics at all. I had hoped that the MEF plugin architecture demoed in the PDC talk would be there for me to create my own "SmashPack", but it looks like that code hasn't made it onto CodePlex just yet. One advantage of MIDI in is that each input has a velocity (how hard you smashed it) as well as what note you hit. I would like to try out using colour or initial size to indicate velocity.
  • BabySmash seemed to cope very well with the speed of new notes coming at it. It probably would feel a bit more responsive if the shapes appeared instantaneously rather than animating in.
  • Apologies for the very poor sound quality on the video. I sound like I am speaking with a lisp, and the piano sounds dreadful. As for the drums, well the less said the better. I am not a drummer and there is something badly wrong when you can hear the sticks louder than the drum sounds themselves.
  • Earlier this week two keys on my piano actually got "smashed" for real by one of my children, which means I have an expensive repair bill coming my way. There's no way I'm going to let any of them have a go at smashing on my laptop!
  • Also apologies for the difficulty of seeing what's on the screen in the video. My laptop screen viewing angles aren't that great (and my cheap camera isn't exactly the greatest either, having suffered yet another "baby smash" incident of its own).

7 comments:

Anonymous said...

Fantastic! I'll get the CodePlex code up this week. I love what you've done with it.

Sergey said...

Can you post full source in archive. I'm beginner... trying to connect naudio and XNA.

Unknown said...

Hi Sergy, I'm afraid there is no more code than that (I no longer have the source available). Get the latest BabySmash code from CodePlex, and you should be able to add my code quite easily.

Sergey said...

Thx, buh I have a questions not in BabySmash.
In XNA no GUIForms. I wanna connect dll only to catch MidiInput. Can't understand Dispatcher, and how to connect Thread.
If you have a time, make an example naudio + XNA (just like this - Letters on display).
Biggy biggy thx.

Unknown said...

Hi Sergey,
I haven't used NAudio with XNA before. You don't need to use a Dispatcher though - that is just for WPF. You will need a window handle to process the message queue. Don't know if that is possible with XNA?

Mark

Anonymous said...

Hi Mark,

is it possible to use the midi interface to map certain buttons in your app to the incoming midi?

So let,s say I hit a button on my interface and then it runs the code of a button in my app.

Unknown said...

wouter - I don't see why not although you would need to write the infrastructure code yourself