Audio WaveForm Drawing Using WPF

A while ago I blogged about how to display audio waveforms in WinForms. Porting this code to WPF is not as simple as it may first appear. The rendering models are quite different. In Windows Forms, you can draw points or lines using GDI functions. You are responsible for drawing them all every time the window is invalidated and its Paint method is called. However, in WPF, you create objects to put on a Canvas, and WPF manages the rendering and invalidation.

My first attempt was to stay as close to the WinForms model as I could. I have a sample aggregator that looks for the highest sample value over a short period of time. It then uses that to calculate the height of the line it should draw. Every time we calculate a new one, we add a line to our Canvas at the next X position, wrapping round if necessary, and deleting any old lines.

As can be seen, this gives a reasonable waveform display. I made use of a LinearGradientBrush to try to improve the visual effect a little (although this requires we cheat and keep the waveform symmetrical). There is a big performance problem however - it is very inefficient to keep throwing new lines onto a Canvas and removing old ones. The solution was to start re-using lines once we had wrapped past the right-hand edge of the display.

private Line CreateLine(float value)
{
    Line line;
    if (renderPosition >= lines.Count)
    {
        line = new Line();
        lines.Add(line);
        mainCanvas.Children.Add(line);
    }
    else
    {
        line = lines[renderPosition];
    }
    line.Stroke = this.Foreground;
    line.X1 = renderPosition;
    line.X2 = renderPosition;
    line.Y1 = yTranslate + -value * yScale;
    line.Y2 = yTranslate + value * yScale;
    renderPosition++;
    line.Visibility = Visibility.Visible;
    return line;
}

This solves our performance issue, but I still wasn’t too happy with the visual effect – it is too obviously composed of vertical lines. I tried a second approach. This added two instances of PolyLine to the canvas. Now, we would add a point to each line when a new maximum sample was created. Again the same trick of re-using points when we had scrolled off the right-hand edge was used for performance reasons.

As nice as this approach is, there is an obvious problem that we are not able to render the bit in between the top and bottom lines. This requires a Polygon. However, we can’t just add new points to the end of the Polygon’s Points collection. We need all of the top line points first, followed by all of the bottom line points in reverse order if we are to create a shape.

The trick is that when we get a new sample maximum and minimum in, we have to insert those values into the middle of the existing Points collection, or calculate the position in the points array. Notice that I create a new Point object every time to make sure that the Polygon is invalidated correctly.

private int Points
{
    get { return waveForm.Points.Count / 2; }
}

public void AddValue(float maxValue, float minValue)
{
    int visiblePixels = (int)(ActualWidth / xScale);
    if (visiblePixels > 0)
    {
        CreatePoint(maxValue, minValue);

        if (renderPosition > visiblePixels)
        {
            renderPosition = 0;
        }
        int erasePosition = (renderPosition + blankZone) % visiblePixels;
        if (erasePosition < Points)
        {
            double yPos = SampleToYPosition(0);
            waveForm.Points[erasePosition] = new Point(erasePosition * xScale, yPos);
            waveForm.Points[BottomPointIndex(erasePosition)] = new Point(erasePosition * xScale, yPos);
        }
    }
}

private int BottomPointIndex(int position)
{
    return waveForm.Points.Count - position - 1;
}

private double SampleToYPosition(float value)
{
    return yTranslate + value * yScale;
}

private void CreatePoint(float topValue, float bottomValue)
{
    double topYPos = SampleToYPosition(topValue);
    double bottomYPos = SampleToYPosition(bottomValue);
    double xPos = renderPosition * xScale;
    if (renderPosition >= Points)
    {
        int insertPos = Points;
        waveForm.Points.Insert(insertPos, new Point(xPos, topYPos));
        waveForm.Points.Insert(insertPos + 1, new Point(xPos, bottomYPos));
    }
    else
    {
        waveForm.Points[renderPosition] = new Point(xPos, topYPos);
        waveForm.Points[BottomPointIndex(renderPosition)] = new Point(xPos, bottomYPos);
    }
    renderPosition++;
}

This means that our minimum and maximum lines join together to create a shape, and we can fill in the middle bit.

Now we are a lot closer to the visual effect I am looking for, but it is still looking a bit spiky. To smooth the edges, I decided to only add one point every two pixels instead of every one:

This tidies up the edges considerably. You can take this a step further and have a point every third pixel, but this highlights another problem – that our polygons have sharp corners as they draw straight lines between each point. The next step would be to try out using Bezier curves, although I am not sure what the performance implications of that would be. Maybe that is a subject for a future post.

The code for these waveforms will be made available in NAudio in the near future.

Where are you going to put that code?

Often we want to modify existing software by inserting an additional step. Before we do operation X, we want to do operation Y. Consider a simple example of a LabelPrinter class, with a Print method. Suppose a new requirement has come in that before it prints a label for a customer in Sweden, it needs to do some kind of postal code transformation.

Approach 1 – Last Possible Moment

Most developers would immediately gravitate towards going to the Print method, and putting their new code in there. This seems sensible – we run the new code at the last possible moment before performing the original task.

public void Print(LabelDetails labelDetails)
{
    if (labelDetails.Country == "Sweden")
    {
        FixPostalCodeForSweden(labelDetails);
    }
    // do the actual printing....
}

Despite the fact that it works, this approach has several problems. We have broken the Single Responsibility Principle. Our LabelPrinter class now has an extra responsibility. If we allow ourselves to keep coding this way, before long, the Print method will become a magnet for special case features:

public void Print(LabelDetails labelDetails)
{
    if (labelDetails.Country == "Sweden")
    {
        FixPostalCodeForSweden(labelDetails);
    }
    if (printerType == "Serial Port Printer")
    {
        if (!CanConnectToSerialPrinter())
        {
            MessageBox.Show("Please attach the printer to COM1");
            return;
        }
    }
    // do the actual printing....
}

And before we know it, we have a class where the original functionality is swamped with miscellaneous concerns. Worse still, it tends to become untestable, as bits of GUI code or hard dependencies on the file system etc creep in.

Approach 2 – Remember to Call This First

Having seen that the LabelPrinter class was not really the best place for our new code to be added, the fallback approach is typically to put the new code in the calling class before it calls into the original method:

private void DoPrint()
{
    LabelDetails details = GetLabelDetails();
    // remember to call this first before calling Print
    DoImportantStuffBeforePrinting(details);
    // now we are ready to print
    labelPrinter.Print(details);
}

This approach keeps our LabelPrinter class free from picking up any extra responsibilities, but it comes at a cost. Now we have to remember to always call our DoImportantStuffBeforePrinting method before anyone calls the LabelPrinter.Print method. We have lost the guarantee we had with approach 1 that no one call call Print without the pre-requisite tasks getting called.

Approach 3 – Open – Closed Principle

So where should our new code go? The answer is found in what is known as the “Open Closed Principle”, which states that classes should be closed for modification, but open for extension. In other words, we want to make LabelPrinter extensible, but not for it to change every time we come up with some new task that needs to be done before printing.

There are several ways this can be done including inheritance or the use of the facade pattern. I will just describe one of the simplest – using an event. In the LabelPrinter class, we create a new BeforePrint event. This will fire as soon as the Print function is called. As part of its event arguments, it will have a CancelPrint boolean flag to allow event handlers to request that the print is cancelled:

public void Print(LabelDetails labelDetails)
{
    if (BeforePrint != null)
    {
        var args = new BeforePrintEventArgs();
        BeforePrint(this, args);
        if (args.CancelPrint)
        {
            return;
        }
    }
    // do the actual printing....
}

This approach means that our LabelPrinter class keeps to its single responsibility of printing labels (and thus remains maintainable and testable). It is now open for any future enhancements that require an action to take place before printing.

There are a couple of things to watch out for with this approach. First, you would want to make sure that whenever a LabelPrinter is created, all the appropriate events were hooked up (otherwise you run into the same problems as with approach 2). One solution would be to put a LabelPrinter into your IoC container ready set up.

Another potential problem is the ordering of the event handlers. For example, checking if you have permission to print would make sense as the first operation. The simplest approach is to add the handlers in the right order.

Conclusion

Always think about where you are putting the new code you write. Does it really belong there? Can you make a small modification to the class you want to change so that it is extensible, and then implement your new feature as an extension to that class? If you do, you will not only keep the original code maintainable, but your new code is protected from accidental breakage, as it is isolated off in a class of its own.

Creating HTML using NVelocity

I recently had need to create some HTML output from a .NET console application. Often in this scenario, I will simply crank out the HTML in code, constructing it bit by bit with a StringBuilder. However, this time round I decided to look for a more elegant solution. I wanted to create a text file with a template, and for my data to be dynamically put into the right place.

While this could be done with XLST, or even some custom string replacement code, I decided to try out a .NET templating engine. There are a number of these available, including NHaml, Brail, and Spark, but I chose to go with NVelocity, whose syntax seemed to be nice and straightforward, allowing other developers to easily see what is going on and make changes to the templates.

Getting the NVelocity DLL

This proved harder than I was expecting. The original NVelocity project has not been updated in several years, but over at the Castle Project they have taken the source and are improving it. However, I couldn’t find a Castle Project download that contained a built DLL, so I ended up having to download the entire Castle Project source using Subversion, and building it.

Creating a Template

This is the nice and easy bit. Here you can see I am printing out a HTML table of books in a collection of books. I think the NVelocity syntax is pretty self-explanatory.

<h3>Books</h3>

#foreach($book in $books)
#beforeall
<table>
  <tr>
    <th>Title</th>
    <th>Author</th>
  </tr>
#before
  <tr>
#each
    <td>$book.Title</td>
    <td>$book.Author</td>
#after
  </tr>
#afterall
</table>
#nodata
No books found.
#end

Applying the Transformation

Now we need to get hold of the template we created and load it into a stream. I embedded my template as a resource. Then we need to set up a VelocityContext, which will contain all the data needed to be injected into our HTML. Then it is a simple matter of creating the VelocityEngine and passing it the context and the template. It returns a string, which can be written to disk if required.

public static string TransformBooksToHtml(IEnumerable<Book> books, string resourceTemplateName)
{
    Stream templateStream = typeof(TemplateEngine).Assembly.GetManifestResourceStream(resourceTemplateName);
    var context = new VelocityContext();
    context.Put("books", books);
    return ApplyTemplate(templateStream, context);
}

public static string ApplyTemplate(Stream templateStream, VelocityContext context)
{
    VelocityEngine velocity = new VelocityEngine();
    ExtendedProperties props = new ExtendedProperties();
    velocity.Init(props);
    var writer = new StringWriter();
    velocity.Evaluate(context, writer, "XYZ", new StreamReader(templateStream));
    return writer.GetStringBuilder().ToString();
}

Limitations

One limitation that comes to mind is that I am not sure what would happen if the data contained characters that needed to be encoded for HTML (e.g. the less than symbol). I haven’t tested this scenario, but I am sure there is some way of working round it (especially as NVelocity is intended specifically for scenarios requiring HTML output).

Binding Combo Boxes in WPF with MVVM

I was recently creating a simple WPF application and was trying to use the MVVM pattern. In this pattern, all the controls on your form are data bound to properties on your “View Model” class. While there are lots of examples of how to do this with Text boxes, List boxes, and even master-detail views, it seems that examples for binding Combo boxes are a little thin on the ground.

What I wanted to do was bind the items in the ComboBox to a list in my ViewModel and to track the currently selected item. I found that there does not seem to be one “official” way of doing this. One approach bound both the ItemsSource and the SelectedValue properties of the Combo box to corresponding properties on ViewModel. The approach I went with uses a CollectionView which is a class included with .NET that encapsulates a list and the concept of a current item as well as supporting the INotifyPropertyChanged interface.

So here is the XAML first. I set the IsSychronizedWithCurrentItem to true to allow us to track the current item on the ItemsSource.

<ComboBox ItemsSource="{Binding Path=Queries}"                 
          IsSynchronizedWithCurrentItem="True"
          DisplayMemberPath="Name" />

The code in the ViewModel first creates the List, and then creates a CollectionView based on that list. This allows us to set the CurrentItem from the ViewModel as well as get notified whenever the CurrentItem changes.

public MainWindowViewModel()
{
    IList<Query> availableQueries = new List<Query>();
    // fill the list...

    Queries = new CollectionView(availableQueries);
    Queries.MoveCurrentTo(availableQueries[0]);
    Queries.CurrentChanged += new EventHandler(queries_CurrentChanged);
}

public CollectionView Queries { get; private set; }

void queries_CurrentChanged(object sender, EventArgs e)
{
    Query currentQuery = (Query)Queries.CurrentItem;
}

I’m not sure yet whether using CollectionView is a better approach than the alternatives I have seen which bind the SelectedValue or SelectedItem property. I would be interested to hear in the comments if you think either approach has benefits over the other. One consideration is that Silverlight doesn’t seem to support CollectionView at the moment.

An Audio Effects Framework for NAudio

This is just a quick post to point out that I have had an article published on Coding4Fun this week. It demonstrates how to make a voice changing effect for Skype using NAudio. Check out the article here: Skype Voice Changer.

I have now uploaded all the code (and release binaries) to a CodePlex project.

The other thing to say is that the audio effect framework I developed for this application will eventually find its way into NAudio. I am still deciding on quite how best to integrate it, but watch this space for further news.

Have fun talking to your friends in silly voices on Skype!

XAML SoundWave Take 2

In my previous post, I described a couple of options for creating a sound-wave icon graphic in XAML. While my final solution worked OK, I have since thought up an even simpler approach by using the XAML Path mini-language.

The reason I didn’t initially go for this option was that I didn’t want to calculate the coordinates of each point. However, we can draw the shape rotated by 45 degrees first, which allows us to put each point nicely on a grid, and then use a RotateTransform afterwards to put it back.

The XAML Path is a very useful tool to master if you prefer to work directly with XAML instead of using a design tool like Expression Blend. Here’s the first path, to create a quadrant shape:

<Path Fill="Orange" Data="M 0,0 h 1 a 1,1 90 0 1 -1,1 Z" />

The M command starts us off by moving to the specified coordinates (0,0 in this case). A lower case h means a relative horizontal line – we move one unit to the right. Now the lower case a command means we are drawing an arc using relative coordinates. The first two arguments are the radius of the arc (1,1 = circle of radius 1), then comes the angle in degrees (90), then a flag indicating if this is a large arc – in our case, no (0), and then another flag indicating if we are going clockwise (1 = yes). Finally, we put the end coordinates (-1,1) which are relative to the starting point because we used a lower case a. This path is then closed with the Z command although this is not strictly necessary for us as we are not using a Stroke brush on this Path.

Now we can draw the other three sections, each of which consists of a horizontal line followed by an arc, then a vertical line, and finally another arc with a smaller radius. I don’t bother with the Z because I have reached my starting point again with the second arc.

<Path Fill="Orange" Data="M 2,0 h 1 a 3,3 90 0 1 -3,3 v -1 a 2,2 90 0 0 2,-2" />
<Path Fill="Orange" Data="M 4,0 h 1 a 5,5 90 0 1 -5,5 v -1 a 4,4 90 0 0 4,-4" />
<Path Fill="Orange" Data="M 6,0 h 1 a 7,7 90 0 1 -7,7 v -1 a 6,6 90 0 0 6,-6" />

We end up with the right graphics but with the wrong orientation:

A RotateTransform (plus a ScaleTransform) allows us to put this exactly how we want it:

Here’s the complete XAML:

<Page
  xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
  xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml" Background="#E0E0E0">
  <Grid Width="8" Height="8">
    
    <Grid.RenderTransform>
      <TransformGroup>
      <ScaleTransform ScaleX="10" ScaleY="10" />
      <RotateTransform Angle="-45" />
      </TransformGroup>
    </Grid.RenderTransform>
    
    <Path Fill="Orange" Data="M 0,0 h 1 a 1,1 90 0 1 -1,1 Z" />
    <Path Fill="Orange" Data="M 2,0 h 1 a 3,3 90 0 1 -3,3 v -1 a 2,2 90 0 0 2,-2" />
    <Path Fill="Orange" Data="M 4,0 h 1 a 5,5 90 0 1 -5,5 v -1 a 4,4 90 0 0 4,-4" />
    <Path Fill="Orange" Data="M 6,0 h 1 a 7,7 90 0 1 -7,7 v -1 a 6,6 90 0 0 6,-6" />
  </Grid>
</Page>

Soundwave XAML

I visited the NoiseTrade website this week and they had a neat looking soundwave icon they are using throughout the site. I doubt it is original to them, but I  it works nicely and I thought I would have a go at creating it in XAML.

Its very simple to do. You create a series of concentric circles and then mask off the bit you don’t want with a Polygon.

Here’s the XAML:

<Grid Width="100" Height="100">  
    <Ellipse Width="10" Height="10" Fill="Orange" />
    <Ellipse Width="30" Height="30" Stroke="Orange" StrokeThickness="5"  />
    <Ellipse Width="50" Height="50" Stroke="Orange" StrokeThickness="5"  />
    <Ellipse Width="70" Height="70" Stroke="Orange" StrokeThickness="5"  />
    <Polygon Stretch="Fill" Fill="White" Points="0,0, 1,1 -1,1 -1,-1 1,-1" />
  </Grid>

The only really clever thing going on here is that we use the Stretch property of the Polygon to centre our cut-out shape over the centre point of the circles. The Stretch property also means we can use nominal coordinates, making it easier to get the coordinates right. Alternatively we could have used the following:

<Polygon Fill="Red" Points="50,50, 100,100 0,100 0,0 100,0" />

The outer Grid must be sized to be a square so that the angles are 45 degrees when using the Stretch=Fill method. Here’s the whole thing with the cut-out in red to make it more obvious what is going on:

I then tried using Expression Blend to subtract the Polygon from the each of the Ellipses in turn, but it struggled to do what I wanted as it closed the new arcs giving the following effect (not sure why the right-hand sides look clipped):

However, I was able to fix this by first going in and removing the final “z” from each newly created Path, which meant that the shapes were no longer closed. Then using the node editor tool, delete the extra node from each path. This gets us almost there, with the exception that Expression Blend has let the central circle drift slightly out of position:

The resulting XAML isn’t quite so clean as our initial attempt, but it has the advantage of not relying on a white shape covering things up, meaning that it can be placed on top of any background image.

<Grid Width="100" Height="100">
  <Path HorizontalAlignment="Right" Margin="0,46.464,45,46.464" Width="5" Fill="Orange" Stretch="Fill" Data="M3.535533,0 C4.4403558,0.90481973 5,2.1548209 5,3.535533 C5,4.916245 4.4403558,6.1662469 3.535533,7.0710659 L0,3.535533 z"/>
  <Path Margin="0,38.661,34.5,38.339" Stretch="Fill" Stroke="Orange" StrokeThickness="5" Data="M11.338835,2.5 C13.600883,4.7620502 15,7.8870525 15,11.338835 C15,14.790609 13.600889,17.915615 11.338835,20.17767" Width="8.808" HorizontalAlignment="Right"/>
  <Path Margin="0,31.59,24.5,31.41" Stretch="Fill" Stroke="Orange" StrokeThickness="5" Data="M18.409901,2.5 C22.481602,6.5716963 25,12.19669 25,18.409901 C25,24.623108 22.481606,30.248114 18.409901,34.319801" Width="11.737" HorizontalAlignment="Right"/>
  <Path Margin="0,24.519,14.5,24.481" Stretch="Fill" Stroke="Orange" StrokeThickness="5" Data="M25.480968,2.5 C31.362314,8.3813419 35,16.506346 35,25.48097 C35,34.455612 31.362314,42.580608 25.480968,48.461937" Width="14.665" HorizontalAlignment="Right"/>
</Grid>

Interestingly, rendering this in Kaxaml, the inner segment now appears in the right place: