Thursday 29 November 2007

Productivity and Enthusiasm

During my holidays in my first year at university I did some warehouse work to earn some extra cash. My job involved offloading pallets of food from lorries using hydraulic pallet jacks and depositing them down a ramp from which the fork-lift truck drivers would collect them to stack in the warehouse.

Now when a lorry came in to be offloaded, there was nowhere to hide. Whether I was feeling full of enthusiasm for doing my job or bored witless, I still had to get to work. The net result was that whether my "enthusiasm" (E on the graph below) was low or very high, my "productivity" (P on the graph below) didn't vary that much.

Productivity versus Enthusiasm for manual labour

What's this got to do with software? Well, the following years I started doing programming as a holiday job, and went on to become a full-time computer programmer. But I soon noticed that in the world of software development, my Productivity versus Enthusiasm curve was radically different.

Suppose I come into work feeling tired and the task I have been given to do is not an enjoyable one (e.g. fixing elusive bugs on a legacy product, or adding a pointless feature to please marketing, writing documentation). In that case I find that a day can easily go past where productivity is close to zero.

On the other hand there are days when I am feeling very enthusiastic about my work. Perhaps it is because I am learning and using some new and interesting technology, or perhaps I am refactoring some badly designed class into a really elegant and simple structure. In this case I find that my productivity increases exponentially. As long as I am not interrupted by distractions, in this state I can become super-productive, fixing bugs and churning out new features at an incredible rate.

Productivity versus enthusiasm for software development

Why the difference between the two graphs? Why could I get work done at the warehouse on low enthusiasm days, but not in the world of software development? The answer is simply that to write software requires concentration and concentration requires interest. At the warehouse I could offload lorries while thinking about all kinds of unrelated subjects. But to write code, my mental focus must be entirely on the task at hand.

If this is true, then the way to maximise productivity is to minimise boredom. If we recognise the things that capture our interest and enthusiasm, we can design development strategies that promote productivity.

This is I think part of the genius of the "agile" approach to software development. Running tests manually is boring, but writing automated tests is much more intellectually stimulating. Bug-fixing is boring, but refactoring code to improve its design while you fix a bug gives a real sense of achievement for a developer who takes pride in their work.

I don't expect boredom can ever be completely eliminated, and so there will always be the need to grit your teeth and show some professionalism. But the route to increased productivity in software development is more closely tied to motivation and enthusiasm than many of our managers seem to appreciate.

Wednesday 28 November 2007


Somehow I missed this one when it was first featured on .NET Kicks, but for those of you who haven't come across it yet, this little utility is highly recommended. LINQPad is a simple LINQ / SQL editing and executing environment and is great for learning LINQ. It also doubles up as a lite version of SQL management studio, allowing you to easily run your LINQ or SQL statements.

What's really great about it is that it comes with a load of example scripts. For those who learn by trying things themselves, this is an invaluable tool. It does for LINQ what XAMLPad does for WPF and SilverlightPad does for Silverlight.


Highly recommended.

Tuesday 27 November 2007

Time Zones and .NET 3.5

Finally with the new .NET 3.5 framework release, Microsoft has given us the ability to convert to arbitrary time zones, and take into account historic changes to daylight savings rules. Before, this had to be done with interop, and even then you ran into all sorts of problems when trying to convert using time zones whose daylight savings information specified exact dates rather than nth Sunday of the month (e.g. the Israel / Jerusalem Standard time does this).

I did some testing today with the new TimeZoneInfo class, and all seems to be good. It works on both XP and Vista, and seems to be making use of the Dynamic DST registry keys under HKLM\Software\Microsoft\Windows NT\CurrentVersion\Time Zones (based on how many years back in time I can go before it starts getting it wrong).

One quirk compared to the old way I was converting local times back into UTC using the TzSpecificLocalTimeToSystemTime Windows API was that it now seems to guess the other way to before when given an ambiguous local time. A bit odd, but not a big deal.

Another thing I noticed is that the result of TimeZoneInfo.ConvertTimeFromUtc comes back with its DateTimeKind as Unspecified rather than as Local. I suppose this is because although it is a local time, it may not be the local time in the current locale. ConvertTimeToUtc does set the DateTimeKind to Utc as expected though.

On top of the TimeZoneInfo class, Microsoft have also given us a new DateTime structure - DateTimeOffset, which effectively allows you to store a local time with an explicit offset from UTC, giving you the best of both worlds. Switching to this from DateTime actually removes the need for TimeZoneInfo in most cases as you have the ability to get both the original LocalTime and the UTC time out of the same variable.

Monday 26 November 2007

Unit Testing is Not About Testing

When I started drafting this post, I was under the illusion that I had just stumbled across an amazing discovery, never before articulated by any programmers before me. But over the past few weeks, I have found out that what I am about to say isn't even slightly original. Everyone is saying it. But it took me quite some time to realize, so here it is anyway, for the two people out there who are as slow as me...

The concept of automated unit testing is appealing to most developers, even if the idea of writing all tests before writing code seems a bit over the top. It is certainly attractive to think that I might never have to perform another boring manual test of my software, because I can check it is working correctly simply by launching NUnit (or similar) and running through a suite of tests.

Unautomatable Tests

The trouble is, automated testing only works for a certain subset of classes - those whose outputs are clearly defined and easily measurable. And of course, those are exactly the type of classes I keep finding myself working on.

For example, I spent many months writing a library of custom Windows Forms controls. They had to be tested with keyboard and mouse entry, and at a variety of screen resolutions and font sizes. Unit testing was sadly out of the question...

Assert.LooksReallyCool(button); // not possible

Then there was the audio related code I worked on. Passing audio to the Windows MME APIs and checking that the sound played back correctly without stuttering. Or passing audio through a DSP algorithm to speed it up to double speed and automatically increase the volume in quiet bits.

Assert.DoesntStutter(audio); // not possible    
Assert.SoundsGood(audio); // not possible

I could go on. How does an automated test check that a webpage renders correctly on IE and FireFox (or harder still, on Mac and Linux)?

And here lies the trouble with the documentation for Unit Testing frameworks. It conveniently ignores these awkward cases and gives us an example of a BankAccount class that has a Withdraw method. Classes that add numbers or sort lists are ideal candidates for automated testing.

Why Bother?

The observation that a large proportion of your classes cannot be meaningfully tested with an automated test therefore puts off many developers from writing unit tests at all.

But what if the main benefit of unit testing was not actually testing? Here's my grand (but unoriginal) idea: the main benefit of unit testing is not testing, but design principles.

Design Benefits

Having a unit test for each and every class in your project means that each and every class is used at least more than once. It forces you to design classes that are loosely coupled. It will highlight hidden static dependencies. It will also enforce separation of GUI and business logic.

The trouble with thinking that Unit Testing is about testing is that "untestable" classes won't have unit tests written for them, which in turn means more badly designed classes. In short, if all your unit tests ever do is create instances of classes and call their methods without doing any checking of what they return at all, you will still benefit from greatly improved design.

Friday 23 November 2007

Pair Programming for Loners

I first heard about pair programming years ago while reading some articles on Extreme Programming. My initial reaction was "That sounds like an interesting idea. It might just work" followed very quickly by "I can think of a few people I would not like to work in a pair with".

The most obvious benefits are:

  • Having someone to bounce ideas off
  • Having someone watch you while you code and make constructive suggestions
  • Watching someone else code and learning from their approach

Obviously if you develop alone, the first two benefits are largely unavailable. The best you can hope for is to get some feedback from people on user forums.

But the third option is perhaps not as out of reach for the lone developer as it may seem. Where can you go to watch other developers at work on real projects and learn from them?

The answer is open source websites like CodePlex or SourceForge. The value of looking at real code (as opposed to sample applications) should not be underestimated.

As I have been slowly learning ASP.NET over the last year I have been frustrated with the numerous options for doing things - different project types, different data access approaches, different theming mechanisms etc etc. Before I could even start a project I was left with a thousand questions about whether I should use Web Site or Web Application, or whether to use DataGrid or GridView, or whether to use custom business objects or SqlDataSource.

So the great thing about downloading some open source projects is that it gives a great chance to see how other people have chosen to do things. The main three I have been looking at to help me with ASP.NET are BlogEngine.NET, SubText and DotNetKicks. Just spending a few hours browsing the code can teach you a lot about how to structure a project, as well as demonstrating various real-world examples of usage of the different approaches to data access.

The developers of each of these projects are all people who are not only experienced ASP.NET projects but clearly have a passion for best-practices in the way they design and structure their code. These are people who I would want to be pair programming with if I could, asking them questions about why they did things certain ways. But for now, I'll have to content myself with silently looking over their shoulders and working out what's going on for myself.

Tuesday 20 November 2007

Setting up the CodePlex Client

Microsoft's open source project hosting site CodePlex seems to be gaining a lot of momentum. I now have six projects hosted there (NAudio, MIDI File Mapper, MIDI File Splitter, SilverNibbles, Asterisk.NET, Silverlight Audio Player).

For source control, CodePlex offers Visual Studio Team System. The trouble with this is for those who work with the Visual Studio Express SKUs, the source control plugin is not available. The CodePlex development team have promised a server side bridge allowing Subversion clients such as TortoiseSVN to be used, but for the time being, there is a CodePlex source control client that can be used. Here is a brief guide to getting it up and running.

First, you need to download the CodePlex Source Control Client and extract the binaries (cpc.exe and tfs.exe). I saved mine to C:\Program Files\CodePlex Client. You can also take this opportunity to customise the cpc.exe.config file. I like to turn on the gui for status and commit dialogs:

<add key='DefaultToGuiForCommit' value='true'/> 
<add key='DefaultToGuiForStatus' value='true'/>

And I make use of the differencing tool that comes with TortoiseSVN:

<add key='DiffTool' value='C:\Program Files\TortoiseSVN\bin\TortoiseMerge.exe'/> 
<add key='DiffArgs' value='/base:{basepath} /mine:{mypath} /basename:{basename} /minename:{myname}'/> 
<add key='MergeTool' value='C:\Program Files\TortoiseSVN\bin\TortoiseMerge.exe'/>
<add key='MergeArgs' value='/base:{basepath} /mine:{mypath} /theirs:{theirpath} /basename:{basename} /minename:{myname} /theirsname:{theirname} /merged:{mergepath} /mergedname:{mergename}'/>

I then create a folder to keep my CodePlex projects in (e.g. C:\Code\CodePlex). In that folder I put a batch file that sets up my environment:

@set PATH=%PATH%;C:\Windows\Microsoft.NET\Framework\v2.0.50727;C:\Program Files\CodePlex Client

Now I can create a desktop shortcut that opens a command window pointing to that batch file. The target looks like this:

%windir%\system32\cmd.exe /k ""C:\Code\CodePlex\CodePlexDevEnv.bat""

And it should be set to start in C:\Code\CodePlex (or wherever your codeplex projects live).

Once you have done that, working with your codeplex projects is very simple. Create a subfolder, navigate into that subfolder, and perform a checkout on any project. It doesn't even have to be a project you are a member of - you can anonymously get. For example, here's how to get the code for the excellent BlogEngine.NET project.

md blogengine
cd blogengine
cpc checkout blogengine

To update to the latest files, simply navigate to the project folder and type

cpc update

To see whether you have anything that needs checking in, type

cpc status

This brings up a dialog, from which you can launch your differencing tool on files that have changed, and add unversioned files or folders to source control. When you are ready to commit your changes, then the commit command allows you to upload them with a comment.

cpc commit

Anyway, the command line client is definitely worth checking out if you use Codeplex, or even if you are simply interested in taking a look at the source for some projects hosted there.

Monday 19 November 2007

Visual Studio Beta 2

I wrote a post a while ago criticising Orcas Beta 1 so I suppose its only fair that I write about Beta 2. I have been using it for a few months now, and have found it to be excellent. I have only discovered one crash so far (to do with intellisense, and apparently fixed already for RTM).

The multi-targetting feature is superb, and the WPF designer has improved a lot (you can create a click handler for a button by double-clicking), although I can't seem to find any way of browsing other available events and adding handlers, short of using the intellisense in the XAML file. Some of my other criticisms still hold true - there is no help in setting FontFamily and Ctrl-Click copying of controls doesn't work.

I was able to attend a presentation by Daniel Moth last Thursday, who demoed some of the new features for web development. I am not an expert in ASP.NET, but have been doing a bit of learning in the last year, and the new features look excellent, particularly getting better intellisense (even in Javascript) and help with CSS. These features should be very helpful for me getting up to speed.

Silverlight support is sadly not going to be built-in, although to be fair, Silverlight 1.1 has still not been officially released. Having said that, the Silverlight development story is very good in VS2008 compared to the pain of working on it in VS2005.

Overall though, I'm a lot more enthusiastic about VS2008 having used the Beta 2. What is even more exciting is the possibility of using the new C# 3 language features against .NET 2.0 projects. This is a really compelling reason to upgrade even if you have no intention of using the .NET 3.0 or 3.5 libraries.

Friday 16 November 2007

InvokeRequired for WPF

Those of you who develop multi-threaded Windows Forms client applications will be well aware of the Control.InvokeRequired  method needed every time you access a control from a thread other than the one on which it was created. Code such as the lines seen below, which seemed so arcane when you first encountered them, have now been committed to memory:

void finder_FoundFile(object sender, FoundFileEventArgs e)
   if (this.InvokeRequired)
      this.BeginInvoke(new EventHandler<FoundFileEventArgs>(finder_FoundFile), new object[] { sender, e });

But what about the brave new world of WPF? Will we have to write the same code, or will WPF controls let us set their properties from any thread? Sadly not. We have to do a similar trick, this time using the Dispatcher object to invoke the method call on the correct thread:

void finder_FileFound(object sender, FileFoundEventArgs e)
    if (listBoxFiles.Dispatcher.Thread == Thread.CurrentThread)
            new EventHandler<FileFoundEventArgs>(finder_FileFound), sender, new object[] { e } );

In fact, its even worse than for Windows Forms, as the thread signature for Dispatcher.BeginInvoke is even more perplexing than the one for Control.BeginInvoke. You not only have to specify a priority, but also you have to split your parameters up into "first parameter" and "array of all other parameters". Very odd.

But what about data binding in WPF? Surely that would be the easy way round this mess. Bind the list box to an observable collection, and let your background thread change the collection. Unfortunately, the same problem awaits you. As soon as you add something to the observable collection an InvalidOperationException fires warning you that you are not on the correct Dispatcher thread.

Does that rule out all hope of databinding in a multi-threaded context? Thankfully not, as I discovered a helpful article from Beatriz Costa explaining a way to make an observable collection that performs the context switch for you. Its not exactly ideal, but at least it will get you up and running. Apparently MS are considering improving the situation for the next drop of WPF.