Test Driven Thread Safety

Friday, September 14, 2007 6:22:46 PM (Romance Standard Time, UTC+01:00)
The other day I needed a thread safe collection class. Since .NET 2.0 lack thread safe generic collection classes and because I wanted a ReaderWriterLock for best throughput I decided to write it myself.
Being a TDD-kind of guy, I wanted to start with some failing tests, and focusing on the thread safety of the class I wanted the tests to fail because of lacking thread safety. In this way I would know for sure when the added thread safety mechanisms was actually working.
But how do you write a test that fails because of lacking thread safety? Well, in this case I could start up some threads, make sure that they used the collection methods simultaneously and then assert that only one thread was allowed to do so at a time.
That turned out to be more easy to think than to code. Lets look at the two problems and their solution:
"Making sure that the threads used the collection methods simultaneously". The collection methods are typically so fast that it is practically impossible to arrange for two threads to try to enter a method simultaneously. However, I decided that for testing purposes, I could slow down the methods and thereby ensure that the first thread was still inside the method when the next thread came around. So inside the Collection.Add()-method I added:
   if (assertEnabled)
{
Thread.Sleep(100);
}

"Asserting that only one thread at a time is allowed to execute a method". A thread cannot know whether other threads are running the same method body. However, inside the method body, we can count the number of threads entering and exiting. And then we can add Assert-statements stating the required restriction on the number of concurrent threads in the method. So again, inside the Collection.Add()-method I added:
   if (assertEnabled)
{
Interlocked.Increment(ref writerCount);
Thread.Sleep(100);
Assert.AreEqual(1, writerCount);
Assert.AreEqual(0, readerCount, "trying to write while reading");
}
// the implementation of the method go here
if (assertEnabled)
{
Interlocked.Decrement(ref writerCount);
}

Having these mechanisms in place I could write a test that failed because of the "trying to write while reading"-exception being thrown.
   [Test]
public void WriterWaitForReader()
{
Thread r1 = CreateReader();
Thread w1 = CreateWriter();
r1.Join();
w1.Join();
Assert.That(exception, Is.Null);
}

Now having a failing test, allowed me to add the actual thread safety mechanism using the ReaderWriterLock in the Collection.Add()-method:
   rwLock.AcquireWriterLock(InfiniteTimeOut);
if (assertEnabled)
{
Interlocked.Increment(ref writerCount);
Thread.Sleep(100);
Assert.AreEqual(1, writerCount);
Assert.AreEqual(0, readerCount, "trying to write while reading");
}
// the implementation of the method go here
if (assertEnabled)
{
Interlocked.Decrement(ref writerCount);
}
rwLock.ReleaseLock();

Bringing me a succeeding test. Then I could add tests for the actual functionality of my collection class and the corresponding implementation.
Depending on the project in which this code is being used I could keep asserEnabled==true in production as well as when running the tests (this requires another condition to avoid enabling the Sleep()). Another choice could be to entirely avoid the overhead of the assertEnabled condition by surrounding it with #if DEBUG ... #endif, so the tests only run in debug mode.
Take a look at the example source code (for Visual Studio 2008). By Lars Thorup

"Stop The Line" Issues

Sunday, September 02, 2007 9:18:21 PM (Romance Standard Time, UTC+01:00)

Creating quality software on a continuous basis is a complicated business.

I think it is a sign of maturity if your team can admit that there are daily problems which are ultimately obstructing flow. While not everyone will appreciate that your team does a complete halt until the obstruction has been removed I think it is a necessity for a team to stay in good shape and continue to deliver frequently.

Just the other day I came to think about what were the most common obstructions of flow we see in software development environments. So here is my preliminary list -- I bet you have seen them all.

1. Build breaks
If you don't have a build, you don't have anything. Nobody is able to do any progress progress as the state of the product is unknown. Developers cannot check-in, as they will not get any response from the continuous integration environment. Testers cannot try out the latest build. Users cannot see the product in action, and the list goes on. A build break is like a blindfold, and you should treat it as such. Don't try to go anywhere until you have your vision back.

2. Test failures
If you have a failing test, you have dysfunctional software. Imaging somebody fitting some cool new styling parts on to a car with a busted engine. Now, who would ever do that? It really is the same with software. Why do any feature work on a software product which has a failing test? I know this can be hard to respect, especially when a deadline is closing in on the team. Deadlines however, does not justify continuing without stopping the line upon test failures.

3. Bug reports
If you have bug reports, you have dysfunctional software. There is no need to do any feature work, until the product is stabilized. Bug reports are even worse than failing automated tests as those bugs can only be reproduced by a human. You should not spend work hours on anything but writing tests for these bugs, and fixing them.

4. Any other blocks keeping the team from a steady flow
If you are experiencing an obstruction which is keeping from moving forward as fast as you could, suggest to the team that you stop doing anymore feature work, until you are able to eliminate the obstruction. One obstacle which I think is quite common is builds that take way to long time. "Yeah, but how can you build 2 million lines of code in less than two hours?" -- Well, I suppose you could start by reorganizing your code and build only the module related to a particular source control check-in. Hopefully you don't have modules consisting of 2 million lines of code!

I'd like to hear about any other obstructions you may have experienced in a software development environment.

By Sune Gynthersen