Test Driven Thread Safety

Friday, September 14, 2007 6:22:46 PM (Romance Standard Time, UTC+01:00)
The other day I needed a thread safe collection class. Since .NET 2.0 lack thread safe generic collection classes and because I wanted a ReaderWriterLock for best throughput I decided to write it myself.
Being a TDD-kind of guy, I wanted to start with some failing tests, and focusing on the thread safety of the class I wanted the tests to fail because of lacking thread safety. In this way I would know for sure when the added thread safety mechanisms was actually working.
But how do you write a test that fails because of lacking thread safety? Well, in this case I could start up some threads, make sure that they used the collection methods simultaneously and then assert that only one thread was allowed to do so at a time.
That turned out to be more easy to think than to code. Lets look at the two problems and their solution:
"Making sure that the threads used the collection methods simultaneously". The collection methods are typically so fast that it is practically impossible to arrange for two threads to try to enter a method simultaneously. However, I decided that for testing purposes, I could slow down the methods and thereby ensure that the first thread was still inside the method when the next thread came around. So inside the Collection.Add()-method I added:
   if (assertEnabled)
{
Thread.Sleep(100);
}

"Asserting that only one thread at a time is allowed to execute a method". A thread cannot know whether other threads are running the same method body. However, inside the method body, we can count the number of threads entering and exiting. And then we can add Assert-statements stating the required restriction on the number of concurrent threads in the method. So again, inside the Collection.Add()-method I added:
   if (assertEnabled)
{
Interlocked.Increment(ref writerCount);
Thread.Sleep(100);
Assert.AreEqual(1, writerCount);
Assert.AreEqual(0, readerCount, "trying to write while reading");
}
// the implementation of the method go here
if (assertEnabled)
{
Interlocked.Decrement(ref writerCount);
}

Having these mechanisms in place I could write a test that failed because of the "trying to write while reading"-exception being thrown.
   [Test]
public void WriterWaitForReader()
{
Thread r1 = CreateReader();
Thread w1 = CreateWriter();
r1.Join();
w1.Join();
Assert.That(exception, Is.Null);
}

Now having a failing test, allowed me to add the actual thread safety mechanism using the ReaderWriterLock in the Collection.Add()-method:
   rwLock.AcquireWriterLock(InfiniteTimeOut);
if (assertEnabled)
{
Interlocked.Increment(ref writerCount);
Thread.Sleep(100);
Assert.AreEqual(1, writerCount);
Assert.AreEqual(0, readerCount, "trying to write while reading");
}
// the implementation of the method go here
if (assertEnabled)
{
Interlocked.Decrement(ref writerCount);
}
rwLock.ReleaseLock();

Bringing me a succeeding test. Then I could add tests for the actual functionality of my collection class and the corresponding implementation.
Depending on the project in which this code is being used I could keep asserEnabled==true in production as well as when running the tests (this requires another condition to avoid enabling the Sleep()). Another choice could be to entirely avoid the overhead of the assertEnabled condition by surrounding it with #if DEBUG ... #endif, so the tests only run in debug mode.
Take a look at the example source code (for Visual Studio 2008). By Lars Thorup