Having worked with database applications for many years, I have come across a number of different object-relational mapping strategies and tools. However, most of them has had built-in design issues that caused me to dislike them after a short while. One of the latest was Castle ActiveRecord.

Fortunately - in the software industry - if you dislike something, you can code your own solution. After a talk with my colleague Lars Thorup and some inspiration by Christian Liensberger I decided it was time to create something that was simpler, more flexible, more testable and promoting better design. Frog.NET was born, and is now an open source project hosted on Google Code.

We have been using Frog.NET on an internal project for a couple of months now, and it has been great to get rid of some of the problems we experienced with Castle ActiveRecord.

Check it out here: http://frogdotnet.googlecode.com/

By Sune Gynthersen

Some interesting new tools in the field of continuous integration and code analytics are getting to speed and are exceeding past the old first-movers.
Two such tools are the build server Hudson and the “Code quality management platform” of Sonar. The first gives you a very quick to install and get running build server, with a very inviting and intuitive user interface and an open platform for third party plugins. Forget the dinosaurish CruiseControl (for Java at least) as the open source build server, Hudson serves you way better.
Your development team will love the excellent dashboard with sunshine and clouds showing how your projects are doing, and the metrics graphs with history of build and test results, build times, tasks etc. In terms of “smartness” it is said to read your Maven pom.xml and ensure that dependent builds are ran in the correct order (still need to try that).



Sonar joins a number of open source code analytics tools and puts it into a nice but at bit over cluttered dashboard. Look here to see a Sonar site of the Apache frameworks. I just want to show one of the features which I find most useful, the coverage cloud:



It identifies the classes probably needing an extra look, because complexity is running high and code coverage are running low. Brilliant way of visualizing code quality.

But remember: one thing is to know the state of your code –  another is to act wisely on it. Code coverage derived from automated test suites does not tell you, if you have assertions in your unit test. Without assertions you can add and delete code without test failing.
The violations of code quality rules and the complexity measures of you components, does not tell you if your design is thought out well, or express the domain your trying to support, only a combined effort of analysis with your customer and experiments can reveal this for you. But on the of path of getting less variation in code and fewer defects – that this getting closer to software nirvana – I will not be without Hudson and Sonar giving me daily updates on how we are doing.
By Jesper Thaning

The name of the tool is Remember the Milktm . It is a free, webbased tool. What I really like about it is that it  is simple, easy to use, and at the same time extraordinary flexible in terms of input and output.

The core of the product is a set of tasklists, each containing tasks, that can be prioritized, have a due dates, estimates and a few other things. It is easy to create and remove lists. 

But what it really impressive is the numerous ways to enter new tasks and the equally impressive amount of ways to manage and be reminded of tasks:    

First of all there is a nice Ajax based Web-UI. From that I can enter  and maintain all my tasks as well as my lists. That probably the least important part of RTM. 

I mostly interface with RTM through my iGoogle Dashboard in Firebox.  I also have access to my tasks via Gmail, Google Calendar, Skype and on my mobile phone.   Every morning I get a friendly email, telling me which tasks are due today, and if they are due at a certain hour, I get a reminder at that time too. Besides iGoogle, Google Calendar and Gmail,  I can enter tasks on my mobile phone or through email.

Try it out! - maybe you too will never need another tool to manage your tasks

By Bent Jensen

Some thoughts that popped up on my first day of attending our Poppendieck course on 23th of June.

Knowledge inventory

The concept of “Zero Inventory” is not desirable and in fact does not hold in Toyota (go to the production plant and notice the trucks on the park yard outside the factory with inventory waiting to load it into the factory – the drivers are sleeping some times), its all about having the correct amount of inventory.
So! Mary keeps a log book containing what programs she installed on her laptop and what problems she experienced (hope she doesn't mind me telling this in public). When there is a problem on the laptop, she goes to look into the log book instead of googling it again spending time on finding the same site as 6 months ago, wasting time on finding already acquired knowledge – relearning, which is waste.  
Well. This log book is knowledge inventory. Knowledge that we keep close to us, ready to use, but it takes up space and the more we have, the more difficult it is to find the knowledge we need. In software we do not have physical inventory, like wheels ready put on a car in a car factory plant, but we have knowledge inventory. So how do we produce the correct amount of knowledge inventory?

Knowledge briefs

Lets go to the source. When we make knowledge and put it into our repository (perhaps a wiki), we should make the knowledge condense, then it is easier to find - and use. But it is difficult to make knowledge brief! If I make a four page documentation of a feature, and come back six months later, I will spend time  

Because knowledge in wikis are knowledge inventory, and tend to grow and grow. Too much get a burden, you cannot find things and when you find a document, its four pages long and the valuable knowledge is hidden in a lot of irrelevant noise. (Can search engines solve the problem of finding useful knowledge in a split second on the internet? I don't think so.)

So what about a wiki that have limited space? You can only put 2400 characters on a page, because if you put on more, you build up too much knowledge inventory. “If you cannot fit your knowledge into an A3, use an A4”. Constrain yourself to condense it even more – extract the important parts e.g. the exceptions in the business case that you only found out about later, and spent time on trying to grasp.  

I want to make an exercise on our next workshop to practice the art of making knowledge brief, to avoid building up knowledge inventory.
And last thoughts: Are knowledge always written? Can you build condensed knowledge in pictures and graphs?

By Jesper Thaning

During a presentation of the scrum principles this afternoon, participants were eager to discuss about what to put around a scrum process, feeling that something is missing in terms of well-designed architecture, thorough analysis of the domain and better knowledge about requirements.

Scrum is flexible and can fit into many different environments and uses, raging from product development, deliveries of customized software and maintenance inside an IT-department. It is also beginning to be spread into other areas than software industry!

So don't look for the silver bullet extension to scrum, but try to embrace the agile and lead principles when finding the right balance of planning your project.

One important lean rule:
If you need to decide on an important part of the system (e.g. your platform, database or ): Find out what is the latest responsible moment when the decision needs to be made. It is not easy, you will probably feel , but the longer you wait the more knowledge you will have the oppurtunity gain to support your decision, and you have the possibility to set start some experiments to search for alternatives. You might change your mind later regarding the latest moment, but that is the core part of agile practices: always be ready adapt to reality.

Don't be surprised if we write more on the lean and agile way of planning on this blog...

By Jesper Thaning

Working with an app that requires live fast response times we experienced some random pauses in the execution. Normally a server request would take about 100 milliseconds but every now and then a request took considerably longer and logging showed that it was not always the same method calls that took up to 2 seconds and sometimes longer.

Suspicion began to fall on the jvm performance (Suns Jvm for Java 1.4.2) and some logging parameters was added to one of the servers in a production cluster.

java -Xloggc:gc.log -XX:+PrintGCDetails

Looking at the gc.log in a graphical tool showed this picture:

Notice that the jvm has over 700MB memory but only uses around 200MB because a full garbage collection is run every minute, and that most pauses is between 0.6 and 0.9 seconds. This interrupts the execution on the server creating the unwanted pauses.

Investigating the subtle workings of garbage collection revealed that remote method invocation (RMI – when a remote client calls an object on the server) has a role to play in how often the server-side jvm performs a full garbage collection. By default the RMI calls System.garbageCollect() every minute, which can give the long pauses.

Increasing the interval between full garbage collections to 30 minutes (here both for client and server) helped to reduce pause considerably.

java ... -Dsun.rmi.dgc.client.gcInterval=1800000 
-Dsun.rmi.dgc.server.gcInterval=1800000

One thing to note is that increasing the memory will only make the pauses longer because more garbage would be able to fill up until a full garbage collection needs to be performed. So putting lots of memory blocks in your box and boosting the jvm with more memory does not necessary help the user experience. Instead you have to suit to the actual needs of your application, for example based on how many objects are cached.

Further more on a multiprocessor server, you can benefit from enabling a concurrent garbage collector that runs in a separate thread, so the main threads are not interrupted.

java ... -XX:+UseConcMarkSweepGC

With this collector you get lots of small garbage collections in the background and practically no long pauses in the application. Notice that here we also set the initial size of the jvm to 256MB letting the jvm itself determine how much memory is needed.

So remember to dive into your jvm when performance issues arises.

Want to read more:

http://www.tagtraum.com/gcviewer.html

http://developers.sun.com/mobility/midp/articles/garbagecollection2/index.html

More general on jvm options

http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp

 

By Jesper Thaning

At my current project I was hired to make some Swing GUI talking to an EJB server. I was new to Swing GUI but I quickly began searching for a test framework fitted for testing them. I came across Fest (Fixtures for Easy Software Testing). Fest is a quite new project, so the API is still subject to some changes (its working towards a “fluent” style), but you can definitely get something out of the current version.

I came from a project where I used Selenium to make automated functional test of web sites, and in fact Fest and Selenium are quite alike in the way test are defined. Its mostly about finding controls like fields and buttons (or links) and then enter text into them or click on them, and off course assert on some attribute of the UI.

When using Selenium you often find yourself setting id attributes on e.g. the links to be able to find them in your test; in Fest you get the same testability of the GUI by setting names on the Swing components. That way you can find them in your test, and avoid basing your test on a specific position in a component hierarchy that are likely to change.

A test in Fest could look like this:

package fest;

import javax.swing.JDialog;
import javax.swing.JTable;

import junit.framework.Assert;
import junit.framework.TestCase;

import org.fest.swing.RobotFixture;
import org.fest.swing.fixture.FrameFixture;

public class FestTest extends TestCase
{
  private RobotFixture _robotFixture;
  private RecieverFrame _recieverFrame;

  private FrameFixture _recieverFrameFixture;
  private FrameFixture _senderFrameFixture;

  public void setUp()
      throws Exception
  {
    _robotFixture = RobotFixture.robotWithNewAwtHierarchy();
    _recieverFrame = new RecieverFrame();
    _recieverFrameFixture = new FrameFixture(_robotFixture,
        _recieverFrame);
    _recieverFrame.pack();

    /*
     * Make sender frame last, to be able to click on
     * buttons as it requires the frame to be in focus
     */
    SenderFrame senderFrame = new SenderFrame();
    _senderFrameFixture = new FrameFixture(_robotFixture,
        senderFrame);
    senderFrame.pack();
    _robotFixture.showWindow(senderFrame);
  }

  public void tearDown()
  {
    _robotFixture.cleanUp();
  }

  public void testSendMessageToReciever()
      throws Exception
  {
    /*
     * Verify that reciever is in a correct initial state
     */
    assertEquals("No message", _recieverFrameFixture
        .textBox("messageField").text());

    _senderFrameFixture.comboBox("forumSelector")
        .selectItemWithText("FestForum");
    _senderFrameFixture.button("foo").click();

    Thread.sleep(500);

    _recieverFrameFixture.textBox("messageField")
        .requireText("foo");

    /*
     * Example of finding a Swing component and asserting
     * directly on it
     */
    JTable messageTable = _robotFixture.finder()
        .findByType(JTable.class);
    assertEquals("Wrong number of messages", 1,
        messageTable.getRowCount());

    _senderFrameFixture.button("deleteAllMessages").click();

    Thread.sleep(500);

    assertEquals("No message", _recieverFrameFixture
        .textBox("messageField").text());
    assertEquals("Wrong number of messages", 0,
        messageTable.getRowCount());

    /*
     * This is a bit clumsy way to verify that no error
     * dialog is present, improvements are possible.
     */
    try
    {
      JDialog errorDialog = _robotFixture.finder()
          .findByType(JDialog.class);
      Assert
          .fail("Expected no dialog present found one '"
              + errorDialog.getTitle() + "'");
    }
    catch (Exception e)
    {
    }
  }
}

Notice the use of finders to get the components in the Swing GUI, you can choose to find a component by name or class, where the latter makes your test more fragile, I prefer the find-by-name where possible. And it is easy to make your own Matchers to suit your purpose.

A nice feature is that Fest fails if you try to click on a button that is not visible in the frame, verifying that your frame is big enough to contain all your buttons. To improve Fest one could make the assertion failures more informative, they tend to be “Expected false but was true” with little context from the test.

I am using Fest to test multiple Swing frames at a time that communicate together via JMS (Java Messaging) and this can easily be done with Fest. One of the challenges when testing such asynchronous behavior is that your test need to be sleep between invocation and assertion. This is by the way the same challenge you face when testing ajax'ified websites, because your test need to wait until the DOM has been updated by the ajax call.

Fest requires Java5 and can be used in JUnit3 or whatever newer test framework you are using.

Thanks to Alex Ruiz  for putting together Fest.

By Jesper Thaning

When writing code I often spot some optimization opportunities, which I typically leave for later, just writing a small comment:
    // ToDo: Optimize this
I believe it is important to first get the behaviour right, and only consider detailed optmizations later.

Then one day the "optimization"-task becomes the top priority on my list, and then I am tempted to start by looking through all my recorded optimization opportunities to try and figure out which one will give me most performance for the least amount work. But then I think twice and instead look around for my favorite profiling tool. Because all the many times I have done this optimization exercise in the past the profiler have always found that the best optimization opportunity was one that I never even considered. So typically I end up never implementing any of the small optimization opportunities I have marked for myself. Maybe I should stop comforting my self by writing them, and just skip them...

I have used several commercial profiling tools in the past, and generally enjoy their ease of use: they can typically profile many different types of applications (web, libraries, command line) and they also typically have a "hot spot"-feature that immediately show you the way to interesting optimization opportunities. On Java I have been using OptimizeIt from Borland, and on the .NET platform I have used dotTrace from JetBrains.

However I recently discovered the open source profiler NProf. And while this tool does not have all the usability features of the commercial alternatives, the core profiling functionality is good enough for its inclusion in an open source tool chain.

NProf can only profile an executable, and I was using it for a web project. But fortunately I have a lot of tests, both unit tests and integration tests. And it didn't take long to write a small executable wrapper around one of my thorough integration tests and then firing up NProf on the resulting executable. And what did I see: that the most important optimization opportunity would be caching of configuration settings, cutting around 33% off the main call sequence of my application, something that had never occured to me...

To site Lord Kelvin: If you can't measure it, you can't improve it.

By Lars Thorup

On my current project (one developer, total of six months, one month left) I have from the very beginning been using the static analysis tool "FxCop". The tool is not Open Source, but freeware from Microsoft, so it is somewhat in between being commercial and non-commercial.

Until now I have experienced both benefits and drawbacks. FxCop actually catches real bugs, albeit not any that I commit very often. Until now it has raised a warning 3 times for one specific kind of error in my code: that I forgot to call Dispose() on a member variable implementing IDisposable. Giving me this warning have probably saved me about 3 hours of debugging.

One major drawback is that it takes quite some time to run FxCop (almost as much time as a recompile), so I will not run it on every compile, meaning that I sometimes forget to run it - that's why I know how long time it takes to debug a case of a forgotten call to Dispose()

The other major drawback is that FxCop will give me an awful lot of warnings about code that it finds suspicious; warnings that I consider to be just silly. In this project I have used some time to rewrite the code to prevent FxCop from giving warnings. Otherwise I could have annotated the code to make the tool suppress just these warnings for just these places in the code. I guess that I have used around 6 hours on the rewriting. And I still have around 10 warnings that I haven't yet taken time to eliminate.

In conclusion I must at this point say that FxCop does not seem to be of real benefit for me.
By Lars Thorup