In database applications it is often important to ensure that your SQL uses indexes to avoid performance problems. Consider an SQL select statement like this

SELECT text FROM response WHERE questionId = 27 AND participantId = 38

If there is no index on response(questionId, participantId), the database needs to do a full table scan, and if the response table have a lot of rows, this will degrade performance. Such a performance problem might only be observed during testing on large amounts of data, which is not typically something a developer will do before every commit.

It would be useful to be able to detect this kind of problem much earlier, preferably during unit testing. Here I will describe one way to do this using Microsoft SQL Server and .NET:

SQL Server makes a lot of information accessible through performance counters, and one of those performance counters is incremented every time SQL Server does a full table scan. Utilizing that information, we can write a unit test that verifies that indexes are properly in place by asserting that the number of full table scans performed by a piece of code is equal to zero:

          
            
              using
            
          
          
            
              (
            
          
          
            
              new
            
          
          
            
              
                
                
              
            
          
          
            
              AssertSqlFullScanCount
            
          
          
            
              (0))
              

{
          
          
            
              
                

var
            
          
          
            
              
                
                responses
= 
            
          
          
            
              Response
            
          
          
            
              .GetResponses(q,
p);
            
            

          
          
            
               Assert
            
          
          
            
              .That(responses,
...);
              

}
          
        

This code uses the following helper class:

          
            
              public
            
          
          
            
              
                
                
              
            
          
          
            
              class
            
          
          
            
              
                
                
              
            
          
          
            
              AssertSqlFullScanCount
            
          
          
            
               : 
            
          
          
            
              IDisposable
{
private
          
          
            
              
                
                
              
            
          
          
            
              int
            
          
          
            
               expected;
            
            

          
          
            
              private
            
          
          
            
              
                
                
              
            
          
          
            
              long
            
          
          
            
               actual;
            
            

          
          
            
               private
            
          
          
            
              
                
                
              
            
          
          
            
              PerformanceCounter
            
          
          
            
               fullScans;
            
            

          
          
            
              public
            
          
          
            
               AssertSqlFullScanCount(
            
          
          
            
              int
            
          
          
            
               expected)

{

            
          
          
            
              this
            
          
          
            
              .expected
= expected;
            
            

          
          
            
               this
            
          
          
            
              .fullScans
= 
            
          
          
            
              new
            
          
          
            
              
                
                
              
            
          
          
            
              PerformanceCounter
            
          
          
            
              (

            
          
          
            
              
              
            
          
          
            
              
                
                "MSSQL$SQLEXPRESS:Access
Methods"
            
          
          
            
              
              
            
          
          
            
              ,

            
          
          
            
               "Full
Scans/sec"
            
          
          
            
              );
            
            

          
          
            
              this
            
          
          
            
              .actual =
fullScans.RawValue;

}


public
            
          
          
            
              
                
                
              
            
          
          
            
              void
            
          
          
            
               Dispose()
              

{

actual = fullScans.RawValue - actual;
            

          
          
            
              Assert
            
          
          
            
              .That(actual,
            
          
          
            
              Is
            
          
          
            
              .EqualTo(expected));
              

}

}
          
        

Do you consider this kind of performance unit testing useful? How would you implement this kind of tests with Oracle

I went to a few sessions at the Agile 2009 conference related to group dynamics, hyper productive teams and corporate culture.

Help me to see... corporate culture with Tobias Mayer and Lyssa Adkins. "Using a simple yet effective collaboration game from the Improv tradition this session will challenge our assumptions and open up new neural pathways." It was a very interesting workshop with a limited number of people. Tobias showed us different techniques to describe a new reality for an agile person, an agile team and an agile organization in a collaborative and simple way without being locked in our traditional beliefs.

Help me to see... 

Scaling Up by Scaling Down: A (re)Focus on Individual Skills with Ashley Johnson and Amr Elssamadisy. The key message on this session was to focus more on developing personal agility, before trying to scaling up. The primary aspect of agile is skills and personalities (Individuals and interactions over processes and tools…).

Scaling Up by Scaling Down

Some of the quotes from sessions and discussions:

• "The pain of change has to be less than the pain of failing"
• "Trust and credibility is the glue, if you don't create it you will fail"
• "Before getting it to work in the large, we need to get it to work in the small"

I think from an agile point of view, we need to understand our craft much better and focus on business value, innovation and continuous learning.

We talk more about "How we create the solution" than "Why do we create the solution?"

We talk more about "How great we are doing" than "How we can continuously improve and be better?"

We talk more about "How can we be more productive" than "How can we create less useless features?"

I think we should focus more on

• Building a better understanding on what we are doing – create a unity of purpose
• Working with continuously improving and learning in the organization
• Teaching and using problem solving techniques
• How we structure our work with cadence, flow and pull 
• How we establishing more transparency and accountability

What do you think?

When slicing the Agile 2009 conference and looking at trends, there was a lot of talk about Kanban, Mainstream Agile and using Games for improving and learning Agile/Lean techniques. But also group dynamics, how we are working together, how we collaborate and how to create a shared vision were included in many sessions and discussions.

Games

There were many games at the conference, and it was very interesting to see how people got involved in another way when playing a game using Agile and Lean techniques, instead of trying to understand someone explaining some principles and practices. Using games in teaching will make it much more effective and fun. I will start to include more games in the future, when facilitating workshops.

Kanban

There was also a lot of "Buzz" about Kanban, and I can see a lot of interest in my Kanban talk on September 15^th. A Kanban system can be used to implement different Lean techniques in your Agile processes. It can be a more visible way to look at flow in the daily work, limit the work in progress, establishing a cadence etc.

I got a lot of interesting input to use on my Kanban talk on September 15^th and also to include a Kanban game instead of different exercises on the Kanban workshop on October 28^th.

Experience reports

Experience reports were also a main topic, and it is very important to capture and share the learning from real life problems, failures and success.

You can get the experience report I wrote for the conference from http://tinyurl.com/CmmiAgile and the slides from the Agile 2009 session on http://tinyurl.com/nk2g5p. It is about Agile with distributed team, scaling Agile teams, creating a kaizen culture across many teams and continues to deliver more business value.

Collaboration tools

Looking back at the conference, I also see a trend for more collaborative Agile tools. Many people talked about tools to support better collaboration and communication. I have a number of business cards from different venders, who created a new Agile tool with more collaboration and asked me to have a look at the free version. It will be interesting to see how those tools will evolve in the future.

I also talked with some people from EMC | Conchango and played with their integration between TFS and a smart board. See picture below. Using the smart board, you collaborate with stories from TFS, do estimation with planning poker cards and have the values saved directly back into TFS. A fun and costly solution.

EMC| Conchango software integration a Smartboard into TFS

A cheaper solution is the Wii virtual whiteboard from Johnny Chung Lee. http://johnnylee.net/projects/wii/. It can be a easy way to make a virtual task board, see also this YouTube video.

In this slice about inspiration from Agile 2009, I will write about the inspiring elements from different sessions and discussions with other participants. It will be more facts than thoughts and trends (I will cover that in later blog posts, slicing the Agile 2009 conference).

For me, the primary source of inspiration was from the workshops involving some kind of games. Games can be a great way to learn about different practices, principles and tools. Especially because they don't have elements from your daily work, so the collaboration between the participants often will be much better and focus on the complete team, a shared goal and not specific roles (well, well, I know we have all those agile cross functional teams…but…).

I went to The Business Value Game with Portia Tung and Pascal Van Cauwenberghe. It was a game about how to deliver maximum business value, prioritization, estimate business value and collaboration between different roles. The session was a bit chaotic with too many people in the room, but it got some really interesting concepts. It was also interesting to see how people got engaged in the game. I observed many people focusing quite a lot on learning the game rules and less on optimizing the business value. For example in my team, we did not talk about remove stories from the game to release more frequent, but tried to maximize the delivery of ALL stories. Maybe it was because the session had too many people and we did not have the time to think on that strategy, but it might also be a pattern in many real teams?

The Business Value Game

Portia and Pascal also facilitated the The Bottleneck Game, and some of my colleges from BestBrains went to that workshop. It was a game about options, Theory of Constraints, System Thinking and collaboration on a common goal. It is a great way to learn those agile and lean techniques. I will defiantly use this game with different teams.

I also went to The Kanban Game with Tsutomu Yasui. It was also a workshop with too many people and it took some time before we had enough problems to manage in the iterations of the game. After the first iteration it got more interesting and had some good elements. I think the game could be much better, by not having the first iteration and maybe facilitate the usage of different Kanban elements.

The Kanban Game

Another fun and inspiring workshop was May the Forces Be With You, Exploring the Forces Driving and Restraining Agile with Rod Claar and Douglas Shimp. We created two teams, the Drivers and the Restrainers and had to present different forces in a humoristic way for some selected judges. It was a fun way to explore the forces driving and restraining agile

May the Forces Be With You

"Flirting" With Your Customers with Jenni Dow and Ole Jepsen was a fresh way of looking at the customer relationship using 8 steps to build a good customer relationship. It was fun and interesting.

Flirting With Your Customers

The last session I will include in this slice, was a workshop, First, kill all the Metrics!, with Niel Nickolaisen and Chris Matts. We talked about many potential meaningful metrics, but did not find some really good examples in my group. I might reflect more on this area in a later Agile 2009 slice.

First kill all the Metrics!

Talking with other participants were also fund and inspiring, but I did not had the time to talk with enough people. It was inspiring to hear agile and lean stories from the trenches around the world, even though many people and teams seem to struggle with retrospectives and problem solving techniques.

"Pfizer could have demanded a discount from firm's hourly rates, Ms. Schulman said, but she hopes for a shift to a system that encourages firms to work more collaboratively with Pfizer."

• Scope described loosely in a few paragraphs, a kind of vision statement
• An hourly price, that is 10-50% below what is normal for pure time-and-material
• A set of milestones, which will lead to payment of a fixed amount. The simple criteria that tells that a given milestone has been reached, is whether the software is deployed in production.
• A development process following agile practices
• A suggested time frame for the overall project and for each milestone

• Focus on bad news first: That makes the organization focus on constant improvements (Toyota does it).
• Copying Toyota is also copying someone else solutions to their problems and that is not solutions to your own problems. Lean from Toyota and improve it.
• To make real user value, you have to see and understand it from eyes of the customer/users.

Outlook and Thunderbird are two examples of programs that aren't laptop ready. The reason being that the SMTP setup is fixed. Most SMTP servers have IP whitelists, meaning that only a few, known IP addresses are allowed to send mail through them. Therefore, the SMTP server that is used should vary with my current IP address which changes all the time.
Apple Mail solved this problem - if an SMTP server rejects the connection, it will try the next known SMTP server until a connection is accepted.

I am quick to tell developers off if they seem too attached to a metaphor. In this case however, the mail metaphor actually works really well. You have a personal mail box where mail sent to you ends up. And then you have the mail boxes on the street corners for outgoing mail. Those are perfect analogues to POP3 and SMTP. Outlook uses the metaphor of an account where stuff enters and leaves, which means that an account knows about a POP3 server and an SMTP server.

1. Increase taxes       
2. Make people work longer
3. Cut budgets

1. Increase prices on products and services
2. Lower average employee salary
3. Layoffs

The name of the tool is Remember the Milktm . It is a free, webbased tool. What I really like about it is that it  is simple, easy to use, and at the same time extraordinary flexible in terms of input and output.

The core of the product is a set of tasklists, each containing tasks, that can be prioritized, have a due dates, estimates and a few other things. It is easy to create and remove lists. 

But what it really impressive is the numerous ways to enter new tasks and the equally impressive amount of ways to manage and be reminded of tasks:    

First of all there is a nice Ajax based Web-UI. From that I can enter  and maintain all my tasks as well as my lists. That probably the least important part of RTM. 

I mostly interface with RTM through my iGoogle Dashboard in Firebox.  I also have access to my tasks via Gmail, Google Calendar, Skype and on my mobile phone.   Every morning I get a friendly email, telling me which tasks are due today, and if they are due at a certain hour, I get a reminder at that time too. Besides iGoogle, Google Calendar and Gmail,  I can enter tasks on my mobile phone or through email.

Try it out! - maybe you too will never need another tool to manage your tasks

Last weekend I took a minor holiday to the german capital, Berlin. On sunday morning I went out to find a place that served breakfast and I came by a nice litte bistro. I was with a small group of friends, so we ordered breakfast based on personal preferences. I had scrambled eggs, some had coffee, bread and croissants, and so on. After taking our orders the waitress went inside (we were sitting outside) for a few minutes. Then she came back -- or so I thought. She went by us, turned left, and entered the bakery that was located pretty much right next to the bistro. A few moments later she came out with a plastic bag that seemed to hold exactly what we ordered. She got back inside the bistro again, and a couple of minutes later our breakfast was served.

Reflecting on what I just saw, I concluded the following. Right after taking our orders she went inside and started on the scrambled eggs. At that time she probably notified the bakery on exactly what items to prepare for here. Afterwards she went out and got the bread that we had ordered, but no more.

Lesson learned. Keeping trusted key suppliers close, allows you to have very low inventory of the most "problematic" items. For instance freshly baked bread, which decays very fast, and is not suited for selling after a matter of hours. Even if you pay a higher price for picking up bread in smaller batches, the bakery -- not you -- will end up with the bread that can no longer be sold, and thus you will rarely have to throw out food. The strategy also allows you to spend the scarce space you might have in your bistro, on more value adding abilities, like setting up a table with a blender and serve smoothies!

The bakery is probably happy with the setup, as their incomming orders are now levelled out on the entire day, instead of having to prepare big batches in the morning. Furthermore the customers are probably more happy as the average time from bread leaving the oven until it is served, is now shortened, resulting in bread that is still slightly warm, and tastes better.

Some thoughts that popped up on my first day of attending our Poppendieck course on 23th of June.

Knowledge inventory

Knowledge briefs

Will Agile Software Development end on the dumping grounds of History? This June Mary Poppendieck will present her view on how the move to agile software development, needs to be done cautiously to avoid repeating the mistakes the IT industry has previously encountered while adopting new methodologies and paradigms.

June 23th and 24th, BestBrains is arranging a 2-day Lean Software Development course in Copenhagen with lean software pioneers Mary and Tom Poppendieck. In connection with this course we have also arranged a free lecture June 24th, at danish IT-Universitetet. Mary and Tom are well-acclaimed authors of the popular books "Lean Software Development" and "Implementing Lean Software Development", which describes how to use Lean principles to improve software development organizations.

See more about the comprehensive 2-day course here (or in danish here), and the free lecture at ITU here.

We are looking forward to seeing you there!

Yesterday Jesper and I was doing an agile workshop as part of the Öresund Agile 2008 conference in Malmö, Sweden. I have to admit we had set out being pretty ambitious about the content of the workshop. We stuffed everything from Release planning, charting, coding, test-driving, continuously integrating, doing reviews, retrospectives, scrum meetings, planning poker, and discussing engineering practices and process improvement into just one day. I think we can conclude the workshop fully lived up to it's name -- Accelerated Agile!

With Malmö filled with swedish football fans, students celebrating final exams, and of course a lot of skilled people from the agile software community, we had some great days. Not only was it cool doing the workshop itself, but being at a smaller software conference where you actually get a chance to chat with most of the attendees I think is really valuable. I hope everyone else enjoyed it as much as we did (in time the feedback forms will tell!).

As promised during the workshop, we have of course put the latest (and hopefully greatest) version of all the slides online. You can download them from here.

After browsing through the pictures that was taken during the day, I selected some that you'll find below.

Notice the red and green lava lamps in the background. These were controlled by the CruiseControl CI server! I personally think that this was actually more than just a gadget. It brought great visiblity into the project. With these lamps connected to the build system, everyone who entered the conference room (or even looked through the door) could immediately spot if there was currently a problem. Now that is TRULY surfacing problems fast!

The self-organizing team working towards a common goal?

Some laid back discussions of interesting engineering practices.

Our one-day iteration ended with a retrospective, that judging from the smiles were rather positive.

All sorts of important stuff on the wall. Tasks, burn-down, WIP chart, and standard work sheets.

A big thanks to all the dedicated attendees, conference coordinator Gustav Bergman from Softhouse who invited us in the first place, and also Sonny, Lars and Thomas Lundström who assisted us during the entire day!

Wednesday we visited NEC and Change Vision, both located in Tokyo. At those visits we got the chance to see one of the major Japanese corporations in the IT industry, as well a small software company.



We started out the day by visiting Change Vision, a company founded by Japanese author Kenji Hiranabe. We got a presentation of the two products of the company, JUDE and TRICHORD. Both seemed to be built on the idea of "mieruka" (making visible), which is a concept used by Japanese organizations for sharing information by making it visible in an easily understandable form.

While noticing the extensive use of simple visuals for tasks, metrics, ideas, releases and retrospectives, I have to admit what personally struck me the most was the bug-tracking system of Change Vision. It was visualized using LEGO bricks in a constrained physical space. This provides kanban-like control of the "bugs in process", by showing not only the variation in complexity of bugs, but also making it easy for everyone to see when you are running out of "slots" for new bugs, and thus have to start fixing some. Personally I just can't wait to try this visualization idea back home.

Judging by the discussions that took place after the visit, I think we all got a lot of inspiration from visiting a truly agile environment.

Later the same day we visited NEC for a techical presentation as well as a presentation focused on their development process. During the visit we learned that NEC had experimented with using Toyota Production System (TPS) a.k.a. Lean in the development process, for instance with the use of a kanban system to control the workload between different parts of the organisaion. Most surprisingly the system was put into place for controlling hand-offs between design, code, and test-phases. I think that as a group of agile thinkers, we were a bit stunned by this implementation.

NEC had also experimented with the TPS "stop-the-line" practice, in their development process, but had abandoned it due to inefficiency. Unfortunately we did not get a chance to hear the reasons that led to this conclusion.

(BestBrains getting a demonstration of Japanese innovation. This is an NEC electronic whiteboard with a built-in printer!)

Tuesday was Toyota day. Destination: Toyotas Tsutsumi plant in Toyota City just outside of Nagoya.

To get there we had to take the Shinkansen from Tokyo Station. Seeing  the  Shinkansen 700 series silverarrow slide into the station, be seated in the comfortable compartment and drive silently towards Nagoya with a speed of more than 300 km/t was an adventure in itself.

In the factory, we were guided through one of the assembly lines, and the automated welding process.  At the plant Toyota produces Camry,  Premio,  Allion, Prius, Wish and Scion tC.

These models are produced on two lines, in a mixed production each car on the line is a  different model and color from the previous one. Only that way Toyota can remove the waste of overproduction by not  producing cars, that has not yet been ordered.

The first operation we saw was the door assembly. Toyota uses door-less assembly where doors after painting the doors are removed from the body and they each follow their own route to later be united again in final assembly. The advantage of this is easier access to the inner parts of the car, and less damage to the doors.

The best analogy to what we saw is a ballet. Workers did their work with a choreographed set of movements, designed to be the minimal effort to do the work and ensure good ergonomic.  Nobody was running and nobody worked ahead of the line.  The spirit of Taiichi Ohno was strong and alive.

Surely the work of producing a lot of completely similar things can never be as varied as other kinds of work in the modern world. However our whole standard of living is based on the society’s ability to produce low-cost, high quality items. And Toyota is able to transform this into a system that protects the individual and gives everybody challenges,  by having everybody involved in improvement efforts – Kaizen.

We also saw several examples of Andon pulls and stop the line philosophy.  Here it was clear that while the Stop The Line philosophy is what creates quality and is making sure systemic problems is eliminated quickly; it is also the lubrication that keeps the line running smooth.  Most of the Andon pulls was just small problems, that was solved immediately by having the supervisor help getting things back on track.

Further into the factory we saw the supply lines to the door assembly. A clever worked out system allowed a worker to create “kits” for each set of doors coming along the line.  By having models be mixed on the individual level (I do not know if any other manufacturer does this?), a need for standardizing is also created.  Assembly will be easier, when the parts are the same in more models.

Finally in the end of the line new cars flowed into to final verification:  a Blue Prius followed by a Green Camry, Followed by a Scion. Impressive.

(we were not allowed to take photos - the picture about is a official photo from the plant)

We sometimes hear Lean be accused of being just a new cost savings philosophy.  Yet another clever trick from management! But Toyota shows it is possible to create a challenging, respectful workplace in what is normally considered the most de-humanized kind of work (think of Chaplins “Modern Times”) . So why shouldn’t we be able to do that to in other areas, It should be easier,  not more difficult to apply lean to healthcare, service,  and Software Development by the way.

(Some of the participants in the Toyota Museum)

The day ended i a much more informal, but not less interesting way. We met with a group of members of XPJUG and spend a couple of hours with mutual presentations and exchanging ideas. After that we had a great Japanese dinner with lots of discussions and laughter in a cozy restaurant. Thanks to all the nice people from XPJUG to receive us so warmly.

Kampai! In the picture is Kenji Hiranabe, Author, Translator and much more and Tsutomy Yasui, who helped arrange the evening

1. Tech Story - 24 story point (As a developer I can ... creating the common component)
2. User Story A - 4 story point
3. User Story B - 4 story point

1. User story A - 28 point
2. User story B - 28 point

 //
Test pressing a key 10 times var i;  for(i = 0;
i < 10; ++i) { device.key("a");
} 

The "device"-object exists inside the test engine and is written as a .NET class:

publicclass DeviceProxy
{ publicvoid key(string key)
{ sendCommand("key
"+ key);
Console.WriteLine("Pressing
'{0}'", key); } } 

 ScriptControl
sc =new ScriptControl();
DeviceProxy deviceProxy =new DeviceProxy();
sc.Language ="JScript";
sc.AddObject("device",
deviceProxy, true);
sc.AddCode(File.ReadAllText(scriptPath)); 

Just the other day I was thinking about the core ideas of Agile software development, and I believe I came to an interesting conclusion. I think the core of agile development is all about one thing -- Aggressively attacking risks. Well, maybe there is more to it ;-) but that is what I'll talk about in this blog entry.

Whether you start up a new software project, or join an existing team that has been constructing something for years, you need to consider what the risks to the project are, and especially what you can do about them.

Risk of not shipping on time (or not shipping at all)
Practice makes perfect. Start shipping from day one -- That is, for instance ship a new release every week. Naturally it depends on the kind of product you are working on, but more than often this is perfectly feasable. In fact, this happens every single day for many web-applications, and users may not even notice.

Risk of building the wrong thing
Have developers work closely with users throughout the project, and encourage them to build an executable and testable model of the business domain, without too much focus on integration. This ensures that the product can in fact solve the core business problem.

Risk of not being able to integrate different parts.
If there is a risk of integration issues, practice! Integrate risky parts from day one of the project. For instance, communication protocols that you haven't tried before may be risky. While accessing a database may be a walk in the park. Find out which integration points require early integration, and balance those efforts against the efforts involved with the risk of "building the wrong thing".

Risk of high long-term maintenance costs.
What you need is a software solution to a business problem. What you don't need is unneccessary technical complexity. Aggressively attack complexity, and make sure your product is still capable of solving the business problem.

Risk of shipping software with defects
Where are your defects? In the business logic, or at the integration points between different modules? No matter which, the answer is simple: Test it (automatically) -- From day one! And use every single defect found, to improve the development process so that you introduce less errors in the future.

Risk of unacceptable performance issues
Have your team commit to a measurable performance goal. ...And you never guessed -- Measure it from day one.

The key to long-term success, is knowing what your long term success factors are, and figuring out how to address them on an everyday basis. What is our long-term succcess factor? I bet you are looking for a cost/earnings ratio close to zero viewing at the entire lifecycle of the software product. This is done by either increasing the earnings, or lowering the cost of development and maintenance. Also a short time-to-market is most likely attractive to your company. Thus, address those every day.

That's it for now. Keep working on reducing your risks out there.

Suspicion began to fall on the jvm performance (Suns Jvm for Java 1.4.2) and some logging parameters was added to one of the servers in a production cluster.

java -Xloggc:gc.log -XX:+PrintGCDetails

Looking at the gc.log in a graphical tool showed this picture:

Notice that the jvm has over 700MB memory but only uses around 200MB because a full garbage collection is run every minute, and that most pauses is between 0.6 and 0.9 seconds. This interrupts the execution on the server creating the unwanted pauses.

Investigating the subtle workings of garbage collection revealed that remote method invocation (RMI – when a remote client calls an object on the server) has a role to play in how often the server-side jvm performs a full garbage collection. By default the RMI calls System.garbageCollect() every minute, which can give the long pauses.

Increasing the interval between full garbage collections to 30 minutes (here both for client and server) helped to reduce pause considerably.

java ... -Dsun.rmi.dgc.client.gcInterval=1800000 


-Dsun.rmi.dgc.server.gcInterval=1800000

One thing to note is that increasing the memory will only make the pauses longer because more garbage would be able to fill up until a full garbage collection needs to be performed. So putting lots of memory blocks in your box and boosting the jvm with more memory does not necessary help the user experience. Instead you have to suit to the actual needs of your application, for example based on how many objects are cached.

Further more on a multiprocessor server, you can benefit from enabling a concurrent garbage collector that runs in a separate thread, so the main threads are not interrupted.

java ... -XX:+UseConcMarkSweepGC

With this collector you get lots of small garbage collections in the background and practically no long pauses in the application. Notice that here we also set the initial size of the jvm to 256MB letting the jvm itself determine how much memory is needed.

So remember to dive into your jvm when performance issues arises.

Want to read more:

http://www.tagtraum.com/gcviewer.html

http://developers.sun.com/mobility/midp/articles/garbagecollection2/index.html

More general on jvm options

http://java.sun.com/javase/technologies/hotspot/vmoptions.jsp

If you attended the JAOO 2007 conference, or follow the Test-Driven Development community you may have heard of Jim Coplien and his arguments about why you should not use TDD. I believe that Jim said something like "TDD will deteriorate your design", and it made me ask myself -- Is TDD a controversial topic?

So is it? Not if you ask me. If you rewrite the original Red-Green-Refactor mantra from TDD, into "Spec. the feature" > "Implement it" > "Reduce technical debt", it seems to make quite a lot of sense. In fact, I don't really see any other way you could work and maintain your professional integrity!

So is there any risk of bad design or architecture as a result of using Test-Driven Development? Well, only if you neglect that refactoring is a continuous activity of TDD. If you refactor (small and large refactorings) on a continuous basis, I simply do not see how you could have a design- or architectural meltdown. If anyone experience this, I seriously doubt that TDD is the root cause of their problem.

Check out Roy Osherove's blog entry on the topic here .

Three important principles of agile development are:

1. End to end development of small batches of work
2. Make problems visible
3. Limit work to capacity

Unfortunately it is not uncommon for teams, adopting agile methods like e.g. Scrum to somehow miss these principles.

I have a number of times watched teams plan and estimate work, they believe they can do in one sprint. After a few days, it is clear that the team lag behind. The immediate reaction is then to start to try to work faster, either by putting in longer hours or by sacrificing quality. In that way a Scrum sprints really live up it's name – a headless rush towards the goal. The result of having sacrificed all three principles mentioned above is a demotivated team and a growing pile of technical debt.

It is important to realize that the great benefits of agile development, will take some time to manifest themselves, and the best way to make them manifest themselves, is to avoid focusing too much on getting a lot dot done when starting to do agile development. Instead you should look carefully on how things are done.

For instance it is a common problem that developers lack information about features they need to work on, which will either slow them down or in order to not be slowed down, make them start guessing what it is they are supposed to do. This is a problem that should be made visible and dealt with at its root cause.

Another common problem is that features are not completely done when the iteration ends. Either they are not doing what they should, or is has not been verified that they actually work.

These two problems are just examples of problems a team starting to do agile development can and will encounter. It is important to embrace problems as opportunities for improving processes in a way that will create a more lean and flexible development environment.

So in your agile adoption:

admit we are all tortoises and we will not become hares just because we do agile development!

if (assertEnabled)

{

Thread.Sleep(100);

}

if (assertEnabled)

{

Interlocked.Increment(ref writerCount);

Thread.Sleep(100);

Assert.AreEqual(1, writerCount);

Assert.AreEqual(0, readerCount, "trying to write while reading");

}
// the implementation of the method go here
if (assertEnabled)

{

Interlocked.Decrement(ref writerCount);

}

   [Test]
public void WriterWaitForReader()

{

Thread r1 = CreateReader();

Thread w1 = CreateWriter();

r1.Join();

w1.Join();

Assert.That(exception, Is.Null);

}

   rwLock.AcquireWriterLock(InfiniteTimeOut);
if (assertEnabled)

{

Interlocked.Increment(ref writerCount);

Thread.Sleep(100);

Assert.AreEqual(1, writerCount);

Assert.AreEqual(0, readerCount, "trying to write while reading");

}
// the implementation of the method go here
if (assertEnabled)

{

Interlocked.Decrement(ref writerCount);

}

rwLock.ReleaseLock();

Creating quality software on a continuous basis is a complicated business.

I think it is a sign of maturity if your team can admit that there are daily problems which are ultimately obstructing flow. While not everyone will appreciate that your team does a complete halt until the obstruction has been removed I think it is a necessity for a team to stay in good shape and continue to deliver frequently.

Just the other day I came to think about what were the most common obstructions of flow we see in software development environments. So here is my preliminary list -- I bet you have seen them all.

1. Build breaks
If you don't have a build, you don't have anything. Nobody is able to do any progress progress as the state of the product is unknown. Developers cannot check-in, as they will not get any response from the continuous integration environment. Testers cannot try out the latest build. Users cannot see the product in action, and the list goes on. A build break is like a blindfold, and you should treat it as such. Don't try to go anywhere until you have your vision back.

2. Test failures
If you have a failing test, you have dysfunctional software. Imaging somebody fitting some cool new styling parts on to a car with a busted engine. Now, who would ever do that? It really is the same with software. Why do any feature work on a software product which has a failing test? I know this can be hard to respect, especially when a deadline is closing in on the team. Deadlines however, does not justify continuing without stopping the line upon test failures.

3. Bug reports
If you have bug reports, you have dysfunctional software. There is no need to do any feature work, until the product is stabilized. Bug reports are even worse than failing automated tests as those bugs can only be reproduced by a human. You should not spend work hours on anything but writing tests for these bugs, and fixing them.

4. Any other blocks keeping the team from a steady flow
If you are experiencing an obstruction which is keeping from moving forward as fast as you could, suggest to the team that you stop doing anymore feature work, until you are able to eliminate the obstruction. One obstacle which I think is quite common is builds that take way to long time. "Yeah, but how can you build 2 million lines of code in less than two hours?" -- Well, I suppose you could start by reorganizing your code and build only the module related to a particular source control check-in. Hopefully you don't have modules consisting of 2 million lines of code!

I'd like to hear about any other obstructions you may have experienced in a software development environment.