Dashboards promote ignorance

It is possible to collect a wide range of metrics on a project. Metrics about the codebase, unit tests, performance tests, build, delivery progress etc. It is also possible to define  red, amber, green thresholds for each metric. Some go for standardized organization wide thresholds while others are comfortable with project specific ones. The obvious next step is to create a dashboard that rolls up all these metrics into a single red, amber, green project status indicator. Middle managers spend a good part of their life managing inputs to these dashboards. Some "programme managers" assiduously refer to their dashboards as balanced scorecards.

Trouble is, most of the time, an overall green status indicator doesn't mean anything. All it says is that the things under measurment seem ok. But there always will be many more things not under measurement. To celebrate 'green' indicators is to ignore the unknowns. The value of a metric is not when it is green. Lets take unit test coverage for example. Is 100% test coverage a cause for celebration? What if most tests don't have any assertions? When measurements become targets, they encourage gaming. On the other hand, a coverage of 50% at least tells you unambiguously that half the code is not covered (or there is an error in measurement).

One may argue that it is only a matter of measuring even more so that the overall green becomes truly indicative of overall project health. This is somewhat unrealistic in a fast paced knowledge work environment. Tools and technologies keep changing. Measurement tools don't keep pace. It takes significant effort to maintain the measurement infrastructure on a project.

Avoiding subjectivity in assessments is also cited as a reason for resorting to metrics. But that's like saying that a person can be certified healthy without the judgement of a physician by a dashboard that rolls up a hundred diagnostic tests.

In my experience, metrics are much more useful when they report bad news than when they are green. In the words of the Way of Testivus, "Good tests fail". Unfortunately, the tendency to roll up metrics into dashboards promotes ignorance. We forget that we are only see what is under measurement. We only act when something is not green.

Then comes the final argument. Don't blame the tool for human laxity. Dashboards cannot promote ignorance or even wisdom. Unfortunately, tools aren't value neutral.

Build time impacts team performance

Velocity often grabs a lot of managerial attention on a project. But while velocity is the effect, build time is a common cause. A slow build has several insidious side effects. Say a build takes 20 minutes:

10:00am	My pair and I work for the day by doing a fresh checkout and initiating a local build. 20 minutes to kill? Maybe check email etc, talk to analyst about my story, take a coffee break. OK, build passes.
10:20am	Work begins.
10:50am	We are ready for checkin. Oh no, we can't wait for a 20 minute build every half hour can we? There goes the motivation for frequent checkins. We decide to develop locally for another hour at least before trying a checkin. Signs of discontinuity in continuous integration!
12:00pm	Ready for checkin again. Run local build.
12:20pm	oops. Build broke. Our code broke some old test. Good catch by the test suite!
12:30pm	Attempt another build after fixing the code.
12:50pm	Build passed! Now we just need to check once again after merging with latest code from the repository. Is the build green? Yes. Safe to checkout from repository. We checkout, couple of auto-merges, no conflicts. One more local build and we are good to go.
1:10pm	Local build passed. We are about to checkin when we notice that the build is yellow. Darn, someone just checked in. Now we have to wait for the build to go green, then checkout and build locally again before we checkin.

Anything short of this rigour leaves a window open for build failure on the CI server. But hey, surely you can't expect us to play the waiting game indefinitely. What if someone checks in again while we are diligently verifying locally? Besides its time for lunch. So we gamble and checkout. No conflicts or even auto-merges. Good sign. It isn't worth building locally again.  We checkin and go for lunch. Halfway through lunch, I get a call.

"@#$!, why did you checkin while the build was still running?"
"Why what happened? Don't tell me your build didn't go through"
"Yes it didn't and now I am having trouble fixing it on top of your changes"

As build time increases, it tests my patience as a developer. I am tempted to take shortcuts that occasionally backfire. When this becomes standard team behaviour, the occasional turns into regular. Secondly, it increases the syncing window (12:50pm to 1:10pm above). This window is at least equal to build time. Greater this window, greater the chance that someone else might check in while I am still getting ready.

Now what if the build time were five minutes instead? There is much less waiting time associated with frequent bite sized checkins. So it encourages frequent checkins. This in turn reduces the likelihood of merge conflicts when I sync with trunk. Finally, if someone else does check in during my syncing window, I only have to wait five more minutes to see if it goes green.

In summary, everything else constant, team performance correlates well with build time.

So what to do?

Keeping a build fast is a topic by itself. Basically try to keep the unit tests fast. At an average of 20 millisecs per unit tests, we can run 5000 tests in 100 secs (just under 2 minutes).  Avoid all I/O in unit tests. If you must have some unit tests that do I/O, keep them in the next stage of your build pipeline. Don’t write unit tests that exercise a lot of code. They aren’t unit tests if they do.  As a manager, be very concerned if you see build time going up steadily.

The point of Object Orientation

Transaction script cost escalation

A common characteristic of large codebases is that it takes disproportionate effort to add or extend functionality. Couple of common reasons:

1. Hamstrung team - The original developers may have moved on, development beyond the first release may have been transferred to a less capable team or handover may have been ineffective.
2. Poor codebase - Test quality and coverage may be lacking, encapsulation may be broken,  or the build may be inordinately long.

Bad encapsulation is a common culprit. All too often, OO-capable languages like Java and C# are used to author transaction script1 applications. The business logic is use case driven rather than domain driven. Use case driven code is characterized by global code acting on global data. Well encapsulated codebases, on the other hand, consist of collaborating objects, each object encapsulating its local data and behaviour.

For a typical business app, everything else remaining constant, good encapsulation will lead to constant or decreasing delta effort for delta functionality. The point of object orientation is that it is meant to help manage complexity of the business domain. This can only happen when we encapsulate along the grain2 of the domain.

1:http://martinfowler.com/eaaCatalog/transactionScript.html
2:Cutting with the grain: the rhythms of design - Kent Beck