Metrics-Driven Agile Improvement: Turning Action into Traction

DSP's Head of Agile, Nils Hagner talks to us about the importance of a Metrics within Agile and how this can boost value to your Agile developments…

“A number of years ago I met a businessman who was both horribly nice and horribly smart. When I asked him how things were going, he responded;

“There is a lot of action in my organisation, but unfortunately not much traction.”

Having worked in the Agile world for a number of years, this is something I have seen before. With continuous improvement being core to Agile, there must be ways some of that action can be constructively turned into traction. Indeed, I am convinced this can be done in a constructive and systematic way.

Continuous Improvement

Some studies have shown that Agile professionals see retrospectives and continuous improvement as one of the absolutely most valuable aspect of Agile methods. Used right, the continuous improvement leads to a good spiral, where both productivity and quality goes up over time, while release cycle time goes down. Moreover, most teams I have seen who have been on the “up” of the continuous improvement curve have also been the happiest teams. On the flip side, it is not unusual for teams that struggle to have internal tension and negative stress.

Agile Metrics

Now, let’s bring Agile metrics to the centre stage! Most people associate Agile metrics with velocities, bug counts and burn up/down charts imposed by senior management to exercise control over the teams and team members. For a fair few organisations, this is also the actual truth. This is very unfortunate, because the right type of metrics used in the right way for the right purpose can be absolutely vital in driving the continuous improvement process. Not only can it help turning action into traction, but also provide direct evidence of the improvement!

Metrics for Whom?

Agile metrics do not have to be defined by senior management or the programme level. On the contrary, the teams themselves can define their own metrics – things that are of specific value to them. Moreover, appreciating that a team’s metric is primarily intended as a tool for the team itself, they do not necessarily have to share them with senior management. If there are very positive things that the team wants to share then they can choose to surface the metrics as evidence. Similarly, if there are issues that the team wants to flag up, then they can choose to share the metrics to support their case for concern.

Get Creative!

What type of metrics should a team use? It is all dependent on the team itself, the environment they are working in, their challenges and priorities. Each team is unique and has its own unique pain points. Ideally the team has a hunch of what problems it is facing and can creatively define metrics that relate to those problems. The metrics will become the focal point of retrospective sessions and will drive the continuous improvement for the team.

Let’s look at some specific examples:

One team I worked with in the past consistently got the task breakdown for user stories wrong. During sprint planning, the team would come up with a task breakdown with a total of say 35 hours. When actually developing the story, the actual amount of work required could easily clock up to 95 hours. So, there is clearly a delta between the estimated tasks and the actual tasks, both in terms of time estimates and actual tasks. A natural metric in this case would be the average difference in percentage between planned amount of work for a story and actual required work. The team can then work on improving their method of task breakdown and thus drive down the difference. The result will be very visible over time thanks to the metrics.

Another example is from a team where there weren’t any automated regression tests. Furthermore, new features tended to break previously delivered features, meaning that a story which was delivered as “done” wouldn’t stay “done”. A natural metric here would be to start tracking the percentage of user stories covered by automated regression tests. Initially, the value would be 0%. The team could set its own target, say 40% and plan their work to gradually work their way to the target.

There are of course much simpler metrics that could still provide very good value. I have seen teams where stand-up meetings would take absolutely forever. Resolution? Track how long the stand-ups are, set a target and work towards the target. User stories without proper acceptance criteria? Track the number/ratio and agree on a target improvement rate with the product owner. The only limiting factor for what types of metrics can be used is the imagination!

How to Track it?

Once the relevant metric has been defined, the next problem is to track it. The worst case scenario is to go old-school, that is, to track the data manually. This can be a collaborative effort among the team. For some metrics, project management systems such as JIRA can help out. However, there are some gotchas to watch out for: In most tracking systems, it is very easy to get a snapshot view over how things are right now (i.e. momentary values), but much trickier to find out how they were yesterday or a week ago. If some metric needs to be collected on a daily basis, the values may have to be recorded elsewhere.

Sounds good! Where do we start?

At DSP we have developed an engagement model in which we go in and help teams through a couple iterations, changing the perception of metrics, helping them define and track metrics and helping them improve their agile and engineering practices in a way which can be demonstrated through metrics.”