Measurement and Agile – Oil and Water? (Part 1)
In this four-part blog series, we’ll explore how to effectively measure metrics in the Agile space to help organizations and teams be more successful.
The truth is, measuring success in software development is difficult. As Edwards Deming said, “The most important things cannot be measured.” We’ve all seen how metrics can be gamed and misused (velocity, anyone?) and many have heard of the “Law of Unintended Consequences” and experienced how measurement can lead to unintended and counterproductive behaviors. This has prompted some to jump on the #NoEstimates bandwagon, arguing that we should stop measuring altogether.
But is it really that simple? The reality is that we operate in a business world where we’re asked to answer critical questions, such as:
-
What’s our customer satisfaction? How is it trending?
-
How are our Agile teams performing? Are we getting better?
-
How is our Agile/digital transformation progressing?
-
Are we realizing the value we expected from our transformation efforts?
-
What ROI are we seeing from our transformation investments?
-
For consultants and coaches: Am I making an impact?
-
Are we helping the organization deliver value faster?
-
Where should we deploy our limited coaching resources?
-
What bottlenecks in our value streams need to be addressed?
These are just a few examples of the questions we need answers to. How can we even begin to address these questions without measurements? As Peter Drucker famously said, “If you can’t measure it, you can’t improve it.”
So, instead of debating whether we should measure, let’s focus on how we can measure effectively, extracting value while avoiding the pitfalls.
Getting Started with Measurement
Before deciding what to measure, we need to clarify what we’re trying to achieve. The Goal-Question-Metric (GQM) approach is a great place to start. Here’s a simplified version:
-
Goal: What are we trying to achieve? (e.g., improve value delivery)
-
Question: What questions do we need to answer? (e.g., what is the time from ideation to value delivered?)
-
Metric: How will we measure this? (e.g., average feature cycle time)
If you prefer a lighter approach, consider Larry Maccherone’s ODIM framework (Outcome, Decision, Insight, Metric), which is simpler but still highly effective.
While many articles cover the establishment of organizational measurement frameworks, this post will focus on basic steps and recommendations based on my own experience.
1. Pick a Few Quantitative Measurements
Start with a few key quantitative metrics that are directly aligned with your goals and the questions you need to answer. Choose metrics that are easy to capture and meaningful. Don’t overcomplicate it by trying to measure everything at once. As you collect data, you can always expand later.
When considering which level to measure (individual, team, release train, program, portfolio, or organizational), think about where the data is being generated. Is it already captured in your Application Lifecycle Management (ALM) tools (like VersionOne, Rally, Jira, etc.)? If so, try to leverage that data before creating new systems to collect more information.
2. Pick a Few Qualitative Measurements
“Qualitative measurements” may sound like an oxymoron, but it’s about turning subjective data into something quantifiable. For example, to measure team happiness, ask team members to rate their feelings on a scale from 1 (unhappy) to 3 (happy). Calculate the average score across the team to get a numeric representation of their sentiment.
While qualitative data often requires extra effort (such as surveys or Excel sheets), it’s invaluable because it adds context that quantitative data alone may miss. For example, just as a doctor uses more than just blood pressure to assess health, Agile teams need more than just delivery metrics to gauge success.
Next Up
In part two of this series, we’ll dive deeper into how to achieve a balanced view of metrics and how to derive real value from the data. Stay tuned for more insights on making measurement work for Agile teams!
