Share
Organizational agility is challenging to achieve. Like any change, it comes with much hesitation, resistance, miscommunication, and a continuous need to reinforce the new way of working. However, once the transformation is complete and the long-awaited goal of being an Agile organization has been achieved, how can you assess whether things are operating better than before the change?
As experts who have assisted numerous organizations through this change, we often receive the question - "how do I know the impact of my Agile implementation? And if it is not working as planned, what is holding progress back?"
The answer is straightforward - measure the right metrics and gain proper insight.
Data-driven Agility is a continuous and systematic approach that utilizes data to evaluate the impact of improvement decisions, address inefficiencies, and optimize systems. It assesses how the organization's way of working is impacting the company. It goes beyond measuring the business' progress toward its KPIs. Rather, it focuses on the contribution of each component within the organization's modus operandi.
However, properly implementing Data-driven Agility is not as straightforward. It largely depends on the metrics you track. While well-thought-out metrics ensure you have the proper insight into what is happening within your organization, it's easy to focus on the wrong metrics. Successfully transformed organizations operate like a well-oiled clock, wherein various "gears" function harmoniously and synchronously. Responsive and Agile organizations perform at their best in ecosystems that are fully aligned and optimized.
Over the year, we have observed recurring metrics anti-patterns that organizations fall into that lead them down the wrong road and result in inaccurate insight. This blog discusses those anti-patterns in detail so you don't fall into the same trap.
Anti-pattern #1: KPIs that are not linked to strategy or don't make sense given the nature of the business.
Issue: Either lacking a crisp strategy or focusing on wrong metrics (see also Anti-pattern #4)
Impact: This approach will lead to focusing on improving metrics that do not matter for your business. This can result in the team losing sight of the bigger picture and decreased morale.
Resolution: Make your strategy clear and transparent within the organization. Derive the set of KPIs that fit your strategic goals and the value streams that are to realize these strategic goals. Utilizing the GQM method can help you define the relevant metrics relatively easily. It is also imperative to revisit these metrics quarterly to ensure you adjust for changes in the business climate and internal circumstances.
Anti-pattern #2: Overemphasis on Velocity.
Issue: Treating velocity as an indicator of team productivity without considering factors such as quality, customer satisfaction and business value.
Impact: Focusing excessively on velocity may lead to teams delivering more story points but at the expense of quality or delivering value.
Resolution: Do not interfere with teams on velocity. They use it for planning purposes. Instead, offer support and inquire where they need help if their velocity trend is lagging.
Anti-pattern #3: Using vanity metrics.
Issue: Focusing on vanity metrics that look good on paper but do not provide meaningful insights into actual performance.
Impact: Teams may optimize for numbers that do not align with business goals.
Example: Lines of code is a recurring example. Why focus on the quantity of code without considering its quality or delivered value?
Resolution: Detect and remove vanity metrics from your metrics framework. The GQM method is a good starting point to discern what metrics indicate desired progress well.
Anti-pattern #4: Lack of Customer/Business-focused metrics.
Issue: Focusing only on Health, Improvement or Vanity metrics and neglecting metrics that measure the impact on business goals or customer satisfaction. (Not sure what health, improvement and business goals are? Find detailed explanation here)
Impact: Teams may lose sight of the overall purpose of their work, and the delivered features will likely not align with customer needs, leading to long-term deterioration of business performance.
Example: A typical example of this situation is when teams measure the Number of Features Delivered without consideration for the value of the feature itself to the end-user.
Resolution: Define metrics directly related to your business goals and customer satisfaction. Convert metrics about several features into outcome metrics that relate to value.
Anti-pattern #5: Using micromanagement metrics.
Issue: Using metrics to gain detailed team insights rather than empowering them to self-organize and make decisions.
Impact: Micromanagement metrics reduce team morale, creativity, and innovation as team members feel constrained by excessive monitoring and control.
Example: Task Completion Time is a metric used to monitor and control tasks at a very detailed level, focusing unreasonably on individual performance.
Resolution: Eliminate metrics designed to micromanage individuals and teams, particularly those linked to the superficial objective of achieving 100% resource utilization. This goal needs to be revised in an agile delivery ecosystem and should be discarded.
Anti-pattern #6: Ignoring qualitative feedback.
Issue: Relying solely on quantitative metrics and ignoring qualitative feedback from team members, customers, and stakeholders.
Impact: This approach can lead to missing valuable insights and perspectives that can provide a more comprehensive understanding of the performance and decelerate the business from reaching its goals.
Example: In addition to Next to Net Promoter Score (quantitative), you can use the Customer Satisfaction metric by having customers complete a satisfaction survey.
Resolution: Customer and employee feedback is precious; use this information. Make them feel heard and listen to what they tell you. This complements raw data and can often provide a much deeper understanding of what they want.
Anti-pattern #7: Easy-to-game metrics.
Issue: Introducing metrics without considering potential unintended consequences, such as gaming the system or encouraging suboptimal behaviors.
Impact: Teams may unconsciously manipulate metrics to meet targets, leading to a disconnect between the reported metrics and the actual performance.
Example: Number of Commits makes Developers check in code as many times as possible. Besides, this is a vanity metric and provides negligible insight to how the project is progressing. A second example is a narrow-minded focus on shortening the lead time (order to delivery) at the expense of the quality, resulting in decreased customer satisfaction. The remedy is to measure both lead time and quality.
Resolution: Understand how metrics relate to each other. This makes sure that gaming is detected early and dealt with accordingly.
Anti-pattern #8: Fixed scope metrics.
Issue: Setting fixed scope metrics without allowing for flexibility in response to changing requirements or market conditions.
Impact: Teams may feel pressured to adhere strictly to pre-defined plans, leading to resistance to change and a lack of adaptability.
Example: Number of change requests can metrics kill agile initiatives. After a certain point, this obstinance can no longer realize the most suitable product because you have fixed the solution.
Resolution: Always preserve options by leaving the scope flexible and remove metrics that no longer align with the goals.
Anti-pattern #9: Cherry-picking metrics.
Issue: Selectively choosing metrics that showcase success while ignoring those that highlight challenges or areas for improvement.
Impact: This creates a skewed view of performance and can lead to misguided decision-making.
Example: Impediment resolution time.
Resolution: Introduce improvement metrics for problematic areas. This will make problems surface so the urgency is raised, and improvement actions can be defined and executed. Note that hesitation to expose challenges also indicates a deeper cultural problem where individuals or teams do not feel psychologically safe enough to earnestly expose areas of improvement.
Anti-pattern #10: Pretending metrics are exact.
Issue: Pretending that the measurements are very precise, even if it involves something simple as counting. Impact: When a complex system is used to measure things, it's difficult to maintain and change when necessary. This makes the measurement seem more accurate than it really is, leading to misunderstandings and wrong decisions.
Example: A complicated way of measuring lead time is to base it on work time. This approach often doesn’t account for holidays, weekends, and off-working hours. But if, from the business point of view, an accuracy of 10-15% is enough, it is much simpler to just measure in calendar days.
Resolution: Keep the metrics simple and easy to use and interpret. When designing metrics keep in mind what the relevant business risks are.
Understanding and avoiding common anti-patterns in defining metrics is crucial for organizations striving for agility and continuous improvement. By embracing a data-driven approach and continuously refining the metrics framework, organizations can drive sustainable improvement and achieve true agility in their operations.
What's Next?
We now have defined valuable metrics that allow you to reach your goals. Regular inspection of those metrics will facilitate the conversation around potential improvements. You can use events like Lean Daily Management, Obeya Performance Review, or Quarterly Business Review meetings.
When running your business, metrics are more than just data. They serve as tools that guide your organization to success. By adopting the GQM method to define metrics, linking them to your goals, and inspecting and adapting periodically, you can ensure that your metrics remain relevant and effective for your organization.
Are you interested in learning more about Data-driven Agility? Check out our service page and book a free consult with our experts.