<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/e4cce3c9-422f-46b7-b0d3-a7f9531e806a/question-icon-t.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/e4cce3c9-422f-46b7-b0d3-a7f9531e806a/question-icon-t.png" width="40px" /> The most successful organizations establish non-negotiable baselines that they strive to meet above all else. They iterate as fast as possible to drive continuous learning and improvement. The starting values for your metrics don’t matter; what matters is establishing baselines and then continually improving through the use of a build, learn, and adapt loop.
</aside>
The gold standard for cloud software development and operations (DevOps) metrics is Google Cloud’s DevOps Research and Assessment (DORA). The DORA team conducted a seven-year research program to identify the technical, process, measurement, and cultural factors that drive optimal software delivery and organizational performance. The metrics they’ve identified are critical for measuring cloud software development and have become the industry standard for benchmarking teams against the industry’s elite. If you’re unaware of this research, I recommend reading the book Accelerate. It covers the analysis and explains why organizations that rate highly in their metrics are more than twice as likely to achieve their goals than those rated lower.
The metrics they’ve identified include:
<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/6a5cb252-4174-46ce-bb2a-6647855a6cda/blank-img.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/6a5cb252-4174-46ce-bb2a-6647855a6cda/blank-img.png" width="40px" /> Deployment frequency How often an organization successfully releases to production.
</aside>
<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/f468ba1e-11ea-4506-a4be-58fe32e6d7f1/blank-img.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/f468ba1e-11ea-4506-a4be-58fe32e6d7f1/blank-img.png" width="40px" /> Lead time for changes The time it takes a code change to get into production.
</aside>
<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/4d8432d2-ddd3-4634-987d-ac0e9bfe926d/blank-img.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/4d8432d2-ddd3-4634-987d-ac0e9bfe926d/blank-img.png" width="40px" /> Change failure rate The percentage of deployments that cause a failure in production.
</aside>
<aside> <img src="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/887ac348-9477-43c4-898a-5d9546790027/blank-img.png" alt="https://s3-us-west-2.amazonaws.com/secure.notion-static.com/887ac348-9477-43c4-898a-5d9546790027/blank-img.png" width="40px" /> Time to restore service How long it takes an organization to recover from a failure in production.
</aside>
Deployment frequency and Lead time for changes measure velocity, while Change failure rate and Time to restore service measure reliability. Google’s research has shown that teams who measure and strive to improve these metrics achieve better business outcomes. They’ve also established ranges for these metrics that enable companies to benchmark themselves against the industry’s elite. These metrics are a great starting point to understand how an engineering organization is performing, but in my opinion adding more measures and baselines will complete the picture while reducing the observer effects and adaptation to the metrics.
In the excellent paper “The SPACE of Developer Productivity,” researchers from GitHub, Microsoft, and the University of Victoria explore the topic of developer productivity and recommend using a mixture of metrics across several dimensions that apply to teams and systems. These dimensions form the acronym SPACE: Satisfaction, Performance, Activity, Communication, and Efficiency. Their research shows that each of these dimensions helps to form a complete picture of software team productivity.
Example metrics for the SPACE framework
The SPACE framework suggests a systematic approach to improving productivity by carefully choosing a set of metrics that are balanced across the dimensions and linked to specific goals or outcomes. The researchers specifically highlight factors that can help productivity. They reference the substantial impacts of work environment and culture, asynchronous processes, effective developer tools, and workflows. They also highlight the positive effects of agency, autonomy, flow, collaboration, and effective team collaboration.
Both the SPACE and DORA research and frameworks have influenced me quite a bit. They’ve also helped me crystalize what I’ve learned throughout my career about the important factors in building high-performing software engineering teams and measuring productivity.
