What should an enterprise do to improve the performance of their DevOps practice? More, or less communication? Remove policies that keep the Dev and Ops teams in silos? More, or less automation?
There are a hundred different ways of looking at this. And we might still not have clarity on what one can do to have a higher performance DevOps methodology. But here’s something that can give us clues: 2017 State of DevOps Report by Puppet and DORA (DevOps Research and Assessment).
The report, now in its sixth year, received responses from 3200 people in the domain around the world. It “looks at the statistical relationships between IT performance, organizational performance, technical practices, cultural norms, and management”. While transformative leadership has emerged as key for DevOps success in this version of the report, we will keep our focus on the technical aspects that the report highlights.
Let’s look at what the report has identified as key parameters that drive high performers in the DevOps space. But first, let’s look at how high performers do better than low performers.
The survey found that in comparison to the low performing DevOps teams the high performers have:
- 46X more frequent code deployments
- 440X faster lead time from commit to deploy
- 96X faster mean time to recover from downtime
- 5X lower change failure rate
What does that translate to? The table from the report illustrates:
Given the metrics we see above, there is a lot to be learned from the high-performance DevOps teams and their DevOps methodology. What does the report suggest?
Elements of a High-Performance DevOps Methodology
Loosely coupled architecture to practice continuous delivery
If the team is replacing or modifying a component, or service, does it require them to make corresponding changes to the services or components dependent on it? If yes, then that’s a bottleneck for you to consider. For example, do minor changes to an application cause breakage? Or is the continuous integration/continuous deployment process crippled by deployments that are big and time taking? These are some warning alerts that could make you reassess your architecture. A couple of approaches for this include:
- Use of APIs, bounded contexts to decouple large domains
- Use of test doubles and virtualization to test services or components in isolation
Similarly, can delivery teams test, deploy and change their systems without depending on other teams? This could be for additional work, resources, or approvals. And can this be done with less back-and-forth communication? Do they need to check schedules with many people, just to get their job moving? The report found that high performing teams have less dependency on other teams thereby speeding up release cycles.
Daily code merging in trunk based development
Periodically merging code to trunk is part of the development workflow for most teams. However, what should be the periodicity or frequency of the merging? To get higher software delivery performance, merge code into trunk on a daily basis, have branches or forks with lifetimes less than a day, and don’t have more than three active branches at a time.
That may sound quite the contrasting approach to software development workflows being followed, but it works.
The survey found that teams that do not have code lock periods have higher software delivery performance. Developers in high-performing teams work in small batches and develop off of trunk or master rather than long-live branches, says the report. High performers have branch life integration lasting just a few hours, while low performers have it lasting days, typically.
More automation
The report says that high performers are doing significantly less manual work than low performers, and so have automated:
- 33 percent more of their configuration management.
- 27 percent more of their testing.
- 30 percent more of their deployments.
- 27 percent more of their change approval processes.
With more automation comes more time for the team to do value-creation activities, building out new features. The report cites that “by undertaking a continuous improvement initiative and investing in automation — including significant investments in automated testing — HP LaserJet was able to increase time spent on developing new features by 700 percent.”
What we've also learned, in our work with teams and in conducting research, is that people are not very good at estimating how automated they are. What they are good at is estimating the percentage of their work that is still done manually. That's not surprising: Manual work is painful, so people are highly aware of it. Once work is automated, it's no longer painful and tends to disappear from people's attention.
The table shows high IT performers report the lowest amount of manual work across all practices — and therefore, the highest amount of automation.
While medium performers do more manual work than low performers when it comes to deployment and change approval processes, the authors regard this phenomenon of manual work and rework as a temporary stage in which the organizations add more manual controls around changes, inevitably slowing them down. The recommendation is not to give in to the temptation and move change review process to earlier on in the development cycle.
Quality and security
It is one thing to be able to deploy code rapidly, and on demand. But what of quality and security? The report measures unplanned work and rework as proxies to measure quality, and found that high-performing organizations spend 21 percent less time on unplanned work and rework. So they can spend 44 percent more time on new work, such as new features or code. They also spend 50 percent less time remediating security issues than low performers. These results point to the fact that dev teams need to involve security and quality teams in the development process early.
These are some of the key technical intervention areas that can lead to a higher-performing DevOps team.
What are some of the issues your teams face in their DevOps processes? Drop a mail here to brainstorm about your challenges and explore how our DevOps consulting services can help you resolve them.
