Companies looking to get the benefits of a DevOps culture often get stuck in the trap of focusing on buying the "best" tool and implementing "best" practices.1 Tools are chosen with very little, if any, collaboration with developers, testers, or operations staff. "Best" practices are determined by committees, again based on scant practical organization experience. While there are benefits to standardization, premature standardization, or standardization with no room for experimentation leads to stagnation and poor overall performance. And what works today will be unlikely to work as well in the future. The pace of change and innovation in DevOps means new opportunities for improvement are arising regularly.
In his article, "The Three Ways of DevOps," Gene Kim (coauthor of The DevOps Handbook and the Phoenix Project) talks about the need for experimentation and risk-taking on the path to improvement.
"The Third Way is about creating a culture that fosters two things: continual experimentation, taking risks and learning from failure; and understanding that repetition and practice is the prerequisite to mastery."2
Process and tools exist to support an organization's strategic intent. The purpose of experimentation is to focus on how to best move forward at this moment in time. There is nothing wrong with how things were done in the past. They may not be the best way to proceed now. The phrase "best practice" is misleading. More accurate would be to say "this is one of the better practices we've been able to discover so far".
"When institutions fail to distinguish between current practices and the enduring principles of their success, and mistakenly fossilize around their practices, they’ve set themselves up for decline."3
Here's an example. An organization values delivering high-quality products to their customer. The current practice is to have automation engineers use a proprietary tool to build UI-driven tests. Should teams be permitted to experiment with new approaches, or required to stay with current practice?
A current way of thinking is that quality is the responsibility of the entire delivery value stream. The line between development and testing is blurring, and developers are now taking on responsibility for more test automation. Experimenting with tools that enable greater end-to-end accountability for quality might better support the principle of high-quality delivery. But you will only know if you try it. And no matter what other organizations have found, your experience will be unique.
Yes, lack of any coherent tool and process strategy can lead to a proliferation of tools and process along with a general lack of coherence. That isn't good, but neither is rigid standardization. Developers, testers and operations staff are professionals, and their ability to apply critical thinking to their work, and to make changes when needed, is essential for effective software delivery.
Some organizations tackle the desire for standards by choosing and mandating standards. In some cases, this may be completely appropriate. For example, if you have a tool for checking the security of code or conducting a penetration test, you wouldn't want that to be an optional process step. However, the more you mandate, the less likely you are to get the continuous improvement you'd hope for in an organization with a DevOps culture. Mandated standards are for the few areas where you can’t be effective without them. And, when you are developing centralized standards, collaborate with the groups who will be ultimately asked to use these standards to get their input.
If the message you send to teams is "here is a long list of tools and practices we expect you to follow, and by the way, don't forget to innovate," you aren't likely to get much innovation.
Other organizations have a standard set of tools, proven valuable over time, that are well-conceived, supported and easy to use. They don't mandate many standards, but they make following a standard path easy to do. So easy, many teams will choose to use the standard tools and practices. However, because the tools and processes are not mandated, teams feel free to innovate if they think they have a better approach. This supports the culture of the Third Way, spoken of by Gene Kim.
Innovations, if they prove valuable in multiple contexts across the organization, might be worthy candidates for further institutionalization and support. In fact, bringing a tool into a shared-services group for ongoing evolution and support can free up innovators to work on the next set of innovations.
Organizations moving towards a DevOps culture need to support a culture of experimentation, risk taking, and learning. Getting better often means breaking from standard ways of doing things. While organizations may mandate some tools and processes for critical parts of their deployment pipelines, ones that succeed mandate as little as possible and encourage teams to try new things in the pursuit of getting better. When tools and practices have proven themselves helpful across the organization, they are candidates for central planning and support.
Ready to start learn more about DevOps or start your own DevOps experiments? If you’d like help, learn more about our DevOps practices and Cloud Services Consulting offerings.