Share
We know the many benefits of DevSecOps. When it comes to the development lifecycle, DevSecOps principles and practices are in high demand. But, although it’s easy to extol, there’s something wrong in the world of DevSecOps: It should be simpler.
Dragons (be here)
Companies are racing to adopt and take advantage of all that DevSecOps offers. However, in this race, many businesses uncover a lot of organizational friction when trying to integrate DevSecOps and make it add value successfully.
An initial problem is that DevSecOps isn’t all about technology and doesn’t have a clear roadmap. Those embarking on this journey without the right mindset and perhaps a knowledgeable guide quickly find themselves in uncharted, sometimes perilous waters. The increasingly complex landscape of DevSecOps tooling illustrates how easy it can be to become lost early on the DevSecOps adventure.
Fires
The DevSecOps journey can be treacherous, presenting a variety of obstacles along the way. Where there are dragons, there’s sure to be fire, and those fires are likely to affect the bottom line. Without the proper planning, tools, or personnel, companies aren’t prepared to fight the fires that arise:
- “Production went down, and we don’t know when or why!”
- “Conflicting migrations are chasing each other around the master database ring, preventing read replication long enough that our caches are expiring and content can’t be served on any of our websites.”
- “A bug fix that should have taken two minutes took four days to get into production.”
- “Deployment depended on a database migration that succeeded in the QA environment database, but the migration SQL failed in production due to data specifics, leaving things in a bad state.”
Fires can happen at any time, and for a variety of reasons. But customer trust in a product, or in the company as a whole, goes up in flames when a company is ill-equipped to fight them. The erruption of fires, or even the very threat of fires, can be enough to erode an organization’s faith in the DevSecOps transformation process. So it’s absolutely crucial to have the entire organization on board and committed for the long haul. Keeping expectations managed from the start of the project is paramount. And, when small fires do break out, squeamish stakeholders might need to be reminded, “Don’t worry, we did warn you this would happen, it’s all part of the process.”
Buses
Skilled DevSecOps experts are in high demand, creating a shortage of qualified talent. Unfortunately, this has resulted in DevSecOps teams sometimes having to be “protected” from the rest of the engineering organization and, worse, siloed. What should have been a highly-integrated Agile team approach with shared ownership just got thrown out of the window, or more accurately, over the fence. This sounds awfully like the saga of Software Engineering vs. IT Infrastructure.
With knowledgeable personnel becoming increasingly hard to come by, companies on a DevSecOps quest are forced to give more serious consideration to the bus factor. Leadership and other stakeholders find themselves in the stressful situation of needing to contemplate and address complex scenarios:
- “If our one engineer who knows about this stuff ever leaves us, we’re screwed.”
- “Sorry. Joanna handles that, and she is currently on holiday. You’ll have to wait until she returns.”
- “It’s not documented or automated, so I guess you’ll have to go through the code base and figure it out yourself.”
What do you do?
Embarking on a DevSecOps journey can be daunting. But, as with all grand adventures, proper planning is essential to success. And for those who already set out on this journey with grand ambitions, but lost a key DevSecOps engineer and are stalled by the bus factor, you can take steps to get back on track.
Before you begin, step back and take time to analyze the big picture. Identify bottlenecks, determine what’s causing them, and assess their potential for automation. Don’t forget the organizational assessment - do we have the collective mindset of DevSecOps? Measure your risk at each point and analyze personnel dependencies that might exist anywhere along the critical path. Goldratt’s Theory of Constraints is apt here, which basically argues that continuously identifying and neutralizing your biggest constraint (e.g., your busiest DevOps engineer) is the most important thing to focus on. Ensure your relevant teams are properly integrated and prioritize cross-training and knowledge sharing. This isn’t only essential for DevSecOps to work as intended; it also helps build trust among the team. There’s exceptional value in building processes and culture that encourage the DevSecOps engineers to spend more of their time on knowledge transfer to other engineers (e.g., through pairing). It might seem counterintuitive, especially with a DevSecOps engineer who is already too busy, but knowledge transfer is the most effective way to free up their time in the long run.
Definitely don’t try to transform the entire organization overnight. Start small, pick a single value stream where the risks are small and the payoff is high. Greenfield projects are a great place to start. Try to demonstrate tangible value quickly to keep everyone on board. Then move onto the next value stream, and repeat as necessary. You’ll learn and improve during each of these mini-transformations, and you’ll build up a catalog of failed experiments and successful transformations you can use as a reference, so it should get easier over time.
Nobody said the DevSecOps voyage was easy. But it doesn’t have to be harrowing either, nor does it have to be taken alone. Getting a second opinion on your charted course is always recommended. This objective health check can identify potential issues missed by your team and leave you feeling more confident in your approach. And, of course, enlisting the help of knowledgeable, experienced guides is a surefire way to navigate the uncharted DevSecOps waters confidently and help you reach your destination safely and successfully.