Despite the general claim, most financial services institutions still operate under siloed and fragmented models that are not delivering the value that was originally thought. This post explores some of the key issues faced by the industry in adoption of true DevOps.
Key takeaways
- Lack of management engagement and investment in transition and cultural transformation places the biggest hurdle in DevOps adoption
Recommended actions
- Opportunities and benefits of a true DevOps model can’t be ignored
- Banks need to get past the perceived obstacles through education and internal cultural change
- Real obstacles need to be managed using a holistic approach to transformation with a DevOps policy and governance framework
Overview
Gartner predicts that 75 percent of DevOps initiatives will not realize full business value by 2022.† The number is likely to be higher for the financial services industry (FSI).
The term DevOps was coined by fusing the words, and by extension the roles, Development and Operations. It was meant to bring both teams together and automate processes/practices in the software development life cycle (SDLC) with the key objective of fast and more reliable build and deployment cycles.
FSIs have been on the DevOps journey for several years now, with mixed results. At the higher end of success, firms were able to reduce operational cost and speed up delivery of their software, resulting in launch of new and innovative services/products to the market. At the lower end of success, nothing much has changed apart from names of the teams and a few internal processes. Most financial services operate within this band.
So, what makes adoption of DevOps so difficult in FSIs and why has the full potential of DevOps not been realized in most firms?
Why bother with DevOps?
With the threat of competition from tech-savvy neo-digital banks and Fintech companies becoming real, Financial services firms have little choice but to continue their innovation and transformation journey.
DevOps is an absolute key enabler for innovation and digital transformation to compete with Fintechs on features, cost, and time to market. The move to cloud-native agile infrastructure and methodologies are complemented by adoption of DevOps. In fact, one is not possible without the other.
The single biggest benefit of DevOps is reliable release cycles which results in fewer bugs post production and enables faster deployment cycles. The automation, which is a key element of DevOps, results in the elimination of manual deployment tasks traditionally done by operations staff. Furthermore, issues encountered due to inconsistent environment between production, SIT, UAT, and pre-production - including configurations -, can be eliminated.
In short, DevOps practices turn the entire development cycle into a well-oiled, fine-tuned machinery which provides reliability and stability of production infrastructure. The end result is not only rapid time to market but also enhanced production and code quality.
Why it does not work
The Culture
Banks have traditionally been divided into Run the Bank (RTB) and Change the Bank (CTB), and financial institutions still categorize and prioritize their budget, resources, and programmes of work within these categories. Unfortunately, these categorizations also promote the creation of teams and subcultures that separate the core development from operations, which is prohibitive to adoption of a DevOps.
DevOps, by definition, is primarily a culture and mindset change – and a big one at that. Many financial services firms have failed to realize that it needs to be implemented and managed as an enterprise-wide change which requires long-term planning and commitment from the top. Simply merging both teams together, naming it DevOps, and expecting a traditional approach to change to an agile one overnight without fundamental changes and groundwork taking place is a proven recipe for disaster.
Accountability
Without enough investment in the evolution of the culture, the subcultures of the Development and Operations teams continue to exist. With that, the sense of accountability gets diluted, creating additional bottlenecks rather than increasing deployment velocity. At the heart of DevOps, developers are accountable, but then what happens to the responsibilities and needs of the highly skilled operational staff whose heart is not in development?
Overalerting and skill set
Operations staff provides a value to the risk-averse financial industry. Sound operational practices, deployment checks, and incident management processes are highly sought after in the industry. Without a proper transition framework for DevOps adoption, the subtle elements which make it successful get overlooked. Development staff suddenly feel overburdened by new operational responsibilities, while Operations staff feels at a loss.
I have often heard that the best way to implement DevOps is by making developers wake up at night for production alerts. The logic being that if they feel the pain, they will fix the core issue and eventually write better software. What a great idea to encourage your key employees on a major transformation! And is that a surprise when it doesn’t work and demotivates your best technical resources?
A well-defined framework will provide a clearer path of transition so that both teams not only gravitate towards a common goal but can also leverage their core strengths. For example, developers will benefit from the skill set of good operational staff, which focuses on effective checks and balances to ensure that production continues to be error-free. At the same time, operational staff will benefit from core development skill set and techniques for continuous integration.
It’s only when both teams are engaged and aligned that the true fusion and alignment of both teams takes place. A common issue seen is with DevOps – developers are made directly accountable to production issues, and as a result they code for more checks and alerts, which results in false positives and alert fatigue. This is where core Operational skill set will be extremely helpful and can segregate meaningful issues from the noise.
Lack of investment in tooling
The change to DevOps doesn’t come overnight, the goal of deployment automation needs proper investment in tooling and identification/removal of bottlenecks. It requires executive level support and engagement to keep the progress on track.
This is where the process often breaks down for most firms, where big announcement on "we are moving to DevOps" is followed by immediate expectation of the benefits without continuous engagement/investment in removal of key issues and bottlenecks.
Lack of planning and engagement
It’s essential to bring in the right level of DevOps expertise in the organization, and invest time and effort in planning its adoption, including investment in tools/training, executive support, and decision-making framework/support. Not investing enough time and effort upfront in planning and engagement inevitably leads to issues and fragmented implementation/adoption of DevOps. Pockets of DevOps in individual teams has some merit, but falls apart when interactions with other teams happen.
Setting forums to bring all teams together, including the Server Team, Testing Team, Dev Team, Ops Team, and Database Team is essential for collaboration. This ensures everyone is playing a part and has a piece of work to do. Interactions and dependencies with other teams work better when people are putting their hand up for a piece of work. A significant amount of time often gets lost debating whose responsibility it is, rather than focusing on how to get it resolved. Everyone needs to be engaged and feel part of it – credit needs to be given to the entire team, rather than to individuals. That said, there are always going to be certain pieces of work where someone needs to make a call. However, with the right engagement and culture, these could be kept to absolute essential items only.
Another crucial part of the planning is to have engagement with other enterprise forums - e.g., Opensource, Cloud, Agile etc., and have a shared understanding on where the organization is headed. This doesn’t necessarily mean that one would need to consider everything from the enterprise perspective, but it helps you to arrive at the right decisions for the right reason. For example, if the enterprise-wide standard is to use a particular O/S or database and it is not feasible for the specific use case, then it’s easier to debate and decide on the merits/demerits of making an exception. A certain level of autonomy, therefore, is an essential part of this, as long as there is an overarching framework to ensure we don’t go too far and/or introduce new risks.
Conclusion
Steps required to overcome challenges in adoption of DevOps begin with understanding the status quo and identifying changes required in the culture and approach, followed by creation of a vision for future organization, identifying and gaining commitment from stakeholders, creating an open communication plan, providing proper training where needed, and ensuring constant engagement with the organization.
Overcoming the challenges of DevOps adoption will allow organizations to enable agile delivery, higher software quality, better cross-team collaboration and technical enablement. It therefore leads to organizational transformation. Practitioners operating in highly regulated businesses agree that the DevOps approach will become widely adopted in the near future, especially where propositions are based on ecosystems of services provided by various providers through standard APIs.
Changes to well-practiced operations model in a financial organization require diligence and care, as the stakes are high. A well-planned implementation of DevOps goes a long way in realizing the significant benefits it provides. While ongoing security issues and their remediation doesn’t get fully resolved by DevOps alone, there are ways for security practices to coexist with mature DevOps.
† Source: Gartner "The Secret to DevOps Success"
