IT:AD:Documentation:SAD:RAW:Example:1:Appendices:DevOps

DevOps is an Enterprise reaction to the documented benefits of Agile delivery, extending it beyond just the development phase to the whole application lifecycle – into the organisation as a cultural change, Agile processes, backed by appropriate automation and communication tools.

As Agile developed as a refutation of the high cost of delivering value using a Waterfall based development process, DevOps rose as a refutation of the high cost of delivering value using ITIL, the “Waterfall based Operations process¹⁾.
Many Organisations have tried to update their SDLC, only to find little gain. Analyse by others indicates the agreed common cause for this failure to deliver on expectations is a lack of a continuous ongoing ALM process that incorporates Continuous Testing.

Traditional Software Development Life Cycle (SDLC) management is commonly limited to the phases of software development including requirements, design, coding, testing, configuration, project management, and change management. DevOps ALM covers a broader scope, and continues after development until the application is no longer used, and may span many SDLCs.

In a 2004 survey designed by Noel Bruton (author of “How to Manage the IT Helpdesk” and “Managing the IT Services Process”), 77% of survey respondents either agreed or strongly agreed that “ITIL does not have all the answers”.

Criticisms of ITIL²⁾ include the following: because of its focus on service management, ITIL does not feed back effectively into the design process. Nor does ITIL directly address the business applications which run on the IT infrastructure; nor does it facilitate a more collaborative working relationship between development and operations teams.

Several different attempts to move away from ITIL and other cumbersome frameworks. Beyond DevOps, the most well-known is the Scaled Agile Framework (SAFe).

Although criticized by by world-class Agile specialists³⁾⁴⁾) for being too cautious it is important to note that both critics and supporters of SAFe agree it yields widespread benefits: although SAFe may be a less effective implementation of Agile, it is a safe starting point for slow to change, large organizations to implement, and enjoy some of the benefits of, Agile.

Although SAFe gained initial attention, the current market is currently strongly backing moving straight to DevOps.

A 2015 survey by CA Technologies⁵⁾ shows that 88% of more than 1,400 IT or line-of-business executives have already adopted or plan to adopt DevOps within the next five years. This is up from about 66% in a similar survey taken in 2014.

Based on several factors – including its proven ability to lower costs and deliver better value while not sacrificing quality – Organisations continue to follow the upward trend of Agile awareness, actively move away from ITIL processes towards DevOps processes:

• 49% of organisations complain that still largely manual testing phases are a bottleneck to speeding up development cycle times⁶⁾.
• DevOps delivers 18% faster time-to-market⁷⁾ * DevOps delivers 19% better app quality and performance⁸⁾.
• 88% of enterprises already have or have plans to adopt DevOps within the next 4 years⁹⁾.
• 63% of over 4000 respondents to the 2014 Puppet Labs and IT Revolution Press¹⁰⁾ survey are already implementing DevOps practices.

Those who had moved to DevOps reported: * 46% Increased software/service deployment frequency¹¹⁾ * 36% Improved application quality and performance¹²⁾ * 34% Reduced application time-to-market¹³⁾ * Up to 40% increase in productivity¹⁴⁾ * Up to 77% faster mean-time-to-recover (MTTR)¹⁵⁾ * Up to 300% increase in the number of weekly deployments¹⁶⁾ * Up to 200% increase in the number of deployed environments¹⁷⁾ * Up to 15 times reduction the manual effort required for release¹⁸⁾ * Up to 9X increase in release volume without adding resources¹⁹⁾ * Up to 85% reduction in transaction response time²⁰⁾ * Up to 5X improvement in testing efficiency, testing times reduced from days to minutes²¹⁾ * 76% reduction in resolution time and prevented 18 outages impact user experience²²⁾ * Gartner Says that By 2016, DevOps Will Evolve From a Niche to a Mainstream Strategy Employed by 25% of Global 2000 Organizations²³⁾.

Stakeholder drivers include: * Time to Market was ranked as a very important part of their corporate strategy by 61% of organisations. * Corporate Image is #1 executive concern when it comes to quality, demanding protection from negative press. * Customer Experience – fit for purpose, availability, ease of use and performance – was determined a key objective.

Other drivers to the rate of current fast adoption are: * Agile processes – many projects have been delivered using Agile processes, therefore more are aware of their concrete benefits. * Cloud infrastructure: inexpensive, easy to manage virtual infrastructure is widely available. * Infrastructure as Code: cloud services made widely available and comprehended the process of remotely defining infrastructure by script and automation²⁴⁾. * Automation: both automation of cloud service infrastructure provisioning and automation in other areas – eg: data centers – is gaining wide recognition. * Continuous tested delivery: continuous delivery pipelines are have gained awareness and acceptance. * Best practices: a critical mass of publicly available best practices is available to remove adoption risk.

The Cultural changes have been summarized as being around: * Amplify Feedback Loops: emphasis communication and feedback in order for all involved to understand the desires of all other stakeholders. * Think of the whole System: understand the feedback from the whole pipeline, starting from the business, as opposed to the performance of a specific or single department or individual. * Empower a Culture of Continual Experimentation and Learning: promote improvement investigation in order to master doing it safely.

The above are important cultural changes. But there are other changes as well.

A key cultural change under DevOps is changing the mindset of organisation groups from blocking verifiers to trusted advisors and enablers.

Due to ongoing increased availability of cloud services, along with the simplification “for the masses” of their management.

Developers have now expected to take advantage of these services and their simple management tools in order to define and manage a project's basic environment provisioning and deployment requirements using Infrastructure as Code, Testing as Code and Deployment as Code patterns.

These coded requirements are then automated rather than executed laboriously by hand.

It is important to understand that DevOps does not mean Developers have free reign to do as they please and that other roles no longer have input. This is certainly not the case. The role of Developers is to develop in response to User Stories that capture the requirements of Stakeholders – including Infrastructure Support Services, Application Support Services and Customer Support Services. All of these stakeholders are empowered to add User Stories to a Project's Agile work item Backlog that must be prioritized and addressed. These User Stories in turn define how Developers must update the Infrastructure as Code and Configuration as Code definitions to meet expectations of the various Stakeholders.

A key cultural change is the checks and balances being updated from a manual process to an automated process. Instead of using positions of authority to verify and potentially block deployment via change control processes, stakeholder become empowered to actively engaged in submitting Acceptance Tests which the automated *Build Service enforces on all stakeholders behalf*.

The perception that DevOps's emphasis on automation will replace ITIL is unfounded within this Organisation. Existing legacy apps that were not developed from the start to be managed by DevOps processes cannot be successfully and economically managed using DevOps processes. Existing roles will continue to be needed for years to come for these applications.

DevOps processes must instead be used in parallel, by the same Resources, but reserved for new projects.

At the heart of DevOps is the adoption of Agile methodologies to break down barriers between people and groups using common communication and work item management tools.

In Scrum Agile, the primary tools are an Work Item Management Service and electronic Kanban board appropriately accessible by all stakeholders.

Mature Organisation's choose certified SaaS based ALM Services that include Work Item Management Services.

In today's world of rapid development cycles developers are expected to ship code very frequently. Increasingly rapid release cycles mean customer needs are expected to be met earlier, by developers shipping code frequently.

On the other hand operations are still expected to ensure no customer is adversely affected by this cycle. Change is their enemy. Where Devs meet Ops there can often be significant tensions.

To alleviate these tensions the DevOps movement has focused on automating as many build/store/test/deploy tasks as possible.

Mature Organisation's choose certified SaaS based ALM Services that include tools and automation where possible of the following services:

• Coding: Code development and review, version control tools, tested code integration
• Building: Automated Build Services and tools
• Testing: Automated testing of qualities and functionality using Testing as Code tools
• Packaging: Artifact packaging and pre-deployment deployment
• Releasing: Assurance, release approvals, release automation
• Provisioning: Infrastructure provisioning and management using Infrastructure as Code tools
• Configuration: Infrastructure configuration and management using Configuration as Code tools
• Monitoring: Continuous applications application monitoring

The benefits of rapid iterative Agile deployments are not able to be delivered when testing still relies on manual processes.

Simply put, an Organisation cannot embrace and reap the value of DevOps if it does not commit to ensuring Acceptance Tests are defined by Testers, converted into Testing as Code by Developers, to be enforced by the automated Build Service.

Finally, organisations struggle with the question as to how to upskill manual testers to become automated testers. The answer is to not, and adhere to the architectural principle of Separations of Concerns.

An important reason it has been proven to be beneficial to keep the automation of tests separate from test definition is Testers tend to try to automate what they already know – manual testing – when manual testing should be seen as only having ever been required because there was no means to automate testing. The focus should be on automated testing, not automated manual testing.

The fresh break allows Testers to stay focused on what they know best – scripting acceptance test definitions – and developers focusing on what they know best – automation of any kind.

The Theory of Constraints identifies the single constraint to DevOps adoption is the inherent aversion to change from departments within the organisation.

Hence guidance on how to implement DevOps in traditional (eg: ITIL based) Organisations has been given by several reputable sources, including Microsoft. For example Gene Kim’s “Three Ways Principles” essentially establishes different ways of incremental DevOps adoption, to minimize risk and cost whilst building the necessary in-house skillset and momentum needed to have widespread successful implementation.

An implementation process can be developed across the whole organisation, per project, or a combination of both.

The benefit of doing it per Project, allows each project to perform a complete migration to DevOps, from top to bottom, taking on board the responsibility of solving the globally identified traditional bottlenecks (eg: converting Manual Testing to Automated Testing for their project only), without disrupting other projects still running using traditional processes.

The recommendation for this organisation is to continue to do the transition the later way.

John Willis and Damon Edwards (and later Jez Humble) coined the acronym “CALMS” to describe key aspects of DevOps²⁵⁾:

• Culture: “Culture eats strategy for breakfast.”(src: Peter Drucker).
• Automation: Automating repetitive, time-consuming, error-prone tasks yield big dividends.
• Lean: apply value-stream mapping, and plan to remove inefficiencies.
• Measurement: you can't improve what you don't measure.
• Sharing: friction-free information improves organizational performance.

It’s a misconception that DevOps is Developers coming to wipe out Operations and do it themselves. The first and most obvious reasons this is a misconception is that Systems are written for the environment and processes they were intended to be run on. Organisations have legacy applications that were intended to be deployed and tested manually – they simply cannot be cost-effectively ported to an automated tested deployment process.

The second reason is DevOps – and its antecedents in Agile operations – are being initiated out of Operations teams more often than not²⁶⁾. This is because Operations have realized that practices need to be automated to keep pace with what is being expected from business stakeholders. The result has not been automating personnel out of a job, but instead – as lower level concerns become more automated – technically skilled staff start solving higher value problems.

The following sources provided facts for the above: * “DevOps with Quality” by GapGemini/Sogeti * https://en.wikipedia.org/wiki/Continuous_testing * History of DevOps * What is DevOps