AgileOps — A perspective from the [Dev]Ops and SRE land

In a world of chaos, simplicity is truly beautiful. I often use this phrase. However, when considering Agile practices in daily work life, the phrase can be adapted to: “In a world of chaos and varied priorities, defining what truly matters is beautiful.”

☝This post is from my Medium Account which you can follow here https://devopsvision.medium.com/

DevOps applies Agile ideas, values, and approaches to packaging and deployment, change control and release management, service delivery, production support and maintenance, security, and compliance. It also takes Agile practices like continuous integration, test automation test-driven development, refactoring, pairing, and peer reviews, and applies them to infrastructure provisioning and configuration management in code.

Personal Insights

Before diving into the main content of this post, I’d like to share some thoughts on companies that haven’t adopted any agile methodologies for their operations. Despite a common belief that most companies have already embraced these methodologies, that’s simply not the case. This is true not only for small businesses but also for medium and large enterprises, especially within the government sector. It’s still common to see Development and Operations teams working without the principles that could provide clear visibility into what is important, what is pending, what has been completed, and what should be prioritized for the next cycle. Without proper frameworks in place to guide these areas, the situation can become chaotic in the long term, with problems escalating over time.

The Ups and Downs of Agile Lessons from Brazil

Drawing from my experiences at some of Brazil’s most prominent public companies, I observed that agile methodologies, when they did exist, were primarily confined to the development teams, but not nowadays anymore, also that is not real for all the public sector, and companies in Brazil.

The Operations teams frequently operated without regard to any specific methodology, reminiscent of earlier times devoid of structured frameworks. Despite the existence of principles aimed at identifying and prioritizing tasks, there were misconceptions and misunderstandings about DevOps. Disrupting misconceptions of how to deal with DevOps demands and tasks. DevOps is recognized as a valuable and effective approach to software development and operations. When properly aligned with the best practices and frameworks, significant benefits can be realized, including faster software delivery, increased efficiency, productivity, and improved quality.

The positive experiences I’ve had working with a team in Brazil that was deeply engaged with practices and methodologies, together, we achieved outstanding deliveries. The team was happier, with priorities clearly defined, avoiding the overwhelm of immediate, pressing tasks. This approach created an environment of success, in a very opposite direction when there is no method of work and clear goals.

Agile cannot run in isolation, it needs to be scaled. Applications need infrastructure to run, and that’s where infrastructure and teams also need to be encouraged to adopt and practice agile. The development and operations teams together can accelerate the path to value creation and cost savings as product releases are managed frequently.

Before dive in

When we talk about “Agile” in DevOps, we’re not talking about trying to use Scrum or Kanban methods in operations and support teams. Nor are we talking about embedding operations staff in Agile project teams and relabeling them as “DevOps.”

What Does Agile Mean in DevOps?

Agile in the context of DevOps is about extending Agile ideas and values from product management and software development to the rest of the service delivery organization. It’s about following Agile and Lean approaches across the full life cycle of a system, from planning and design to release, and to ongoing operations and maintenance and support. It’s about breaking work down so that it can be delivered incrementally and iteratively by small, collaborative, and cross-functional teams; moving decision-making down so that these teams can respond more effectively to new information; continuously learning, adapting, and improving.

Agile methods like Scrum have helped many organizations deliver better software, faster and cheaper. But Agile ends when the software development project is done, whereas DevOps teams design, deliver, and run online services: they own everything from the application to the runtime and support environment. As they say at Amazon, “you build it, you run it”.

DevOps and the need for speed

Speed plays a central role in DevOps culture and philosophy. The Agile Manifesto, DevOps’ founding document states:

We follow these principles:

Our highest priority is to satisfy the customer
through early and continuous delivery
of valuable software.

Welcome changing requirements, even late in
development. Agile processes harness change for
the customer’s competitive advantage.

Deliver working software frequently, from a
couple of weeks to a couple of months, with a
preference to the shorter timescale.

According to this statement, one of DevOps founding principles is to create valuable software quickly, continuously adjusting and improving as they go along. DevOps engineers often measure themselves on how quickly they can release new features and how many updates are made in a certain period of time.

This does not mean, of course, that quality is compromised. The commonly used DevOps best practice of CI/CD (continuous integration and continuous deployment) ensures that changes in code are delivered reliably, on a regular basis, and at the highest quality.

From the Agile Manifesto to CAMS and CALMS

In place of an Agile manifesto, DevOps has CAMS, an acronym that John Willis and Damon Edwards came up with to summarize what they thought was important in DevOps: Culture, Automation, Measurement, and Sharing.

Culture

Although a lot of talk about DevOps focuses on technology and technical practices, cloud platforms, containers, configuration management in code, and continuous delivery, DevOps, like Agile, is about people first. People working together, sharing common values goals, and priorities, building a common culture across service development and operations and security. Creating an environment in which people are safe to make mistakes as long as they learn from them. And effecting the necessary cultural and organizational changes to make all of this possible.

Automation

DevOps relies heavily on automation to scale out, accelerate delivery, and maintain control at speed. Teams take extensive advantage of tooling to simplify and standardize work, and to eliminate manual steps and handoffs, increasing consistency, predictability, and transparency. Automation paves the road ahead for developers by defining safe defaults in templates, images, and recipes, and providing self-service capabilities for developers to provision their own run-times and push changes out when they need to.

Everything is expressed in code: tests, configuration, security and compliance policies, provisioning and deployment steps, and even the pipelines that are used to manage code changes.

Measurement

DevOps teams are metrics-obsessed, in the same way that high-performing Agile teams are test-obsessed: they instrument code, pipelines, and runtimes to drive empirical feedback loops.

Measurements such as dwell time, response time, click-through rates, uptake, and conversion rates are tracked in A/B experiments to shape design decisions based on how users use the system. Monitoring becomes a critical part of testing.

Cycle time and lead time are continuously measured to optimize delivery workflows. Release frequency, incident rates, mean time to detect, and mean time to recover from incidents are used to answer important questions: “Are we going fast enough,” or “Are we going too fast?”

Sharing

In DevOps, sharing happens at multiple levels:

Collaboration across silos, defining shared goals and shared priorities, shared responsibilities, and a shared sense of ownership
Sharing tools, practices, and methods across development and operation
Creating information feedback loops from operations to development and development to operations, learning and improving together to build better teams and better systems

Lean

Later, Jezz Humble added “L” for “Lean” and CAMS became CALMS. Lean tells teams to map out value streams, look for ways to manage work efficiently, and use automation and measurement to optimize flow. It tells teams to eliminate friction and bottlenecks, to use the information to improve visibility and to learn and improve.

From “Working Software” to Services in Production

Agile teams commit to building “working software.” Team members agree on a definition of done: a contract that defines the conditions of satisfaction that need to be met for their work to be accepted.

DevOps teams are responsible for building, delivering, and running services. For DevOps teams, their work isn’t “done” until it is running in production and they have received feedback that it is working correctly. This means that when work is “done,” the software, and the team, both must be production-ready.

This requires more work throughout design and delivery: understanding operational dependencies, dealing with security risks and compliance constraints, making sure that the system is configured correctly, coordinating how changes are rolled out, and preparing to deal with incidents in production when something goes wrong.

Agile Principles

IT teams have been adopting agile methods that encompass the 12 principles shown below.

You can understand more details about the 12 agile principles in the following article: "How the 12 Principles in the Agile Manifesto Work in real life"

The journey of continuity

The journey of continuity is an important need for making DevOps successful. The principles of DevOps bridge the gaps between siloed teams and leverage automation for optimizing the processes that connect the entire lifecycle of the product. Each phase in the lifecycle is driven by its own set of tools, processes, and teams. While CI and CD paved the way for integration in the application world, the elements in the infrastructure world were still running in silos. A DevOps model integrates each phase and each tool streamlines the processes, and focuses on a common vision — “We build it, we run it.” Adopting new tools, moving to the cloud, leveraging APIs, etc., have enabled teams to get connected quickly and share common processes and workflows. And the core principle lies in the belief that this is a continuous journey of improvement.

source Hands-On Guide to AgileOps: A Guide to Implementing Agile, DevOps, and SRE for Cloud Operations

DevOps in the Product Lifecycle

The Agile practices in the business planning step establish business goals and adjust them based on customer feedback, which helps improve agility and business outcomes. This helps in gaining the trust and confidence of the customers on the value being delivered through the deliverables. This first phase in the product lifecycle is a stepping-stone in building the product journey correctly. With the right adoption of tools and techniques, this phase becomes essential and integrated with other phases. Conducting the right ceremonies, estimating and planning for delivering faster, and embracing the culture of change within the team all play a vital role. This phase is generally managed through tools such as Atlassian Jira, Azure Boards, VersionOne, LeanKit, etc. These tools are extended with other lifecycle tools to get end-to-end traceability.
Continuous integration is a practice in which software developers frequently integrate their code in the codebase with the code of the application where other team members also add their code. This helps in the early detection of integration defects while building the code, which if caught at a later stage will prove to be expensive to remediate. Building a CI pipeline is a common practice that is running across many organizations today and is a de facto standard too. While this pipeline was implemented in the application space, today it is applicable for building infra pipelines too. Tools like Github Actions, GitLab, Jenkins, TravisCI, CircleCI, TeamCity, etc., are all well-known tools that orchestrate the key steps, from building the code to delivering the executables/binaries. A complete CI pipeline comprises source code management, build execution, unit testing, code coverage, and artifact/binary deployment.
The environment build focuses on instant infrastructure provisioning by adopting runbook automation, configuration management tools, and self-service catalogs. Automation in this space helps to reduce IT sprawl. Building an environment comprises various activities such as provisioning and configuring infrastructure, preparing runbook automation, performing and automating security scans, and integrating infra pipelines with ITSM tools.
The continuous delivery practice focuses on releasing the product across different environments. A well-defined release and deployment process ensures the timely delivery of a quality product. Every stage in deployment passes through a series of quality checks. After its success, it is then deployed in the target environment. For example, if the CI build is successful, the CD pipeline picks up binaries from the development environment and deploys them into the test/QA environment. If the CI pipeline is not successful, the deployment will not proceed. Similarly, if the artifacts have to be transferred from the test/QA environment to preproduction, Artifacts will require more quality gates such as percent coverage that is configured and accepted, security score, etc. Tools such as Jenkins, Azure DevOps, GitHub Actions & Runners, Atlassian Bamboo, etc., are good examples that perform continuous deployment operations.
Continuous testing is a practice that means testing earlier and continuously to detect defects early in the lifecycle, which will result in reduced costs. This helps in establishing continuous feedback on the quality of the product. Continuous testing can be achieved by having your test cases automated and executed with each code integration and build process. The testing scope has expanded today, and it is not restricted just to functional, performance, and security testing. As applications are moving toward microservices-based architecture, addressing the growing market needs of the customer base, the need for extended testing has become essential including API testing, accessibility testing, resiliency testing, etc. A few tools in this space are Selenium, Appium, JMeter, HCL One Test Suite, etc.
The continuous monitoring and feedback practice involves monitoring applications and infrastructure across all phases and also acknowledging feedback from customers. This will help to lay out actions to optimize and improve the application and thus enhance customer experience and value. Every incident or problem with the deployed application is closely monitored and addressed with agility. ITSM tools like ServiceNow, Remedy, etc., come to the rescue in this space.

So, we have agile principles and a DevOps-connected model that enables organizations to work closely and move faster.

When to Adopt Agile?

Agile is an approach that recommends, facilitates, and provides guidance around iterative delivery, which is possible with connected teams, open collaboration, end-to-end integration of product lifecycle phases, and continuous work on customer feedback. Adopting agile is helpful when teams need to deliver a minimum viable product (MVP) in short sprints where customer feedback is important to drive the next change or feature in the product. Thus, development teams gain more confidence and trust from the customers of their product. Rather than releasing once a month and pushing integration testing to the end, the focus is on releasing fast, getting feedback, and providing working software early on in the cycle.

Summary

Agile and DevOps complement each other, but they should not be considered replacements for each other. On one hand, if agile focuses on iterative development with continuous feedback principles, DevOps focuses on bringing teams together that collaborate and plan for a continuous journey of improvements. Organizations that practice agile easily transition and extend into a DevOps working model. IT teams that practice agile and DevOps reap long-term benefits such as the following:

Collaborative and self-organized teams
Embracing change through trust and transparency
Faster time to market with automation
Continuously improving with feedback loops
Simplified processes and integrated workflows
Lower costs
Higher customer satisfaction scores
Better business alignment

Blog MasterMind

Header Ads

AgileOps — A perspective from the [Dev]Ops and SRE land