DevOps is the New Waste in 2023

When I first heard about DevOps around 12 years ago I was working for a company called Rally Software. When I asked one of my co-workers what DevOps meant he said “It’s when developers have root access in production”. My immediate though was “The horror!!”.

For the last 12 years I’ve studyed the writings and talks of DevOps champions like Gene Kim, John Allspaw, and Jesse Robbins (from OpsCode). As part of my DevOps journey I’ve also lead my fair share of large scale DevOps transformations for a wide range of Fortune 500 companies from health care to insurance to telecommunications companies.

Before I get into why DevOps Is the New Waste in 2023 I want to make sure we have a shared understanding of what DevOps means. While many people have written entire books on the topic I will boil it down to waste elimination. Waste elimination so we can deliver better software faster with more reliability and less downtime. DevOps (and Agile) both spring from LEAN. LEAN tells us there are 8 wastes which can easily be memorized from the anacronym DOWNTIME: Defects, Overproduction, Waiting, Non-Utilized Talent, Transportation, Inventory, Motion and Extra-Processing.

I remember when I lead my first cloud transformation project when I was the manager of Cloud Systems Development back at Pearson. It took upwards of a year to procure and deploy new servers in the datacenter. In one case, a team of engineers needed a hadoop cluster and physical servers sat in the datacenter for 18 months waiting to be racked and stacked.

Our CTO asked me to help lead the charge into AWS and so it began. My team and I set out to build an entire cloud provisioning platform called Nibiru (this is in the days before tools like terraform even existed). It revolutionized the way Pearson deployed applications and services. We took a process that could take 12 – 18 months and turned it into minutes. We were able to eliminate a significant amount of waste and teams were deploying software 100s of times a week. The results were obvious. DevOps was a success.

Let’s fast forward a bit from 2010 to 2023. Almost every company realizes that traditional datacenters are slow and cumbersome. At minimum companies are implementing their own internal private cloud ecosystems and have made some investment into public cloud platforms like AWS, Azure or GCP and every engineering teams is trying to “do DevOps”.

If you have been paying attention an interesting trend has emerged. In 2022 when Google DORA published the 2022 Accelerate State of DevOps Report they identified 0% elite performing teams/organizations, a 4 year low in high performers and a massive uptick in medium performing teams/organizations.

DORA proposes a hypothesis that COVID lead to a reduction in information and knowledge sharing. I propose a different hypothesis. My hypothesis is that DevOps is the New Waste.

What do I mean by that? Well if we think back to the John Allspaw and Adrian Cockcroft debate around NoOps we hear Allspaw debate Crockcoft’s NoOps concept by saying if you are doing any of these things (backups, monitoring, patching, etc..) you are still doing “Ops”. What Crockcoft was saying with his NoOps theory is that they built a platform that enabled engineering teams to focus on feature development not backups, monitoring, infrastructure automation, etc.. the platform provided these services to the engineering teams for free. This is back when Netflix was using Asgard mind you.

Today we have all decided DevOps is king, that all companies must adopt DevOps, and that everyone must do DevOps. What we created is more waste, specifically overproduction and extra-processing. How many lambda terraform modules does a company need? How many CloudFormation templates does a Fortune 500 need to deploy an Aurora RDS Cluster? How many CI/CD pipelines do we need to invent to deploy containers to a Kubernetes cluster?

What I’ve observed over the years as consultant and someone who has run architecture teams for large Fortune 500s is that more and more we see individual teams locally optimize (which is a DevOps anti-pattern) CI/CD for their application. Now every engineering team is spending sprints writing terraform, writing Jenkins pipelines, solving application security testing, and figuring out how to integrate with ServiceNow for change management. Imagine an organization with 1,000 or even 10,000 developers making up 100s or even thousands of development teams all locally optimizing CI/CD.

What Netflix did with Asgard, and dare I say what my team and I did with Nibiru, or Utopia (self-service platform my team and I built at Aetna), or SEED (self-service platform my team and I built at Comcast), is that we decided it was more efficient to solve problems like infrastructure automation, container deployment, blue/green deployments, monitoring integration, centralized logging, SAST and DAST integration, etc.. into a single platform so engineering teams could focus on feature delivery not infrastructure automation and CI/CD.

Should everyone do DevOps yes, but not every engineer needs to be a terraform expert, not every developer needs to be a Kubernetes expert, not every engineering team needs to figure out how to create, update and close change tickets as part of their unique CI/CD workflow. Companies in 2023 and beyond need to go back to building Cloud Systems Development teams that build self-service platforms that enable engineering teams to go back to feature development vs duplicating DevOps 100x over again across the enterprise.