The Deadly Mistake Every Fast-Growing Startup Makes (You're About to Make Too)
Premature optimization of an organization is the root of all evil
👋🏻 Welcome to Lead and Scale, the newsletter for CTOs and leaders navigating the complexities of scale-ups.
If you’re not a subscriber yet, here’s what you missed this month:
From pre-seed to IPO, which type of CTO does your company need?
Why CTOs should add emotional intelligence to their skill set
There’s a mistake (almost) every fast-growing tech company does.
They hire a DevOps engineer, tick the DevOps checkbox in the perfect startup playbook, then move on to something else.
Let me tell you the full story.
As the company transitions from startup to scale-up, the CTO progreesivley steps back from running the company’s infrastructure to focus on delivering growth. They spend more time with customers, hiring, building their engineering culture, and need someone to build the infrastructure.
The DevOps engineer joins the company, with the promise of a “DevOps paradise”, where the developers own what they build and ship until their code reaches EOL.
They build a state of the art CI/CD pipeline. Most of the code is untested, and everybody patches in production anyway.
They move everything from a VPS to a fully auto-scaling Kubernetes cluster. The cloud bills skyrockets, as it’s easier to throw hardware at performance problems than solving your technical debt.
They get isolated, and slowly build their own silo, separated from development and operations. This happens both by acting as a department in itself, and by building a tooling only them know how to use, which is the opposite of the DevOps philosophy.
Finally, they get bored, or burn out. They realize their DevOps paradise only existed in their job description and all they do is investigating bugs they didn’t create.
Sounds familiar? Let’s make a postmortem of this.
What happened?
What did these CTOs do wrong?
They're guilty of premature optimization.
Premature optimization is the root of all evil. — Donald Knuth, 1974
These CTOs tried to implement a model that only works in structured companies. Their organizations weren't ready, lacking the necessary maturity while growing super fast. They also underestimated the complexity of building a modern, reliable infrastructure that scales.
Why Did it Happen?
Most CTOs I’ve met have faced the same issues. They wanted to apply a model called “you own what you ship”, but their engineering team was not ready. Their developers wrote the code, pushed it and… moved to something else.
What does “owning what you ship” means?
Understanding the business: You can challenge the tickets created by the product when you feel something’s off and they missed it.
Writing the code.
Testing the code, whether with unit tests, functional tests, or end to end tests: Too many startups start doing it too late so they only focus on shipping, until it hurts too much to be ignored. This is what you call “strategic technical debt”
Deploying the code.
Monitoring the code, for regressions, but also during business as usual: I consider monitoring the trickiest part, as it requires lots of time and experience to understand what you want to measure first.
Debugging what’s going wrong: preferably in the middle of the night, with a quick fix in production the next deployment will delete.
Setting up “you own what you ship” in a company requires time, energy, and experience, which most fast growing companies don't have (yet). Well structured companies with hundreds of engineers, or scale-ups with a majority of seniors can implement it.
This is the most common root cause: trying to implement models that simply don’t work at your company’s stage. What works at Amazon, Google of Facebook doesn’t work in a small or medium business.
How Could They Avoid It?
Scale-ups are 100% focused on delivering growth and keeping their edge as new competitors enter the market. This makes it impossible to take a step back to look around and understand what's happening at the broader level.
One cannot stand at the window to watch oneself pass in the street — not Spinoza, but could have been.
There are multiple ways to avoid falling into the DevOps premature optimization trap, depending on your company's budget, engineering skills, and product technical complexity.
Option 1: Experienced Contractors
The easiest solution is to work with a contractor.
Contractors bring their experience to your business, working with your team to define what you need to implement, measure, and how to make it compatible with your growth trajectory. By properly onboarding your engineers, they will avoid creating single points of failure and foster group dynamics. The financial cost can be high, but the short, medium, and long-term impacts are substantial.
Option 2: Developer-Led Infrastructure
The solution with the highest learning curve is to hand over your infrastructure to your developers and let them learn the hard way while you take the time to hire an SRE team.
Developers are eager to learn new technical skills, and most engineers I work with want to give DevOps a try—at least the fun parts. The financial cost is lower, though it does impact delivery timelines.
Option 3: DevOps-as-a-Service
Finally, when the product's technical structure isn't too complex, you can use a DaaS platform.
DaaS (DevOps as a Service) provides a set of online tools that allows companies to delay hiring a team of DevOps engineers. Most cloud providers offer such platforms, which can handle the job with a low learning curve, though they do create vendor lock-in.
Finding Your Balance
There's no silver bullet when it comes to implementing a complex, new technology stack while sustaining hyper-growth. Being a systems engineer, my favorite solution is to create a solid, cross-functional team of DevOps and SREs to handle the infrastructure while the rest of the team focuses on delivery. I know it's far from the "DevOps way," but in a company that needs to scale fast, it gets the job done.
Thank you for reading. See you next Tuesday to talk about leadership mistakes to avoid in a scale-up environment.
I feel like we've all seen so many "DevOps engineers" that we've kind of forgotten that DevOps is not a job title but a philosophy (which you also mentioned in the article). Often I see that behind these titles are just infrastructure people who are siloed in their own little world and are not always helpful for building actual DevOps culture. Don't get me wrong- I love infrastructure people and I think they're essential, but DevOps should be a collaboration between the dev team who owns what they ship and the infra engineers that work to enable the dev team to do it. So often it doesn't really look that way...
Amazing article. This is so true A-Z