Software quality is obviously getting worse. I'm not talking about LLM slop features nobody asked for. I'm not talking about services collapsing under unprecedented LLM-powered demand. Companies are obviously shipping user-visible bugs at an accelerating rate. And consequently, the software we are using is getting worse and worse.
As a quick aside, I'm experimenting with recording my blog posts for YouTube. So if you'd rather see that, here it is:
Back to our regulary scheduled blogging!
You can thank GitHub for this post. I recently commented on a deleted line in GitHub and hit submit. An error dialogue appeared saying that some block of client-side code couldn't find the line ID. GitHub has had almost two decades to perfect "comment on code." Yet it regressed. And hilariously, I just triggered a Google Docs copy/paste bug while typing this[0]. And heaven help any heavy Claude Desktop users. Everything's getting worse around us.
I blame three factors for this.
- As you write code faster, the acceptable error rate drops.
- Implementation time is becoming decoupled from competence.
- The value of implementation becomes so high that slack time will trend to zero
I have faith that we can overcome these as a discipline, but we can't do it by applying our old bag of tricks. Our old bag of tricks got us our old error rate. We need to evolve.
As you write code faster, the acceptable error rate drops[1]
I think this is obvious but it's worth stating. If you're shipping code faster, and you have a certain regression rate per change, then obviously you are shipping more regressions. It's just math.
So there are 2 pieces of this: are we shipping faster, and how do our users experience that?
Why does this matter? Let's say that you flip a switch and can immediately double your code production. Nothing else changes. Just twice as much code as before. There are a few consequences:
First, you ship 2x more user-visible regressions over any time period. This makes sense, right? Each change has about the same chance to introduce a regression as it did before. So on average, double the code means double the regressions.
The problem with user-visible regressions is that users encounter them. After you flip this "double code production" switch, the number of active issues in your product will trend towards 2x the baseline. If users are lucky, you notice the issues they encounter in some way, and you can go and fix the issue yourself with your magical new implementation rate. But often, users have to tell you that you fucked up. "This menu is broken in this configuration and I can't even click this option anymore." And that's slow. You need to aggregate the reports, try to reproduce, prioritize the fix, etc. And your best users, your power users? They're the poor saps that trigger all of your new bugs, over and over again.
I'm sure somebody wants to counterargue "LLMs can detect all of these kinds of issues and automatically fix them," and I don't want to hear it. Software is obviously degrading all around us! Whatever LLMs are currently doing isn't enough to resolve it. And if you have some magic technique that the rest of us aren't applying, please scream it from the rooftops. Even better, try to get it integrated into the official Claude Code harness the official way: by randomly Tweeting at members of the team on the off chance one of them notices.
So when you hear someone say "we're shipping 100x faster" and they can't explain how they ship 1% the number of errors that they did before, run away from their software before it explodes.
Implementation time is becoming decoupled from competence
In the old days[2], If you didn't know how to do something, then it took you a long time. But if you put in the reps, you'd develop expertise and get faster and faster, and eventually it became a natural part of your workflow.
This slowness was a blessing. The slowness was learning. You allowed the problem to impress itself on your brain. You weren't just reading the theory. You were actually developing the muscle memory for execution. You were learning every wrinkle, every pitfall, every exception, and you learned how to handle each of them.
Man, that went out the window, didn't it? Now you can be as clueless as you want to be. When I set up my blog recently, I spent weeks hammering the codebase into the right shape. Even though I generated it with Claude Code, I still have a good idea of how the code is organized and what each piece does. I chose to understand the project.
And when it came time to actually deploy my blog to Digital Ocean, I didn't want to understand. I didn't want to remember how to use Ansible and look up guides for hardening VPS instances. I didn't want to spend days tracking down the cause of obscure error messages. I just have one hour per day of side project time. I don't want to waste it. I told Claude everything I wanted: the Makefile command names, Ansible deployments, hardening, etc. It finished within 20 minutes. And sure, I checked over everything to make sure it wasn't leaking API tokens or anything. But I just read the generated code. I never truly allowed it to flow through my brain.
Did it do a good job? Not any worse than I would have done setting up my first VPS with Ansible in 6 years.
Did I learn anything? Absolutely not! I'm running Caddy in production now, and I have no idea how it's different from Nginx beyond automatically setting up HTTPS.
And that is my central point here. Implementation time is becoming decoupled from competence. Whether I knew how to set up a droplet or not, it would have taken about 20 minutes either way. Sure, an expert might have added bells and whistles, set up some extra monitoring, got Tailscale going, and whatever else experts do with their VPSes. But I did 3 days of 1-hour-a-day side project time in like 20 minutes. And that's bad! I shouldn't be the type of person that can set up a VPS in 20 minutes. It should actually take me a long time because I don't know what I'm doing. In some ways, it's actually dangerous that I can do this.
And that's what we're seeing throughout the industry. People can accelerate tasks outside of their own expertise. So review and expertise are becoming an increasingly important part of the job. Just because someone sent you a change no longer means that they have the competence level required to get it working. I sure didn't when I deployed my blog.
I'm not impressed anymore when somebody says that they pointed Claude at a ticket with its MCP or CLI, implemented the code, wrote tests, and pushed the result to a GitHub pull request with the Github CLI[3]. That's where the work starts now: actually evaluating the prompts and output for correctness, for scalability, for maintainability. For removing all of the little quirks that LLMs introduce. I'm impressed when engineers say, "I found this problem that I wouldn't have otherwise" or "it tuned this better than I could tune it myself" or "I had this insight I never would have had by myself."
Does this LLM speed boost lead to software correctness? If anything, you can now ship code faster if you're clueless because you're unconstrained by reality. You're not pouring over the code looking for API keys it's leaking, looking for obvious scaling bottlenecks, looking for unnecessary bundles you're sending the client. You've merged, and you're already looking for the next ticket. And the number of active regressions in your product just ticked up a bit.
The value of implementation becomes so high that slack time will trend to zero
Slack[4] is an important concept in system resilience. It's the amount of time that an entity is unallocated. It is your tolerance to deviations from the norm. In manufacturing, slack might be the amount of time that a factory is not utilized. At zero slack (i.e. the factory has to run every hour to meet its demand), then even a single hour of downtime needs to be made up. That's when the system comes under pressure. This is when accidents and errors creep in.
For software engineers, slack looks like unscheduled time. From the top down, unscheduled time sounds bad. This is time where engineers don't have guaranteed outcomes. But in reality, a little bit of slack can be some of the most important time that they spend. This is the time that they delete dead code, that they build that observability dashboard that everyone has been putting off, that they say, "This weird thing has been bothering me for a few weeks, I need to look into it. Oh fuck!". It's when you have a chance to say to your teammates, "Why does this part of the codebase feel wrong? What can we do about it?" and whiteboard for a week and come up with the architecture that powers you for the next 5 years.
Obviously no leader says that they want to suffer Knight Capital's fate, or that they wished they didn't have that dashboard that noticed that launch regression, or that they wished the subtle data loss bug was still live in prod, etc. But how do you make it a repeatable business outcome when it comes as the result of unallocated time? You can't.
You might say, "All of these things should obviously be part of any project." But that just misses the point of how software is made. I wrote more about it here, but the tl;dr is that in most methodologies, you set an objective like "Make widget Foo", you set the launch date along with the initial scoping of the project, and then repeatedly negotiate the scope until you have Foo on the launch date (or maybe pushed back a week or two). When your schedule starts to slip, do you know what gets descoped first? Your nice-to-have dashboards, the code cleanup tickets from the last project, etc. All the slack work goes right out the window.
I expect the industry to operate with less and less slack in the future. If we can really accelerate feature implementation, then there's some rate of implementation where it doesn't make sense to give your engineers unallocated time anymore. It just becomes too valuable to perform feature work. "So you're telling me that it used to take my team 3 months of work to figure out whether we might get +/- 2% from an engineering change? And now we can turn it around in 6 weeks?" Your quarter just got 6 weeks back, and you can bet that you are not spending it on anything but implementing more product features. Every implementation hour just got twice as valuable. That downtime between projects you used to have? Now you're spending it writing the specs for the next project.
And it leads to worse software outcomes. Without slack you can't even handle minor bumps in the road. What happens if a project suddenly needs more headcount? There's nowhere to borrow it from; everybody's already allocated. Something needs to get bumped, but you already burned a bunch of engineering time on it. You're already starting to lose the gains. Who's going to go back and delete all of the branches of the old experiments? Nobody. Who's going to spend a week whiteboarding to determine the future architecture of your company? Nobody. Who's going to investigate that weird thing before it becomes a huge problem? Nobody.
Hope for the future
I don't want to just be Doom and Gloom about the future. We can do something about this. Are we going to reduce the error rate by 100x? Probably not. But increased implementation speed applies to everything. It means that we don't have to be stuck in the old paradigm where we pair every single implementation change with a unit test and say that the test means that we verified that our software works.
I can imagine a future where a LLM with the right skill could actually verify that every single line of code has a test that fails if it regresses.
I can imagine a future where it becomes so cheap to produce integration tests that we default to integration tests over unit tests. Anyone who's worked with me knows that I hate mocking frameworks and thinks they lead to worse engineering outcomes, so this could become the perfect axe for me to grind.
I can imagine a world where we get so good at writing integration tests (because we exercise the muscle so much more) that they don't flake all that much.
And it's a bit dangerous, right? It's dangerous for me to assign LLMs these magical capabilities. Just because they can be given a prompt and produce an output doesn't mean it's correct. That doesn't mean it would help.
But the good news is that software engineering is more verifiable than it's ever been. It's never been easier to just take a change and open up one worktree or one checkout and do one technique there and then run a second implementation in parallel and compare the two outputs. What is different? Do you like one more than the other? Did one have more errors on the other?
But I'm not trying to prescribe Exact Solutions. We can imagine a world beyond what software engineering was in 2023. It doesn't have to just be an implementation paired with a new set of unit tests until you retire. We can work together and share our results, share what's working.
Footnotes
[0]: They fixed it since I started writing this, but here was the repro: triple-click a single-line paragraph. Type Ctrl-C (Cmd-C on Mac). Triple click a headline. Type Ctrl-Shift-v (Cmd-Shift-v) to paste without formatting. The line is replaced with the paste content plus three newlines in a row. Even assuming that it needed to keep both the source and destination paragraph's newlines, where did the third one come from?
[1]: While I was writing this post, I read a different treatment of this subject here that views this through the lens of maintenance costs. If you feel like you'd be more swayed by a devex argument, check this out!
[2]: Before 2025.
[3]: I mean, I'm very impressed with the Claude Code and Codex teams that they made an agent where this is even possible. Holy cow. What a time to be alive.
[4]: I am absolutely not referring to the SaaS product here.