Ball says agents cause fewer regressions than humans
Two engineers who already doubt unit tests tell each other the regressions never showed up.
Thorsten Ball posted that he has found regressions to be far less of a problem with agents than with humans, flagging it himself as possibly controversial. No numbers, no benchmark, just the impression of someone who ships a lot of agent-written code.
David Crawshaw backed him up with the closest thing to a figure in the thread. He said production errors are shockingly few these days, that a panic from a missing nil-check happens once every couple of months, and that when it does it usually involves legacy code humans partially wrote. His conclusion was blunter than Ball's.
panic from a missing nil-check happens once every couple of months, and usually involves legacy code humans partially wrote
The unit test argument again
Ball connected it straight back to the fight he keeps having about test suites. He said he feels weird saying it when he doubts the value of unit tests and people respond with "but what about regressions?!", and, as he put it, turns out it's not a big deal.
That is the actual claim being made here. Not that agents write better code in the abstract, but that the specific failure mode people invoke to justify heavy unit test coverage, the silent break of something that used to work, is not showing up for them in practice. The same question came up when Armin Ronacher argued agents should build their own test suites and Mario Zechner pushed back hard.
What this is and is not
For working engineers, note what the evidence is. Two people who already hold this position agreeing with each other, one of whom offers a rough frequency for one class of error in one Go codebase. That is a data point about how a particular team works, not a measurement. Nobody in the thread ran a comparison.
It is still worth reading, because the people saying it are not hobbyists and the specific mechanism is testable on your own repo. Count your regressions by origin for a month. If the agent-written paths break less than the hand-written ones, Ball and Crawshaw are describing something real about your codebase too. If they break more, you have the counterexample this thread does not contain.
Ball has been pushing adjacent ideas for a while, including dropping branches entirely and pushing to main while agents handle the chores. The through line is the same, that a lot of process built to catch human mistakes is aimed at a mistake pattern that has changed.
Have the good opinion, not the loud one
One email a day. Under three minutes. What working engineers found out about coding with AI.