Ball one-shots 1,780 error-free lines, then aims at 8,000
Thorsten Ball is building a language called Brezel in single agent attempts and posting the line counts.
Thorsten Ball has been running a one-shot ladder, asking how many thousands of lines an agent can write error-free in a single attempt. The test subject is Brezel, a language he is implementing from a spec. The first milestone he posted was 1,780 lines with no errors, with Valgrind, a memory checker, queued up next.
And you think agents will need the same unit tests we need? Nah
One attempt, tokens streamed straight into file, no compilation errors.
His description of how that code arrived is the part worth sitting with. One attempt, tokens streamed straight into the file, no compilation errors. No edit loop, no round of fixing what the compiler complained about.
The levels get bigger
Level 2 was not a toy. Ball's own spec for it called for a bytecode compiler and stack VM with no tree-walking execution and no C recursion per Brezel call, a disassembler behind a `--disasm` flag, closures with shared captured variables, lists, a `fuer ... in` loop, a tracing garbage collector that is actually correct, file and stdin I/O built-ins, and precise error line numbers. He posted "Nailed it."
Level 3 aims for 5,000 to 8,000 lines in a single attempt. He posted "Nailed it" again, and then said it was time to one-shot the JIT.
What this is and is not evidence for
Ball used the run to take another swing at test discipline, which he has argued before does not transfer straight from humans to agents. The jab this time was short and attached to a screenshot of the code.
For working engineers, the thing to hold onto is what kind of problem this is. Brezel is greenfield, written against a spec Ball wrote himself, in a domain with decades of published reference implementations for an agent to have absorbed. A bytecode VM and a tracing GC are hard, but they are hard in a well documented way, and there is no existing codebase with undocumented decisions to trip over. That is close to the friendliest case for one-shotting that exists.
It is also a real signal about volume. Ball is not reporting a demo that compiles and then falls over, he is reporting compilation-clean output at a scale where most people would expect the agent to lose the thread, and he is pushing the number up on purpose until it breaks. Where it breaks is the interesting number, and he has not hit it yet.