Theo gave three models one bug, Sol's fix did not work
Opus filed a PR, Muse pointed at someone else's existing patch, and Sol crashed on its own question tool before shipping a fix that failed.
Theo handed the same bug to Opus 5.5, GPT-6.1 Sol and Muse Wonder Full and posted all three outputs side by side. Opus found the root cause and filed a PR. Sol asked a question, crashed from the async question tool interrupt, then found the bug, fixed it and wrote 300 lines of tests around it. Theo later added that he had confirmed Sol's fix would not have worked.
Muse Wonder Full found a small PR someone else had filed and said "you should probably merge this instead
The third result is the one worth noting. Muse Wonder Full did not write a fix at all. It went looking, found a small PR someone else had already filed, and told Theo to merge that instead. His reaction was short.
The ranking is partly about prose
Theo also posted his current model rankings, with Opus 5.5 and Muse Wonder at the top, GPT-6.1 Sol a tier below, and a gap before everything else. He is explicit that this is not a capability claim. "I'm not saying that Opus is just as capable," he wrote. "I'm saying I like them a similar amount." He says he is doing most of his work with a combination of Opus and Wonder, and that what he likes most about Muse is how different the prose feels, direct, readable, and honoring his AGENTS.md.
I'm not saying that Opus is just as capable. I'm not recommending replacing Opus with Muse. I'm saying I like them a similar amount.
That preference is doing real work in the ranking. Sol's demerit in the bug test was partly that it explained itself in what Theo calls impossible to understand terms, and that was before the fix turned out to be wrong. Muse's win was partly that it was the most readable of the three.
For working engineers, this is one developer, one bug, three runs. It is a taste test, not a benchmark, and the sample size is one on each side. What it does show is that the failure mode worth watching is not a model that declines to try. It is a model that produces a confident fix plus 300 lines of tests that pass around a fix that does not work, which is exactly the output that is most expensive to review.
Separately, @steipete reports that GPT-6.1 Sol in ultrafast mode is far better than 6 Astra at roughly the same cost, at 250 tokens per minute, which lines up with his earlier corner of this story.
Have the good opinion, not the loud one
One email a day. Under three minutes. What working engineers found out about coding with AI.