AI Coding Weekly

BridgeMind turns down Theo's $10,000 NerfBench bet

The fight over whether anyone can measure model nerfing ends with no transcripts and no bet.

BridgeMind has refused Theo's $10,000 wager over NerfBench, declined to hand over run transcripts, and told him to go build his own benchmark. Theo had offered to match BridgeMind's $10,000 towards benching, with a qualified third party auditing both sides, and said he would donate another $10,000 to a charity of BridgeMind's choice if his own benches showed meaningful nerfing. If they did not, he wanted NerfBench taken down and replaced with an apology page listing the flaws, written by him.

That last condition is where it broke. BridgeMind called some of Theo's questions fair, promised to share more on the NerfBench methodology soon, and then declined the rest.

What Theo wanted answered

His list of things the published methodology post never says runs to nine items, among them which harnesses are used, which tasks, how many times each task runs, which APIs are called, how the plus or minus 10 percent variance band was chosen, and how many runs each dot on the chart represents. He also asked why token counts are weighted alongside cost when cost is the metric that matters, and why Opus runs moved from weekly to daily.

Theo later said he found something worse while prepping his own bench. He accuses BridgeMind of running 6 of 30 tests in a benchmark in April, seeing 2 fail, and calling it a nerf on Opus 4.6 without running the other 24. BridgeMind has not addressed that claim in the posts cited here. Theo says BridgeMind refused the transcripts and called him jealous, and he responded with an accusation about a crypto scam, which is the point at which this stopped being about benchmarking.

Why the chart still matters

The trigger was a NerfBench post putting Claude Opus 5.5 at 94.2 percent, down from above launch the day before, against GPT 6 Astra at 98.0 percent, Sonnet 5.5 at 100.9 percent and GPT 6.1 Sol at 106.7 percent. BridgeMind said 94.2 percent is still inside normal variance and that it is not calling a nerf yet.

For working engineers, nothing here has been resolved. Nobody has published a reproducible method for detecting whether a deployed model quietly got worse, so every chart claiming your model degraded is still an assertion you cannot check. This is where the dispute started, and it is where it stalled.

BridgeMind
@bridgemindai
X
It's about your ego, and not wanting anyone else to exist in this space.
Oct 3, 2026 · View on X
BridgeMind
@bridgemindai
X
94.2% is still inside normal variance, so we can't call it a nerf yet.
Oct 2, 2026 · View on X

Have the good opinion, not the loud one

One email a day. Under three minutes. What working engineers found out about coding with AI.

Get the next one by email

Coding with AI, read daily so you do not have to. The experiments, the receipts and the arguments from engineers shipping real software. Not a changelog.