Theo: Gemini hallucinates in over 80% of runs
Theo added a "gemini the jester" slot to his multi-model review skill and says Gemini made things up in over 80% of runs, including a line 173 in a 72-line file. Meanwhile Steinberger's two week agent loop finished a framework migration in about 600 PRs.
The big disagreement
Opus 5.5 got dumber, says Thariq; Theo ends the week calling it the best ever
Thariq asked whether Opus 5.5 is dumber, and the week ended with Theo saying it is the greatest model of all time and that he is still struggling to find the ceiling. Earlier Theo had mocked NerfBench, which showed Opus 5.5 scoring 5.8% smarter since launch, as possibly the worst benchmark ever. No one in the thread posted a test behind the "dumber" feeling.
This might legitimately be the worst benchmark of all time
Receipts
Theo's 18-hour search hill climb became a daily shipping pledge
After rebuilding T3 Code's search in an 18-hour loop, Theo committed to one meaningful improvement a day for as long as he can keep it up, and counted the search overhaul as day 21. The loop itself: clone real data from the production DBs, collect queries that returned bad results, synthesize 50 more examples, then hill climb with Opus, Sol and Wonder Full until 53 of 54 cases passed.
Hill climbed with Opus, Sol and Wonder Full until I hit 53/54 cases
Steinberger's Lit to Solid migration landed in about 600 PRs
Peter Steinberger's two week agent loop finished moving OpenClaw's Lit web frontend to Solid 2.0: roughly six days until Lit could be removed, another week to make the patterns idiomatic, about 600 PRs, checked on every other day. He reports smaller JS and CSS, finer-grained updates, and a few tiny regressions the agent missed, mostly button shapes. Astra planned, Sol 6.1 wrote the code.
Technical dept is can be counted in token now.
Theo says Gemini hallucinated in over 80% of his review runs
Theo added a "gemini the jester" section to his council skill, which reviews work with several models, and says Gemini hallucinates in over 80% of the runs. In one run he lists a quoted line 173 from a 72-line file, invented IP addresses, made-up script arguments and function exports that do not exist. Nobody else has published a counter-run.
Gemini is a straight up liability in your stack.
How they actually work
Mario Zechner's whole agent method: tell them, then yell at them
Asked how he works with agents, Mario Zechner answered "i tell them what to do. when they do it badly, i yell at them", then posted it as his 2026 workflow and told people to ignore every other guide. It is a jab at elaborate prompt scaffolding, not a benchmark.
Frequently asked questions
Is Opus 5.5 better or worse than before?
There is disagreement: Thariq questioned whether it got dumber, while Theo called it the greatest model of all time. NerfBench showed it scoring 5.8% smarter since launch, but Theo called that benchmark possibly the worst ever, so the debate runs on opinions rather than agreed-upon evidence.
What is hill climbing in AI development?
Hill climbing is an optimization process where you repeatedly test and improve something (in this case, search relevance) by making small adjustments until you get better results. Theo used it to fix search by collecting real failing queries, generating more examples, and iterating with different models until 53 of 54 test cases passed.
What is Lit and Solid in web development?
Lit and Solid are web frameworks used to build user interfaces. Peter Steinberger migrated a codebase from Lit to Solid 2.0, which resulted in smaller JavaScript and CSS files and more frequent, precise updates to the page.
Why does Gemini hallucinate so much in code review?
Theo found that Gemini hallucinates (makes up false information) in over 80% of his review runs, inventing things like file line numbers, IP addresses, and functions that do not actually exist in the code. The problem is that these fabrications look plausible enough to pass review.
Do AI agents need complex prompts to work well?
Not necessarily: Mario Zechner ships using a simple method of telling agents what to do and correcting them when they fail, while others use elaborate prompt scaffolding (structured instruction techniques). Both approaches claim to work, suggesting the complexity of your instructions is not the main factor in success.