Theo says Gemini hallucinated in over 80% of his review runs
One engineer's harness, one set of receipts, and a failure mode that looks exactly like a real code review.
Theo added a section he calls "gemini the jester" to his council skill, the setup that reviews the same work with several models before anything merges. His claim after running it is blunt. Gemini hallucinates in over 80% of the runs.
Gemini hallucinates in over 80% of the runs
The receipts are what make it worth reading rather than another ranking argument. In one run he lists what the model invented inside a single review. It quoted line 173 of a file that is 72 lines long. It produced random IP addresses. It produced command and script arguments that do not exist, and function exports and returns that are not in the code.
Gemini is a straight up liability in your stack.
Why fabricated review output is worse than bad review output
A reviewer that misses a bug costs you a bug. A reviewer that confidently cites line 173 of a 72-line file costs you the review itself, because the output reads like the work of something that opened the file. Quoted line numbers, exported function names and script flags are exactly the artifacts a human skims to decide whether a review is grounded. When those are generated rather than read, the usual tell is gone.
That is also what separates this from the familiar complaint about a lab being slightly behind. Theo is explicit that he does not mean the model is a few points off the leaders.
Gemini is a straight up liability in your stack.
How much weight this carries
One engineer, one harness, his own codebases. The 80% figure is his count from his council runs, not a published benchmark with a method you can rerun, and he has not shared the harness output beyond the examples above. Nobody has posted a counter-run with their own numbers, so right now the strongest evidence in either direction is his.
It is consistent with what he said earlier this month, when he called Gemini 4 Argon unusable after testing it against his codebases, so treat it as the same person getting the same result twice rather than two independent data points.
For working engineers, the practical takeaway is narrower than the quote. If you run multi-model review, the cheap check is to verify that cited line numbers and symbol names actually exist before you read the critique. That is a few lines of script against any model in your council, and it is the only thing that catches this class of failure, since a fabricated finding is by construction the one that sounds most convincing.
Have the good opinion, not the loud one
One email a day. Under three minutes. What working engineers found out about coding with AI.