Gemini 4 Argon aces Theo's bench, then fails his other one

Google started rolling out Gemini 4 Argon, and the first two reports from people who had it early do not line up.

Theo, who says he got early access through friends at DeepMind, called it a huge jump for Gemini and good value for the price, and said it is the first ever model to get a perfect score on Skatebench-private. That is the top of his own eval, not a public leaderboard, so take it as one person's private benchmark rather than an industry result.

Theo - t3.gg

@theo

X

It’s still a Gemini model though. Does weird shit regularly and I don’t feel like I can trust it at all.

Then he pointed it at a second eval. On his codebase analysis bench, Argon came out as the only model with a false claim rate above 50%, meaning more than half of the statements it made about the code did not hold up. His summary of the gap was blunt.

dax has it in production use

@thdxr, separately, called Argon a frontier model and said it was the second highest used model by his team last week. He did not publish a benchmark, but that is usage data from a team shipping with it rather than an impression from a demo, and it is the strongest signal in either direction so far that the model holds up on real work.

dax

@thdxr

X

google has finally done it this is very much a frontier model

So the disagreement is not really about capability. Both engineers think Google made a real jump. The split is about whether you can take the model's word for anything it tells you, and that is a different axis than how well it writes code.

For working engineers

A perfect score on a coding eval and a bad false claim rate are not contradictory. One measures whether the model produces a correct artifact when you ask for one. The other measures whether its description of an existing system is true, which is what you are relying on every time you ask an agent to explain a module before you let it touch the module. A model can be excellent at the first and unreliable at the second, and the second is harder to notice, because wrong answers about your codebase read exactly like right ones until something breaks.

The practical read is to treat Argon's analysis output as a hypothesis and its code output as code you review normally. Two engineers with early access, one benchmark each, no third-party numbers yet. Nobody has published a broader evaluation.