AI Coding Weekly

Two teams benchmark Jev, and it wins on speed and accuracy

Independent evals from Malte Ubl and the fx team both put Jev ahead of the small models they were already running.

Malte Ubl ran TypeSafe's Jev against an existing classifier eval that had been served by Gemini 2.5 Flash Lite. Jev won on both axes, saturating the eval on quality and coming in 6x faster on speed. He added that it is now on AI Gateway.

Malte Ubl
@cramforce
X
It won both on quality (saturated the eval) and speed (6x)
Sep 16, 2026 · View on X

That is the second look at Jev from engineers who had already kicked the tires on it earlier, and this time there are numbers attached.

The fx numbers

Guillermo Rauch posted results from a different workload. The default mode in fx is auto, with a safety reviewer analyzing every command, and that reviewer runs on GPT Luna today. Against it, Rauch says Jev is up to 18x faster at p95 and more accurate. He says it is coming to Vercel AI Gateway and is likely the new default.

Guillermo Rauch
@rauchg
X
Jev is up to 18x faster (p95) *and* more accurate.
Sep 16, 2026 · View on X

Pranit published the underlying benchmark of the fx auto mode safety classifier, putting Jev at roughly 5x to 18x faster and more accurate than gpt-5.6-luna, which he describes as their current top choice. Two independent teams, two different classification jobs, same direction of result.

Why latency is the whole story

A safety reviewer that reads every command before it runs sits directly in the critical path of the agent loop. So does a router, a scorer, or anything else that decides what happens next. At that position, tail latency is what the user feels, which is why the p95 figure is the one Rauch led with rather than an average.

dax showed the same idea in motion, posting a preview of browser use powered by Jev and opencode's browser use CLI, with speed as the point of the demo.

For working engineers, this is not another chat model to swap into your editor. It is the small model you already call thousands of times a day for decide, route and score work, and the claim on the table is that the fast option no longer costs you accuracy. Both results come from teams reporting on their own evals rather than a public benchmark, so the honest read is two strong data points, not a settled ranking.

Get the next one by email

Coding with AI, read daily so you do not have to. The experiments, the receipts and the arguments from engineers shipping real software. Not a changelog.