Theo says Gemini 4 Carbon's slowness is server side, not model
The 11 second wait before first token is 20x Fable, and Theo says it is an inference problem a better checkpoint will not fix.
Theo dug into why Gemini 4 Carbon feels so slow in agent work and landed on an answer that has nothing to do with the model's quality. Almost all of it is server side, he said, with 11 seconds elapsing before the model starts to respond, 20x longer than Fable.
Almost all of it is server side. Their inference is awful.
That follows his earlier run of timings, where he called Carbon unacceptably slow in real world usage. The numbers he posted were 22m21s for Gemini 4 Carbon on low reasoning against 9m33s for Opus 5.5 on high, and a fix he asked it to write for T3 Code Windows installations that took fourteen hours.
The prefill excuse
Theo says Varun attributed the delay to prefill being hard. Prefill is the stage where the model chews through the input prompt before it emits a single token, and it is exactly the stage that balloons when an agent hands over a large context on every turn. Theo's response was that the other labs appear to have figured it out, so Google can too.
That distinction matters more than the raw stopwatch numbers. A slow model that is slow because of weights gets fixed by a better checkpoint. A slow model that is slow because of serving infrastructure stays slow until the infrastructure changes, no matter how good the next Gemini is on evals.
It is one engineer's timings on his own workload, not a benchmark, and Theo has been consistent that the Gemini team made real progress and that there are useful capabilities buried in the model if you go looking. His complaint is narrow. On a later post about a model billed as built for speed, he noted he ran Sol and Opus at regular tier and did not even use fast or ultrafast variants, and still found Gemini's speeds poor by comparison.
For anyone running agents in a loop, time to first token is the number that decides whether a tool is usable, and right now nobody outside Google knows when that number moves.
I had it write a fix for T3 Code Windows installations, and it took FOURTEEN HOURS to do it
Have the good opinion, not the loud one
One email a day. Under three minutes. What working engineers found out about coding with AI.
