Gergely Orosz, an AI-bullish company now caps SOTA models daily
One secondhand account of a team that went from no budget ceiling to a daily one in a year.
Gergely Orosz talked with a company that had unlimited AI budgets a year ago and now runs a daily budget limit for SOTA models. He quotes them directly: use Fable and Astra only for planning, cheaper models are good enough for everything else.
Use Fable and Astra only for planning, cheaper models are good enough for everything else.
Orosz stresses the part that makes it interesting. This is not a company that was ever squeamish about AI spend. The CEO is technical and very bullish on AI, and Orosz describes the place as AI-pilled and one that never looked at AI budgets before. The limit is not a finance department clamping down on a skeptical engineering org. It is a team that wanted to spend the money deciding the top-end models were not worth it for most of the work.
What the cap actually says
Read the quote closely and it is a routing policy, not a cost cut for its own sake. The expensive models keep a job, planning, which is the part of an agent loop where a bad decision propagates into every file that follows. Everything else, presumably the bulk of the tokens, goes to models the team has judged good enough. That is a claim about where model quality stops paying for itself, and this company has drawn the line at the boundary between thinking about the change and making it.
One caveat worth holding onto. This is a single secondhand account, relayed by Orosz from a conversation, with no numbers attached and no name on the company. It is not a trend and it is not a benchmark. What it is, is a concrete instance of a ceiling teams have mostly been arguing about in the abstract.
The abstract version has been getting less abstract for a while. dax earlier moved part of his team off Astra after effective spend doubled. Orosz's company arrived at a similar place from the opposite direction, starting from no limit at all and discovering one.