Theo calls Jev compaction terrible, Armin Ronacher disagrees
A six point teardown of using a small classifier to prune agent history, and the counterargument that harnesses have to prune anyway.
The idea on trial is using Jev, the small fast classifier engineers tried last week, to score an agent's history per tool call and delete whatever it judges irrelevant. Theo posted a six point teardown of that, and Armin Ronacher replied that the idea is worth keeping.
This is a terrible compaction strategy that fundamentally doesn't understand how compaction and context management work.
Theo's six points
Compaction is not a filter, he argues, it is a cleanup that should run sparingly when context gets too long, not constantly to keep context small. The classifier works per tool call and, in the implementation he saw, does not even know the result of the tool call, which he says dooms you to "stupid loops" where the model keeps retrying the same thing.
Then the parts that are about plumbing rather than taste. Frontier models from OpenAI, Anthropic, XAI and Google do not share reasoning traces over the API, they share encrypted payloads that Jev cannot see and often drops, and Anthropic requires you to preserve the entire history to get any reasoning data back. Labs have spent the last year training models on their own compaction flows. And the economics. Cache writes are the biggest cost by far, over 60% of Theo's own LLM spend in Claude Code and Codex, and editing anything early in the history invalidates the cache for everything after it. Delete item two from a history of six and you pay to rewrite three through six, which he says costs more than leaving the junk in.
The pushback
Armin Ronacher did not dispute the mechanics. His point is that harnesses already prune on compaction for cost reasons, so a cheap classifier has somewhere useful to sit.
He also added a detail that cuts across the whole argument. On Anthropic models on new accounts you lose reasoning on retained messages, which is a property of the platform rather than of this particular hack. He suggested hooking into compaction via pi extensions, or hitting the Jev API directly.
Theo later replied to @moinerus, who appears to have benchmarked the approach, with "Thank you for benching it! Would have been awesome if I was wrong here. Sad to see I wasn't." The numbers were not in Theo's post, so nobody reading it can check them yet.
For working engineers, the cache write arithmetic is the part that holds regardless of who wins. Any scheme that edits history near the top pays to rewrite everything below it, so measure token cost, not just context length, before you ship a pruning layer.
I would not dismiss the idea of using Jev for compaction at all. First of all because most harnesses need some pruning on compaction anyways for cost reasons which Jev might help with.