dax says custom agent setups fix problems that no longer exist
Kent C. Dodds agrees and says give the agent tools, not instructions it has to religiously follow.
dax posted that the models improve faster than the tinkerers, and that the elaborate custom workflows people build are mostly patches for weaknesses the labs have since fixed. His conclusion is deliberately uncomfortable for anyone with a carefully tuned rig. Someone using the default setup with no modifications, he argues, is more likely to be running at the current frontier than someone who spent a month shaping one.
the person naively using vanilla codex is more likely to be experiencing state of the art
The argument is about decay rather than taste. A workflow is a snapshot of a model's failure modes on the day you built it. The failure modes move. The workflow does not, unless you keep rebuilding it, and the people who enjoy building it are usually not the people who enjoy tearing it down.
Tools over instructions
Kent C. Dodds quoted the post and agreed, with a rule of his own. He wrote that you should be striving to make your setup as vanilla as possible, and that one way to do it is to give your agent tools it can use rather than text of instructions it must religiously follow. That is a sharper version of the same idea. An instruction file is a list of things you currently believe the model gets wrong. A tool is a capability that stays useful whether or not the model needed the hint.
David K replied to dax with "Hooray my laziness is paying off", which is the whole thread's mood in five words.
For working engineers
None of this is a measurement. It is dax's read from watching people's setups, plus two people agreeing with him, and it cuts directly against a year of rules files, custom prompts and harness tweaking that the same crowd has been shipping and sharing. The sources here do not include anyone running a controlled comparison of a vanilla setup against a customized one, so treat it as a strongly held position rather than a result.
The practical version is cheap to try. Look at what your instruction files are actually telling the model not to do, and check whether the current model still does it. Anything that is defending against a fixed behavior is now just tokens you pay for on every turn, and the worst case is that it actively steers the model away from something it has learned to do better than you specified. The parts that survive the audit are usually the ones encoding things about your codebase the model could never have known, which is a different category from the parts compensating for the model itself.
giving your agent tools it can use rather than text of instructions it must religiously follow