Claude Code and Codex Tests Broke the Supervision Chain. Crypto's Agent Boom Depends On It
Controlled tests with Claude Code and Codex found that weaker models reading evidence for stronger supervisors produce confident, wrong answers. That handoff is exactly how most crypto AI agent stacks are priced, and nobody has written a rule for it.
Two frontier coding agents, Claude Code and Codex, were put through controlled tests with a deliberate handicap built in. A weaker model was handed the evidence. A stronger model sat above it as supervisor. The weaker model got things wrong, and the supervisor, reasoning from what it was given, didn't catch it.
That's the finding, and it's enough to make anyone shipping autonomous software sit up. Not because the models are bad. Because the architecture that half the industry is racing to deploy depends on an assumption these tests just broke.
The Setup
The design is familiar to anyone who has priced an agent pipeline. You don't run the expensive frontier model over raw context if you can avoid it. You put a small, cheap reader in front, let it work through the logs, the filings, the transaction history, the test output, and have it hand up a condensed version. The frontier model makes the call from that summary.
Here's the thing. The failure didn't come from either model working alone. It came from the handoff.
When the summary is wrong, the supervisor often has no way to know it's wrong. It reasons cleanly from bad inputs. It produces a confident, well-formed answer built on nothing, and it sounds exactly like the answer it would have produced from good inputs. That's the part that should worry people. A sloppy model that hedges is annoying. A sloppy model that hands up a crisp summary is dangerous.
Anyone who has managed junior staff will recognize the pattern. The managing director who never opens the filing. The partner who trusts the memo. The problem was never the analyst. It's the process that gave the analyst no channel to flag uncertainty upward, and gave the partner no reason to check.
Details on which failure mode dominated are thin. That matters, because a summary that drops a caveat is a different defect from a summary that invents a number, and they need different fixes. But the broad result holds either way. Reading the evidence is a skill, and it doesn't transfer for free.
The Cost of a Cheap Reader
Now look at how crypto actually builds. Trading agents that watch order books and hand a thesis to a bigger model. Wallet monitors that summarize on-chain activity before a risk model scores it. DeFi automation where a small model reads state and a large one decides whether to move funds. Compliance tooling that condenses transaction histories for review.
Every one of those is a two-tier chain. And every one of those was justified on cost. There's roughly a tenfold price gap between frontier models and small ones on a per-token basis, sometimes more. At scale, that gap isn't a rounding error. It's the difference between a product that ships and one that doesn't.
So the industry made a cost decision and then told itself it was an engineering decision. The tests suggest it was neither. It was a risk decision, and it was made by default.
My first strong opinion: the cheap-reader pattern is going to produce the first serious, well-publicized agent loss in crypto, and it won't be a model hallucinating out of thin air. It'll be a perfectly logical decision made from a summary that dropped one number.
Second: on-chain transparency doesn't save you here. This is the part people get wrong. Putting an agent's reasoning on-chain, hashing its inputs, publishing the full trace, none of that fixes a bad summary. A wrong read recorded immutably is still a wrong read. It's just permanent now. Auditability tells you what the model saw. It doesn't tell you whether what the model saw was true.
Who benefits from all this? Firms selling verification layers, and honestly, they should. Whoever builds a reliable way to score a summary against its source, cheaply enough to run on every handoff, has a real product. Who loses? The teams that assumed the expensive model upstairs would catch everything. It won't. It can't see what it wasn't shown.
There's a regulatory shoe waiting to drop, too. Banks have run model risk management under Federal Reserve guidance, SR 11-7, since April 2011. It requires validation, documentation, and an owner for every model in the chain. Crypto firms holding trust charters or banking partners are about to discover that a two-tier agent pipeline is two models, not one, and each needs its own file. Reading the legislative tea leaves, no one on Capitol Hill has written a thing about hierarchical model supervision yet. That gap won't stay open.
The Takeaway
The question now is whether teams will treat the reader model as infrastructure or as a liability. Infrastructure gets validated. Liabilities get trusted and forgotten.
The practical fix isn't glamorous. Force the reader to pass up its sources, not just its conclusions, and make the supervisor spot-check a sample of them on every run. Contradiction checks beat confidence scores. And if a summary can't be traced to a line in the original, treat it as unverified, no matter how clean it reads.
The models aren't the weak link. The handoff is. Build like you believe that, because the tests already do.
Explore More
Key Terms Explained
An autonomous program that can perceive on-chain data, make decisions using machine learning models, and execute blockchain transactions without human intervention.
Following the laws and regulations that apply to financial activities, including crypto.
An Ethereum Layer 2 in the Optimism Superchain ecosystem that incentivizes developers and users through its referral and fee-sharing system.
Transactions and data recorded directly on the blockchain.