Vitalik Buterin Backs an AI That's Worse on Purpose

Moonshot AI shipped Kimi K3.1 Pro with a hole in its head, and it put it there on purpose.

The company stripped top-end hacking skills out of the model before release. On its own internal exploit benchmark, the filtered version solved 81% of tasks. The unfiltered one solved 90.5%. That's a 9.5-point pay cut, taken voluntarily, before any regulator asked for it.

Vitalik Buterin thinks that's the right trade. The Ethereum co-founder backed the decision, arguing that capping what an AI can do beats capping who's allowed to use it. His warning cut the other way too. The alternative, he suggested, is KYC for AI. Identity checks, permission slips, a bouncer at the door of every model.

That's an argument worth taking seriously.

Who gets to hold the gate

Access controls sound tidy until you ask who's standing at the gate. It's banks, it's governments, and it's the four or five labs with enough compliance staff to file the paperwork. A solo security researcher in Lagos doesn't get through that line. Neither does a teenager in Manila teaching herself reverse engineering on a secondhand laptop.

Proponents of KYC-for-AI will tell you it's about accountability. Fair enough. But accountability systems have a way of concentrating capability in the hands of whoever can afford to be accountable.

Capability gating is blunter. It makes the tool worse for everybody, including the overwhelming majority of users who'd never write an exploit in their lives. But it's inspectable in a way that identity databases aren't. A benchmark number is something a competitor, a journalist, or a rival lab can rerun. A KYC registry is something you've to trust.

That asymmetry is the whole ballgame, and Buterin is right to point at it.

The 81% problem

Here's where my enthusiasm cools. Four out of five is still four out of five.

If your threat model is a patient attacker, a model that clears 81% of exploit tasks isn't a locked door. It's a door with a slightly better lock. And the unfiltered weights presumably still exist somewhere inside Moonshot, which means the safety margin is a policy decision, not a technical one. Policies change. So does leadership.

The question worth asking: what did Moonshot actually buy with those 9.5 points?

Probably a defensible story for a regulator. Possibly a real reduction in casual misuse, since the ugliest incidents tend to come from low-effort actors wielding high-effort tools. Both of those count for something. Neither is a security guarantee.

Color me skeptical, but a lab grading its own homework on its own internal benchmark is the weakest form of evidence on offer. Moonshot published a number. Nobody outside the building has confirmed how it got there.

What to watch

The tell will be whether Moonshot opens that benchmark to outside reruns. If independent researchers can reproduce the 81% and the 90.5% on the same task set, the number means something. If the methodology stays behind glass, it's a press release with extra steps.

Watch the other labs too. A voluntary capability cut is cheap when nobody's competing on it. It gets expensive the moment a rival ships an unfiltered model and starts winning benchmarks.

My takeaway is narrower than Buterin's. He's right that KYC on AI is the worse outcome, and right that it would freeze the market around whoever can afford the paperwork. But a self-imposed handicap isn't governance. It's a gesture, and gestures hold only as long as the people making them stay in charge.

His thesis rests on labs choosing restraint over time. Track record says that's a shaky bet.