Another AI Insider Quits With an Existential Warning. Don't Expect Anything to Change.

Here's the uncomfortable truth about AI safety: the people who know the most keep quitting, and the people who stay keep building.

Jacob Coxon, a researcher who spent the last three years on pre-training work at both Anthropic and OpenAI, says he walked away from Anthropic because he couldn't stomach what he was seeing. In a series of posts on X, he alleged that the two most valuable AI labs on the planet are racing toward self-improving superintelligence while privately fearing it could kill us all. Gambling with human lives, as he put it. That's not a metaphor to him. It's a job description.

So now we've another insider out the door, another warning posted to the timeline, another round of hand-wringing that will almost certainly change nothing about the speed of the race itself.

The Pattern Is the Story

Coxon's credibility deserves a fair read. He didn't spend six months as a safety intern. He spent three years in pre-training, which is the messy, compute-heavy work of teaching models to predict text better. That's not a peripheral role. That's the engine room.

And he didn't pick one side. He worked at OpenAI first, then moved to Anthropic. That's like quitting the Yankees to play for the Red Sox and then quitting them too because you think baseball itself is dangerous. The track record matters here. When someone has seen both versions of the same machine and concludes that both are pointed at the same cliff, that's not a partisan complaint. That's a structural one.

The timing checks out too. We're in the middle of a compute arms race where labs are spending billions before they've real products that justify the spend. The incentive structure is simple: get to superintelligence first, figure out the rest later. Safety teams exist. They get budget. But they don't set the roadmap, and everyone inside knows it.

The Case for Skepticism

To be fair, I should steelman the other side, because it isn't stupid.

First, Coxon is one person with a subjective read on a company he just left. Resignation can sharpen grievances. The people still working at Anthropic might genuinely believe the safety culture is real and the risk is manageable. They might be right.

Second, doomsday warnings from AI insiders have a track record too, and it isn't spotless. We've been told for years that alignment is the existential question of our time, and yet here we're, still arguing about whether models can do basic arithmetic reliably. The gap between apocalyptic theorizing and mundane reality has been wide before. It could be wide again.

And third, there's a selection bias problem. Insiders who feel the labs are too cautious rarely make headlines. Insiders who feel the labs are reckless always do. The incentives of media coverage mean we hear more from the Cassandras than from the engineers who genuinely believe things are fine.

Granted. All true. None of it changes the core issue.

So What's the Takeaway?

The question worth asking isn't whether Coxon is right about the specific timeline or the specific risk. It's why his warning, like the ones before it, won't meaningfully alter the behavior of the labs in question.

Color me skeptical that it will. OpenAI and Anthropic compete for the same talent, the same capital, the same narrative dominance. Neither can afford to publicly admit the other is moving too fast, because that would mean admitting their own pace is equally reckless. So they'll issue statements about safety, hire more researchers, maybe even release an adorable system card or two. The build continues.

History suggests otherwise whenever someone tries to tap the brakes on a gold rush. The individuals leave, the structures stay, and the race accelerates because the reward for winning is too large and the punishment for losing is total. Coxon gets to sleep better at night. That's a real victory for him personally. It just doesn't do much for the rest of us.

Watch what happens to Anthropic's next funding round. Watch the headcount of its safety team six months from now. If those numbers tell a different story than the press releases, then maybe this resignation actually mattered. I wouldn't bet on it.