OpenAI's EU Watermark Fails Its Own Test, and Nobody Outside OpenAI Can Check

OpenAI switched on an invisible watermark for ChatGPT text and Codex coding output across the European Union this month, a compliance move that got a lot less impressive the moment its own test results came attached.

Here's the number that matters. Swap out a quarter of the words in a watermarked passage for synonyms, and OpenAI's detector drops to 17% accuracy. Roughly one in six. Admittedly, that's a stress test, not a real-world average, and most people won't sit there hunting for synonyms before they paste text into an email. But it tells you what the watermark actually is. A signal, not a lock.

The timeline

The story starts back on August 1, 2024, when the EU AI Act entered into force. Then on August 2, 2025, obligations for general-purpose AI models kicked in, and providers who signed the bloc's Code of Practice agreed to behave ahead of the hard deadlines. OpenAI signed. Meta didn't.

What's happening now is the run-up to Article 50, the transparency rule that requires machine-readable labels on AI-generated text. That one lands on August 2, 2026. Roughly a year out. OpenAI is early, which is either diligence or a head start on looking diligent.

So the watermark goes on. Invisible, statistically embedded in token choices, applied to chatbot output and to code from Codex. And the detector that reads it? That stays inside OpenAI for now. The company says the tech is immature. Fair enough.

What actually changed

Not much, and that's the honest answer. A watermark nobody can verify isn't a verification tool, it's an assertion. Color me skeptical, but compliance you can't audit isn't compliance, it's a press release with a timestamp.

The people who feel this first are the ones building on top of OpenAI in Europe. If you're shipping a product to EU users and leaning on model output, you've now got a supplier who claims its text is marked. You can't check it. You can't test it. You just take the note.

Third-party AI detectors, meanwhile, get nothing new to work with. Their track record on text detection is already shaky, and this watermark doesn't hand them a key. It hands them a promise from a competitor.

And developers using Codex have a more practical worry. Watermarks that survive code edits are genuinely hard, since a variable rename or a reformatted block can wipe a statistical signal clean. Nobody has published numbers on that yet.

What to watch

Watch August 2, 2026. That's when the transparency obligations bite for everyone, not just the volunteers. If OpenAI's watermark holds up under real enforcement, other labs will copy the approach, and the EU will point to it as proof the rule worked. If it doesn't, expect a scramble toward provenance metadata, the C2PA kind that rides alongside a file instead of hiding inside the words.

The question worth asking: does a watermark need to be strong, or does it just need to exist? Proponents of the labeling rule argue the second one, that any signal beats none and the point is disclosure, not courtroom-grade proof. The skeptics, and I lean their way here, think a standard that fails after a thesaurus pass is a standard that teaches bad habits.

OpenAI says it'll publish the detector when it's ready. No date. No criteria. So watch whether the company ships that detector before August 2026 or only after the pressure becomes unavoidable. My guess is after.