← Provenote blog
reliability principles

The case we cannot solve — and why nobody else can either

An author drafts with AI, then edits it into their own voice. Can anyone detect that? No. Here is exactly why, in the model builders’ own words, and what follows from admitting it.

Deepak R Chandran, Ph.D.
Deepak R Chandran, Ph.D.
Founder, Provenote · · 7 min read

Here is the scenario every institution actually worries about, and it is not the lazy one. A capable writer opens a model — not Claude, say, but whatever they prefer — asks it for a draft, and then does real work on it. They restructure the argument. They cut the padding. They put it in their own voice. What arrives on your desk is genuinely theirs in every way that matters to a reader, and was not theirs at the moment of first composition.

Can we detect that? No. And we would rather tell you that plainly than sell you a number.

Four reasons, and any one of them is enough

First: most AI text carries no mark at all. Watermarking text is not an industry norm; it is a handful of implementations. A model run on someone’s own hardware has no watermark by construction, because nobody was there to add one. If the tool your author used does not watermark, there is nothing to look for. Not a faint signal — nothing.

Second: editing degrades the mark, and the people who build these systems say so. A statistical watermark is a bias in which words get chosen, spread thinly across a long passage. It survives some handling and not other handling, and the boundary is documented. OpenAI, describing the text watermarking method it built, reported it was “highly accurate and even effective against localized tampering, such as paraphrasing”, but “less robust against globalized tampering; like using translation systems, rewording with another generative model”. It withheld that method for roughly two years on fairness and circumvention grounds, and announced deployment on 5 October 2026: watermarking for ChatGPT and Codex output in the EU, opt-in for API customers elsewhere, and detector access opened by application to approved researchers and institutions. The documented boundary above did not move. Only who is allowed to look.

Read that carefully, because the distinction matters. Tidying a sentence may leave the signal intact. Running the whole thing through another model, or translating it and back, does not. And “I rewrote it properly” sits on the wrong side of that line.

And this has been measured, not just asserted. In the same 2023 Patterns study, the researchers had ChatGPT write essays to the 2022–2023 US Common App admission prompts. Detectors spotted them reliably at first. The researchers then asked the model to self-edit with the single instruction “Elevate the provided text by employing literary language” — and detection rates fell to near zero. A parallel run on scientific abstracts behaved the same way, and in both cases the text’s perplexity rose sharply after the edit. One prompt. Published, peer-reviewed, three years old. Their own conclusion was that a simple manipulation of prompt design can bypass current detectors.

The organisations most able to build a watermark are the ones telling you what it cannot withstand. That is worth more than any accuracy figure on a pricing page.

Third: there is no credential. A Content Credential has to be attached when a document is made or exported. Text pasted out of a chat window carries none, and one cannot be bolted on afterwards and still mean anything. A credential added later says only that someone added a credential later.

Fourth: style comparison cannot close the gap. Comparing a document against an author’s previous writing needs previous writing to compare against, and it degrades precisely when a person’s style legitimately changes — a new topic, a co-author, a strict house style, a second language. In Provenote it is a capped corroborating signal and can never, alone, support a conclusion about a person. We are not going to promote it into one because the alternative is admitting we found nothing.

This is not our limitation. It is the field’s.

It would be convenient to present this as a gap we are working on. It is not. Where the editing pass leaves behind no robust watermark and no contemporaneous record, there is frequently no physical evidence left to find, and attribution from the final text alone is then not reliably achievable by us or by anyone. We should be precise about the boundary: this depends on the watermark, the kind of editing, the length of the text and the conditions it is evaluated under, and some deliberately embedded watermarks do survive some paraphrasing when enough signal remains. What it is not is a hard engineering problem awaiting a clever solution. In the case above it is an absence.

Which is why the claim deserves attention whenever you hear it. A tool that reports high confidence on this scenario is not detecting a watermark, because there is none. It is scoring linguistic features — often how predictable the prose reads, though detector architectures vary and not all of them reduce to that — and careful, well-structured, plainly-written work scores as machine-like on exactly those features. So does writing by someone working in their second language. That is the mechanism behind the false positives, and it is why the same tools flag the same innocent people over and over.

OpenAI listed that fairness problem among its reasons for holding the technology back: watermarking could, in its words, “stigmatize use of AI as a useful writing tool for non-native English speakers”. The company holding it withheld release for about two years partly because of who would be harmed when it was wrong. When it did ship, in October 2026, it shipped as a transparency measure rather than as a detection product, and kept the detector behind an application process rather than selling it.

What follows from admitting it

One sentence contains the whole strategy:

You cannot detect the absence of provenance. You can only create provenance.

If a document was made without any record of how it was made, no amount of examination afterwards manufactures one. The question can only be answered in advance, by whoever was present at the writing. That is an unwelcome conclusion for anyone selling detection, including the parts of what we do that look like detection.

So the useful move is not a better guess about the past. It is a verifiable record made during the work: a signed chain of drafts an author can produce on request, provider watermarks read properly where they exist, a content credential where a tool can write one. Evidence, gathered while it is still gatherable.

And it has a precondition we state every time, because omitting it would be the same overclaim in a friendlier costume: a record only helps if it exists before the question is asked. We can evaluate historical evidence that already exists. We cannot retrospectively create an authentic contemporaneous record of events that were never recorded. Nobody can.

What we will tell you instead

Send us a document of unknown origin and we will not perform confidence we do not have. You will get an account of which provenance signals could be checked, which could not, and why — and if nothing could be checked, we say that the result carries no information about that document, is not an indication a person wrote it, and must not be used as one.

That is a less satisfying product than the one being advertised elsewhere in this market. It is also the only version that does not eventually put its user in the position of having accused someone on the strength of a number nobody checked.

Note

More on how signals are weighted in our Methodology, the measured limits in Reliability, and the lines we will not cross in Principles.

Sources

  1. OpenAI — “Understanding the source of what we see and hear online” (text-watermarking update, Aug 2024)
  2. Anthropic — How Claude’s text watermark works
  3. Liang et al., “GPT detectors are biased against non-native English writers,” Patterns 4(7), 2023 — the Common App self-edit experiment
  4. OpenAI — text watermarking for ChatGPT and Codex in the EU, API opt-in, and detector applications, announced 5 Oct 2026. NOTE: openai.com returns 403 to our automated fetches, so this is corroborated from independent reporting and was not read from the primary source
  5. Provenote — Reliability
  6. Provenote — Principles
Deepak R Chandran, Ph.D.
Deepak R Chandran, Ph.D.
Founder, Provenote

Deepak R Chandran, Ph.D., is the founder of Provenote. He writes about content provenance, AI watermarking, and building verification that is honest about what it can and cannot prove.

Get provenance you can prove

Provenote is in private beta. Request early access →

More reading: Absence of a watermark is not proof of a human →