Signal Check Research Prototype

How AI text detection actually works

Four approaches, what each one measures, and the specific reason each one fails.

Last reviewed 16 August 2026

"AI detector" covers several quite different techniques with different failure modes. Knowing which one a tool uses tells you a lot about what its number is worth.

1. Perplexity

Perplexity measures how surprised a reference language model is by a piece of text. Feed it word by word and ask, at each step, how probable the next word was. Text made of highly probable choices has low perplexity; text that keeps taking unexpected turns has high perplexity.

The logic is that language models sample from high-probability regions of their own distribution, so their output should look unusually predictable. There is real signal here.

Where it breaks

• Predictability is a property of genre. Technical documentation, legal boilerplate, and formulaic academic prose are low-perplexity because the conventions constrain word choice. So is writing by someone working in a second language with a narrower active vocabulary.
• It depends on the reference model. Perplexity is measured against whatever model the detector uses, which is rarely the model that produced the text. A mismatch produces systematically wrong numbers.
• Sampling settings move it. The same model at a higher temperature produces higher-perplexity output. A one-parameter change on the generating side shifts the measurement.

2. Burstiness

Burstiness measures variation — typically in sentence length and complexity. Human writing tends to lurch: a long winding sentence, then a short declarative one, then a fragment. Generated prose is often more metronomic.

This is usually the strongest single statistic in a surface-level detector, and it is the heaviest-weighted signal in the tool on this site's home page.

Where it breaks

• Edited writing is smoother. Revision tends to regularise sentence length. The more carefully a human edits, the more machine-like they score.
• Structured formats are inherently even. Abstracts, executive summaries and lab reports have low burstiness by design.
• Short samples are noise. Variance estimated from a handful of sentences is statistically meaningless, which is why most tools produce wildly unstable results on anything under a few hundred words.

3. Trained classifiers

Rather than hand-picking statistics, train a model on a labelled corpus of human and machine text and let it learn the boundary. This is what most commercial detectors do, and it is strictly more powerful than the surface approach — a classifier can pick up patterns nobody thought to specify.

Where it breaks

• It learns the training distribution, not the concept. If the human samples skew toward one genre and the machine samples toward another, the classifier learns to tell those genres apart and reports it as authorship. This is the single most common failure, and it is invisible in benchmark scores because the benchmark shares the same skew.
• It decays silently. Calibrated against models that existed at training time. Each new release shifts the distribution and nothing in the interface announces it.
• Confidence is not calibrated. A classifier reporting "98% AI" is reporting distance from a decision boundary, not a 98% probability of being right.

OpenAI's own attempt is the instructive case. Its AI Text Classifier, released January 2023, had full institutional knowledge of how its models generate text. Published performance: 26% of AI text correctly identified, 9% of human text falsely flagged. It was withdrawn on 20 July 2023 for low accuracy.

4. Watermarking

Watermarking is the one approach that is not really detection at all. Instead of inspecting finished text for clues, the generating model deliberately biases its own word choices in a statistically detectable pattern — imperceptible to a reader, verifiable by anyone with the key.

This is fundamentally stronger, because it does not infer authorship from style. It reads a signal that was deliberately embedded.

Where it breaks

• It only works if the generator cooperates. Any model without watermarking — including open-weight models anyone can run locally — produces text with nothing to find.
• Paraphrasing degrades it. Rewriting disturbs the token-level pattern the watermark lives in.
• Absence proves nothing. No watermark means "not produced by a cooperating watermarked model," which is not the same as "written by a human."

A common thread

Three of these four approaches infer authorship from properties of the finished text. That is the structural problem: the properties they measure are also properties of careful, formal, conventional human writing. You can improve the statistics, add signals, or train a bigger classifier, and the confusion remains, because it is not a modelling error — the categories genuinely overlap.

Watermarking escapes this by not inferring anything, which is why it is the only approach with a clean theoretical story. It also requires cooperation from every generator in the world, which is why it does not solve the practical problem.

---

Sources

OpenAI. New AI classifier for indicating AI-written text (January 2023; discontinued 20 July 2023). openai.com
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased against non-native English writers. Patterns, 4(7). arXiv:2304.02819