Every text-only detector has the same blind spot, and it is structural rather than a bug that better engineering will fix.
Last reviewed 16 August 2026
The premise of an AI text detector is that machine-written prose carries statistical fingerprints a human would not leave. There is something to that. The problem is that almost every fingerprint anyone has found turns out to be a marker of register — how formal, how even, how conventional a piece of writing is — rather than a marker of who or what produced it.
That distinction sounds academic. It is the whole ballgame, because it means the writing most likely to be flagged is not machine writing. It is careful, formal, conventional human writing.
Strip away the branding and text-only detectors compute some combination of a short list of quantities:
Now ask what kind of human writing scores high on that list. Even sentence lengths, predictable word choice, few contractions, conventional phrasing, restrained punctuation. That describes a scientific abstract. A legal summary. A technical manual. A policy brief. A student writing carefully in a second language. A first-year undergraduate imitating the academic voice they think is expected of them.
The people most likely to be flagged are the people writing most carefully. That is not an ironic aside. It is the direct, predictable consequence of what these statistics measure.
The clearest study on this is from a Stanford group, published in Patterns in 2023. They ran 91 TOEFL essays — written by non-native English speakers — through seven widely used GPT detectors, alongside essays written by US eighth-graders.
Nearly one in five of those essays — 19.78% — was flagged unanimously by all seven detectors at once. A student in that position has no recourse: every tool their institution might reach for agrees, and every tool is wrong.
The mechanism the researchers identified is exactly the register problem. Second-language writers draw on a narrower band of vocabulary and more standard constructions. Detectors read that as machine-like. When the researchers used a language model to rewrite the essays with more varied, idiomatic language, the misclassification largely disappeared — which tells you the detectors were never measuring authorship in the first place.
In January 2023 OpenAI released an AI Text Classifier. By its own published figures it correctly identified only 26% of AI-written text while incorrectly flagging 9% of human writing. On 20 July 2023 OpenAI withdrew the tool, citing its low rate of accuracy.
This is worth sitting with. The organisation with the deepest possible knowledge of how its own models generate text — full access to the weights, the training data, and the sampling procedure — could not build a reliable detector for its own output, and said so publicly rather than keep selling it.
Turnitin launched its AI detection feature claiming a 1% false-positive rate. Vanderbilt University disabled the feature in August 2023 and showed the arithmetic that made it untenable:
Vanderbilt submitted 75,000 papers to Turnitin in 2022. If this AI detection tool was available then, around 750 student papers could have been incorrectly labeled as having some of it written by AI.
Seven hundred and fifty false accusations at one university in one year, from a rate the vendor advertised as a selling point. Rates that sound reassuring in a marketing table become hundreds of individual people once you multiply by real volume.
No amount of tuning turns a register measurement into an authorship measurement. You can reweight the inputs, add signals, or train a classifier on top of them — but if the underlying quantities track formality, the output tracks formality. Better fitting to a benchmark usually means fitting harder to whatever genre that benchmark happens to contain.
A detector is calibrated against the output distribution of the models that existed when it was built. Each new model shifts that distribution. Detectors decay silently: nothing announces that the thing has stopped working, and the confident percentage in the interface looks exactly the same as it did when it was accurate.
Running generated text through a paraphraser, translating it out and back, or spending ten minutes revising by hand will move a score substantially. The people deliberately cheating are the ones most motivated to take that step — so detection pressure falls hardest on people who are not trying to evade anything.
A plagiarism checker can point at a source document. That is a verifiable claim: this passage matches that one. An AI detector has nothing to point at. It produces a number derived from the text's own properties, with no external referent, which is why it can never move past suspicion to evidence.
A low score means very little; plenty of machine text scores low. A high score means the writing is even, formal and conventional — which might be because a model wrote it, or because a careful person did.
The defensible use is as a prompt to look more closely with your own judgement: read the work, compare it to what you know of the writer, look at drafts and version history, talk to them. If the closer look turns up nothing, the score was noise, and the correct action is to drop it.
The indefensible use — and it is common — is treating the number as the finding, working backwards to justify it, and putting the burden on the writer to prove a negative.
If you have been accused on the strength of a detector score, we have a separate guide on what to do: Accused of using AI when you didn't. If you are on the other side of that conversation, see the guide for educators.
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased
against non-native English writers. Patterns, 4(7).
arXiv:2304.02819
OpenAI. New AI classifier for indicating AI-written text (January 2023; discontinued 20 July
2023). openai.com
Coley, M. Guidance on AI detection and why we're disabling Turnitin's AI detector. Vanderbilt
University Brightspace, 16 August 2023.
vanderbilt.edu