A detector that is "99% accurate on human writing" can still be wrong about a large share of the people it flags. The arithmetic is short, and it is worth doing once.
Last reviewed 12 November 2026
Vendors of AI detectors usually quote one headline number: the false-positive rate, the share of human-written text the tool wrongly labels as machine-generated. One percent sounds like a rounding error. It is not the number that matters to a person who has just been flagged.
The question that person needs answered is different: of all the texts this tool flags, how many are actually human-written? That depends on the false-positive rate, on how much AI text the tool catches, and — the part that is usually left out — on how common AI-written work is in the pile being checked.
Take 1,000 essays. Suppose 100 of them were written with AI, and the detector catches 80% of those while wrongly flagging 1% of the human ones.
So even with the vendor's best-case number, about one flag in ten is a false accusation. Nothing in the tool's output distinguishes those nine essays from the other eighty. They all get the same confident percentage.
Now change one assumption. Suppose only 20 of the 1,000 essays involved AI — a class where most people are doing their own work.
The detector did not get worse. The same tool with the same error rate became far less trustworthy because the thing it looks for became rarer. This is the base-rate problem, and it applies to any screening test, from medical scans to airport security. A flag means the least in exactly the groups where honest work is the norm.
The 1% figure is a vendor claim measured on the vendor's own test set. Go back to the first example — 100 AI essays in 1,000 — and move the false-positive rate to 5%:
Independent testing has found rates much higher than 5% for some groups. In one published study, seven detectors wrongly flagged an average of 61.22% of essays by non-native English writers. For those writers a flag carries almost no information at all. Our guide on AI detectors and non-native English writers covers that study.
The rate applies every time a piece of work is checked. A student who submits 20 pieces of work in a year, each with a 1% chance of a false flag, has about an 18% chance of being wrongly flagged at least once that year. Across 40 submissions it is about 33%.
Those figures assume every check is an independent coin flip, which is too kind. False positives follow writing style, so they cluster. A student whose natural prose is formal and even is likely to be flagged again and again, while others are never flagged at all.
Institutions feel the same multiplication. Vanderbilt University worked out that a 1% rate applied to the 75,000 papers it submitted in a year would have meant around 750 wrongly flagged papers, and switched the feature off.
A flag is a reason to look more closely, and nothing more. It cannot carry an accusation on its own, because nobody — including the vendor — can say how likely it is to be right for this particular writer and this particular piece of work.
If a flag has already been used against you, start with Accused of using AI when you didn't. If you are deciding how to use these tools in a course, see the guide for educators.
Liang, W., Yuksekgonul, M., Mao, Y., Wu, E., & Zou, J. (2023). GPT detectors are biased
against non-native English writers. Patterns, 4(7).
arXiv:2304.02819
Coley, M. Guidance on AI detection and why we're disabling Turnitin's AI detector. Vanderbilt
University Brightspace, 16 August 2023.
vanderbilt.edu
The worked examples on this page are illustrative arithmetic, not measurements of any specific
product.