In 2023, Stanford researchers ran 91 human-written TOEFL essays through seven widely used AI detectors. Averaged across the seven, 61% of those essays were flagged as AI-generated. Every one of them had a human author. The same detectors were nearly always right about essays written by native English speakers.
That gap is not a bug in one product. It follows directly from what these detectors measure, which is not authorship. This page explains what they measure, why fluent and evenly paced writing scores as machine-made, and what to do if it has already happened to you.
Liang, Yuksekgonul, Mao, Wu and Zou, working at Stanford, published GPT detectors are biased against non-native English writers in 2023. It is the one external finding this site leans on, so the numbers on the right are the paper's own, unrounded, with the sample sizes attached.
The authors' recommendation is stronger than ours. They advise against using GPT detectors in evaluative or educational settings at all, and particularly against using them on work by non-native English speakers.
A detector never sees who wrote your text. It sees a sequence of words and asks a language model how surprising that sequence is. Two numbers come out of that question, and both of them punish careful, even writing.
Feed a sentence to a language model one word at a time and ask, at each step, how likely the real next word was. If it is usually the obvious word, perplexity is low. Generated text has low perplexity by construction — a model picks likely words because that is what it does. So low perplexity gets read as machine-made.
Human drafts wander. A long clause, then a short one. A tangent nobody planned. A sentence that lands badly and gets left in. Generated prose holds a steadier line: similar lengths, similar structures, similar surprise from paragraph to paragraph. Low variance gets read as machine-made too.
Both numbers are properties of the text, not of its history. Nothing in either measurement can distinguish a sentence a model produced from an identical sentence a person produced. A detector reporting a percentage is describing the statistical shape of your prose. It is not reporting evidence about you.
The researchers did not stop at the correlation. They took the same TOEFL essays, asked a language model to enrich the word choices to sound more like a native speaker's, and ran them through the detectors again. The average false-positive rate fell from 61.22% to 11.77%, and the number of essays all seven detectors agreed on dropped from 18 to 1.
Then they ran it backwards. They took the US eighth-grade essays — the ones the detectors had handled almost perfectly — and asked the model to simplify the word choices as if written by a non-native speaker. Misclassification on those essays rose sharply.
Same authors, same arguments, same essays. Only the vocabulary changed, and the verdict flipped in both directions. Whatever these detectors are measuring, it is a property of word choice, and it is not authorship.
Low perplexity and low burstiness describe a specific kind of writer: fluent, careful, evenly paced, reaching for the reliable construction over the surprising one. There are a lot of reasons a person writes that way, and none of them is being a language model.
If you learned English from a textbook, you write the constructions the textbook taught, and you reach for the word you are certain of rather than the word you half-remember. That is a correct, careful sentence and a low-perplexity one. This is the cohort the Stanford study measured, and it is where the 61% comes from.
Written for second-language writers →A precise, formal, consistently structured register is a common autistic writing style. It is also, feature for feature, what a classifier trained to spot generated prose scores as machine-made: even sentence lengths, explicit connectives, little tonal drift. The mechanism predicts this cohort gets flagged. The Stanford study did not test it.
Speech-to-text, word prediction and grammar assistants smooth text toward whatever the model behind them considers likely, which is the definition of low perplexity. People who use those tools for dyslexia, aphasia or a motor impairment are writing their own sentences through a ramp. The ramp is what the detector reads.
After a stroke or a traumatic brain injury, writing often comes back as shorter, evenly paced, deliberately plain sentences, because that is what is available. Low variance between sentences is exactly the signal burstiness is built to punish.
One of those four cards is a measurement and three are an argument. The Stanford study tested second-language writers and nobody else. The other three follow from the same mechanism — they describe writing with the same statistical shape — but no published experiment we know of has put a number on them, so we have not put one there either.
A classifier can be wrong in two directions. It can miss AI text, or it can flag human text. Vendors report these together as an accuracy figure, which quietly implies the two errors are worth the same. They are not close.
A missed AI essay costs almost nothing. It gets graded on its merits, like every essay that was ever skimmed by a tired marker. No process starts. Nobody is told. The error is invisible and, in the great majority of cases, harmless.
A false positive starts a misconduct process. It can end a degree, void a visa that was conditional on that degree, cost a job offer, or follow someone as a permanent note on an academic record. The accused writer is asked to prove a negative about a document, which is not a thing anyone can do. And because the mechanism is statistical, the same person tends to get flagged again on the next paper, and the one after that.
So the errors are not symmetric and neither are the people they land on. The Stanford result says the false positives concentrate on writers already carrying the most friction in the system. That is what makes a detector's headline accuracy figure the wrong number to be looking at.
Take a class of 500 essays. Assume 20 of them are AI-written, and assume a detector catches 95% of those while falsely flagging 5% of the honest ones. It catches 19 real cases and flags 24 innocent students. Under half the flags are correct, and the tool is performing exactly as specified. Nothing in that paragraph is measured — it is arithmetic on numbers we made up, and its only job is to show how a small false-positive rate becomes a large pile of accusations when almost everyone is innocent.
The paper states its own limitations. One study is one study.
The samples are small: 91 essays on one side, 88 on the other.
Most of the detectors tested were built on GPT-2 as a backbone model. The paper says explicitly that results may differ for detectors built on newer models.
It covers perplexity-based and supervised classifiers, which is what gets deployed at scale, not every detection method that exists.
It measures one genre of writing, timed academic essays, by one population of second-language writers.
What it does establish is narrow and solid: on this corpus, with these seven detectors, the false-positive rate for non-native writers was an order of magnitude higher than for native writers, and changing the vocabulary moved it in both directions. That is enough to say a detector percentage should never be treated as evidence of who wrote something.
Being flagged is not the same as being caught, and the conversation goes better when you arrive with the mechanism, the paper and your own drafting record rather than with a denial.
A detector score is a probability from a classifier, not a finding of fact. Ask which tool produced it, what the reported percentage actually means, what threshold the institution treats as an accusation, and whether the vendor publishes a false-positive rate for writers like you. Most cannot answer the last one.
The strongest thing you can put in front of a committee is a peer-reviewed paper whose authors recommend against using these tools in exactly this setting. It is short, it is free to read, and it tests the effect in both directions rather than just asserting it.
You cannot prove a negative about a document. You can show how it came to exist: drafts, revision history, notes, the order the paragraphs arrived in. Contemporaneous process evidence is the thing that moves these conversations, and it is worth keeping before you ever need it.
Our editor exports a signed evidence report: your local drafting record, sealed with an ECDSA P-256 signature over a SHA-256 hash. Anyone can check it at /verify against the public key the report carries, in their own browser, without contacting us. It proves the record has not been altered since you signed it. It does not, and cannot, prove who typed the words.
A signed evidence report proves a drafting record has not been altered since it was signed. It proves a drafting record has not been altered since it was signed. It does not prove authorship.
Back to Humanize AI, or check a draft in the editor — the check runs in your browser.