Do AI detectors work? How to tell if text was written by AI

Are AI text detectors accurate?

They are reasonable as a hint and poor as a verdict. On a long text generated in one pass and left untouched, a detector will often recognize the pattern. Outside that ideal case, errors climb.

There are two kinds of error, and both matter. A false positive attributes human writing to AI. A false negative passes generated text as human. No detector removes both at once: tune it to catch more AI text and it also flags more innocent people.

How do AI detectors work?

A detector does not know who wrote the text. It measures statistical features of the writing and compares them with what it expects from a person or a machine. Two approaches are common, and they are sometimes combined.

• Predictability and uniformity. Language models tend to choose likely words, so their text is more predictable. That measure is often called perplexity. Detectors also look at how much sentences vary in length and structure, often called burstiness: people tend to mix short and long sentences, while generated text tends to be more even.
• A trained classifier. A model learns from many examples of human and generated text, then decides which group a new text resembles more.

Either way, the output is a statistical resemblance. Predictable, tidy writing looks like AI writing, whoever produced it.

Why do AI detectors get it wrong?

They fail because many people write predictably, and because generated text stops looking generated as soon as someone changes it. The most common cases:

• Short text. A paragraph or a brief answer does not contain enough material to measure anything with confidence.
• Edited or paraphrased AI text. Swapping words, reordering sentences or running the text through another tool removes much of the signal.
• Formal or formulaic writing. Reports, legal text, academic abstracts and form letters follow fixed structures and expected vocabulary, just as generated text does.
• Non-native speakers. They often use a narrower vocabulary and simpler sentences, and there is a documented concern that their writing is flagged as AI more often.
• Translated text. Machine translation leaves an even style that can be mistaken for generated text, even when the original was written by a person.
• Mixed use. Someone who writes a draft and then asks for a grammar cleanup, or who expands an outline they got from a tool, produces text that fits neither category.

Is an AI detector score evidence?

No. A result says the text statistically resembles generated text; it does not say who wrote it or how. It cannot be reproduced like a forensic finding, it varies from one detector to another, and it sometimes changes in the same tool after a minor edit.

That is why nobody should be accused or penalized on a detector result alone. The risk is highest for students and employees, who have little room to defend themselves against a number that looks objective. An accusation of cheating or misconduct needs more than a resemblance: it needs evidence about how the work was produced.

If your school or workplace uses detectors, a sound rule is that a result may only open a conversation. The decision rests on other evidence.

What can teachers and editors do instead of relying on a detector?

Assess the process as well as the product. A person who wrote a text can show how they got there and talk about its content, which is much harder to fake than the final draft.

• Ask for drafts and version history. Online word processors record how a document grew over time.
• Talk about the work. A few spoken questions about the ideas, sources and choices quickly show whether the person understands what they submitted.
• Grade in stages. Outline, first draft and final version, with feedback in between.
• Compare with earlier writing by the same person, keeping in mind that style changes and this is only a hint.
• Set clear rules in advance. Which uses of AI are allowed, which are not, and how to disclose them. Many disputes start with vague rules.

How should you read a detector result?

Be wary of percentages. A figure presented as a percentage of AI implies a precision the method does not have, and it does not mean that share of the text was generated.

A responsible reading has three parts: an estimate in broad levels, how confident that estimate is, and what its limits are. Our AI text detector is built that way. It gives an automatic estimate made by a general-purpose AI model, in three levels (probably written by a person, inconclusive, or probably written with AI), with a confidence level and stated limits. It is guidance, not proof, and it can be wrong like any other detector.

If the result is inconclusive, take it literally: there is no basis for any claim. The same holds for pictures, as explained in how to tell if an image is AI-generated.

What should you do if you are falsely accused of using AI?

Answer with evidence of your process, not with another detector. Gather whatever shows how you wrote the text:

1. Save and present the document's version history.
2. Collect drafts, notes, outlines and photos of handwritten notes.
3. List the sources you consulted and, if you can, when.
4. Offer to explain the work in person and answer questions about it.
5. Ask in writing what the accusation is based on and which tool was used.
6. Read your school's or employer's policy on AI use and its appeal procedure.

As a habit, write in tools that keep version history, and keep your drafts until the work has been graded or approved.

What mistakes do people make with AI detectors?

• Treating the result as a verdict. It is a statistical hint.
• Checking very short text. There is not enough material.
• Reading a percentage as an exact measurement. It is not one.
• Trying several detectors until one confirms the suspicion. That only selects the answer you wanted.
• Ignoring who is writing. Language background, training and genre all change the risk of a false positive.
• Penalizing without hearing the person. The conversation is part of the check.