Can AI Catch AI? What Detection Tools Can and Cannot Tell Us

Can AI Catch AI? What Detection Tools Can and Cannot Tell Us

This publication is written by Aleksandar Plamenac, Teaching assistant, University of Montenegro, as part of the Ulysseus project METACOG.

In January 2023, OpenAI, the company behind ChatGPT, launched a tool to tell human writing apart from AI-generated text. Six months later, it quietly withdrew it. In OpenAI’s own evaluation, the classifier caught only 26% of AI-written text and wrongly flagged human writing 9% of the time (OpenAI, 2023). If the makers of the world’s best-known chatbot could not reliably recognize machine-written text, how much should we trust the detectors that universities, newsrooms and factcheckers use today?

The question matters because the temptation is strong. As generative AI makes synthetic text, images and voices cheap to produce, it is natural to reach for a technological fix: if AI created the problem, surely AI can detect it. Detection tools can help. But they are far less reliable than their confident percentages suggest, and using them well starts with understanding what they actually measure.

What a Detector Actually Measures

An AI text detector does not find a hidden signature inside a text. It makes a statistical guess. Most detectors ask how predictable a piece of writing is: do the word choices and sentence patterns look like what a language model would most likely produce? Smooth, predictable text (in technical terms, text with low “perplexity”) gets pushed toward the “AI” label.

The problem is that predictable writing is not unique to machines. Careful writers, students following a template, people writing in a second language and anyone producing short, formal text can all write this way. A detector’s score is a probability about style. It is not evidence about who wrote the text.

The Fairness Problem

This is not a hypothetical worry. In 2023, researchers at Stanford tested seven widely used detectors on 91 essays written by non-native English speakers for the TOEFL exam, and on 88 essays by US eighth graders. The detectors were close to perfect on the US essays. On the TOEFL essays, they wrongly labeled more than half as AI-generated, with an average false positive rate of about 61%. Of the 91 essays, all written by humans, 89 were flagged by at least one detector (Liang et al., 2023).

How seven AI detectors labeled 91 TOEFL essays written by non-native English speakers. Data: Liang et al. (2023). Illustration created with AI (Claude)

The cause is the mechanism described above. People writing in a second language tend to use a narrower vocabulary and more standard sentence structures, so their text is more predictable. When the researchers enriched the vocabulary of the same essays, the number flagged by all seven detectors fell from 18 to just 1.

For a European university alliance where most students and many staff write in English as a second language, this matters directly. The people most likely to be falsely accused are the people these tools understand least.

Easy to Fool, Hard to Trust

The same study showed the opposite failure. When AI-generated essays were run through one extra prompt asking the model to use more literary language, detection rates collapsed from as high as 100% to 13% at best. Detectors punish honest writers and are easily bypassed by dishonest ones.

Some universities have drawn the same lesson. Vanderbilt University disabled Turnitin’s AI detector in August 2023 after doing simple arithmetic: it had submitted 75,000 papers in 2022, so even the 1% false positive rate the vendor claimed would have meant around 750 students wrongly flagged in a single year (Vanderbilt University, 2023).

Images, Audio and Video: The Lab Versus the Wild

Deepfake detectors for images, audio and video face a related problem. Many report excellent accuracy on academic benchmarks, but those benchmarks age quickly. In 2025, researchers released Deepfake-Eval-2024, a collection of real deepfakes that circulated online in 2024, drawn from 88 websites in 52 languages. When leading open-source detectors were tested on it, their performance dropped by about 50% for video, 48% for audio and 45% for images compared with the older benchmarks. Commercial tools did better, but still did not match trained human forensic analysts (Chandra et al., 2025).

The lesson is the same as with text. A detector learns the fingerprints of the generators it was trained on. When a new generator appears, or a video is compressed and reposted across platforms, those fingerprints fade.

From Detection to Provenance

Because guessing after the fact is so hard, the field is shifting toward provenance: marking content at the moment it is created. One approach is watermarking, which hides a machine-readable signal in the output itself. Google DeepMind described its text watermark, SynthID-Text, in Nature in 2024 and released it as open source (Dathathri et al., 2024). The other approach is content credentials based on the C2PA standard, which attach signed metadata recording how a file was made and edited.

Europe is now writing this into law. Article 50 of the EU AI Act has applied since August 2, 2026. It requires providers of generative AI to mark synthetic output in a machine-readable way and requires those who publish deepfakes to disclose them. A Code of Practice published in June 2026 explains how, including a common EU icon for AI-generated and AI-modified content (European Commission, 2026).

Provenance is no silver bullet either. The SynthID authors themselves note that text watermarks can be weakened by editing or paraphrasing, and a screenshot can strip metadata from an image. Above all, a missing label does not prove that content is authentic. Still, provenance changes the question from “does this look fake?” to “can we trace where this came from?”, and the second question is far easier to answer.

What This Means for Students and Lecturers

  • Treat a score as a reason to ask, never as proof. No detector output should be the only basis for an academic misconduct decision.
  • Know your tool. Find out what it was tested on and how it performs on short texts and on non-native writers before you rely on it.
  • Make the process visible. Drafts, version history, short oral follow-ups and in-class writing say more about authorship than any percentage.
  • Students: keep your drafts. The version history of your document is your best evidence if your work is ever questioned.
  • For images and video, check provenance first. Look for content credentials, run a reverse image
  • search to find the earliest upload, and ask who published it first.
  • Combine tools with the verification habits discussed earlier in this series: triangulation, source independence and authority.

A Signal, Not a Verdict

AI detection tools are worth having. METACOG itself develops and teaches AI-based tools for spotting disinformation, and they help decide what deserves a closer look. But the first lesson we teach with them is about their limits.

The real danger is not that detectors make mistakes. Every tool does. The danger is that a number on a screen feels more objective than human judgment and quietly replaces it. A detector can tell you where to look. Only people, checking sources, context and process, can decide what is true.

Sources consulted for this article:

About METACOG

METACOG is an EU-funded AI literacy programme designed to combat disinformation and fake news by promoting civic engagement and shared European values.

The initiative aims to strengthen citizens’ ability—particularly among higher education students and teachers— to recognize and respond to disinformation through an innovative curriculum, AI-based tools, and best practices.

By acknowledging the dual role of artificial intelligence in both generating and countering disinformation, METACOG seeks to empower individuals with the critical thinking and digital literacy skills essential for navigating today’s information landscape.

Targeting higher education institutions, government and non-governmental organizations, media professionals, and the general public, METACOG is led by Haaga-Helia University of Applied Sciences (Finland) in collaboration with the Technical University of Košice (Slovakia), the University of Montenegro (Montenegro), and The Hague University of Applied Sciences (The Netherlands). The project is funded under the Erasmus+ Programme (KA220-HED) and will run from 1 September 2024 to 31 August 2027.