AI Detection Tools Produce False Positives, Enable Extortion
Three AI detection tools tested by journalists produced false positive results classifying human-written content as AI-generated, then solicited payment for correction services. Researchers warn the tools can be exploited to damage reputation and contribute misinformation, particularly targeting students and academics.
What happened
Journalists from Agence France-Presse tested three AI detection tools—JustDone AI, TextGuard, and Refinely—claiming to identify AI-generated content. Testing across four languages (Dutch, Greek, Hungarian, and English), the tools classified all human-written texts as AI-generated, including 1916 Hungarian literary classics.
JustDone AI analyzed a human-written article about war in the Middle East and incorrectly concluded it was '80%' or '88%' AI-generated, then offered to 'humanize' the text for a fee up to $9.99. TextGuard and Refinely produced similarly false positive results with attached paywalled services. Some tools operated without internet access, suggesting pre-written results rather than genuine analysis.
Reported impact
Researchers warn the tools can be deliberately misused to damage credibility and reputations. In Hungary, government-aligned influencers screenshotted JustDone AI results to falsely claim opposition campaign documents were entirely AI-generated.
The tools target students and researchers seeking to conceal AI use. Companies claim users include prestigious universities such as Cornell, though Cornell denied any association to Agence France-Presse. Academics describe the tools as contributing an additional layer of misinformation to an already fragmented information environment.
Index analysis
TextGuard stated its results are 'reliable within the limits of the technology' but cannot guarantee accuracy or compare to other systems. JustDone acknowledged that 'no AI detector can guarantee 100% accuracy' and admitted its free version may produce less accurate results due to high demand and use of a 'lighter model for speed of access.'
A user complaint cited that JustDone classifies entirely human-written texts as AI-generated. Researcher Deborah Fiber-Wolf, based in Germany, stated these are 'not detection tools but fraud under the guise of humanization,' offering absurd alternatives.
Classification
Sources and evidence
This record was researched and written by the Index, based on the following original reporting.