AI-Detection Tools Have Made a Major Leap Forward — But Can They Be Trusted?

Posted on 31 August, 2026 by Mehrdad Fathi

When Daniel Evanko asked a scientist whether they had used artificial intelligence to write their peer-review report, he didn’t expect a confession. Most researchers don’t reveal AI help, says Evanko, who is the director of journal operations at the American Association for Cancer Research (AACR). But this time was different. “Wow, you guys are good!” the reviewer wrote back, admitting he had used a large language model (LLM) after running out of time.

Evanko had a secret weapon: a commercial AI-detection tool called Pangram, deployed by the AACR because of concerns over the number of peer-review reports that appear to use AI without disclosure, contrary to publisher policy.

After years of disappointing results, multiple firms now claim their software can reliably distinguish AI-written from human-written text. Pangram Labs, a New York City start-up, advertises “99.98% accuracy.” Its competitor GPTZero claims 99% accuracy and “the most precise, reliable AI detection results on the market.”

From Statistical Guesswork to Machine Learning

Until recently, AI-detection software was notoriously unreliable, says Marzena Karpinska, a computer scientist at Simon Fraser University who has independently analysed these tools. Older approaches relied on statistical proxies: AI text tended to show higher “perplexity” (word predictability) and lower “burstiness” (variation in sentence length), or telltale stylistic quirks. These methods too often misfired, flagging human-authored text as AI-generated — enough that some universities banned the software outright.

The shift came in early 2024, when Pangram co-founders Max Spero and Bradley Emi introduced a different approach. They gathered millions of human-written texts, asked LLMs to generate “mirror” versions with matching title, length and tone, then trained machine-learning models to distinguish the two — refining performance by repeatedly feeding back the hardest cases. The resulting models are effectively black boxes: they detect subtle, model-specific fingerprints that resist plain-language description.

The results were striking. Pangram reported a 0.02% false-positive rate in 2024, and independent researchers confirmed strong performance. In July 2026, Epoch AI, a San Francisco research firm, found zero false positives across 495 human-written texts for Pangram. GPTZero, having adopted a similar machine-learning approach, also scored zero false positives on the same benchmark.

Real-World Adoption Is Accelerating

The tools are moving well beyond peer review. According to a January study using Pangram, one in eight biomedical articles last year contained some AI-generated text. In June, the NeurIPS computer-science conference rejected 18% of submissions after screening with Pangram. arXiv preprints can now be checked via a mirror site called alphaXiv, the University of Chicago is vetting coursework with the tool, and Substack integrated Pangram across its platform in July, letting readers see AI-detection scores on posts.

GPTZero, meanwhile, is used by five computer-science conferences and three universities, with more piloting the tool.

The Trade-Off: Fewer False Positives, More False Negatives

Accuracy in flagging pure human writing comes at a cost. To avoid mislabeling human text, both firms tolerate higher false-negative rates — AI text that slips through as “human.” Epoch AI found that while both tools caught nearly all AI text generated with basic prompts, about 8% of passages passed as human when AI was asked to mimic a specific author’s style.

Karpinska’s research also tested “humanizer” tools that rewrite AI text to erase detectable signatures, finding that false-negative and false-positive rates both rose under this condition. Firms argue such tests age quickly: Pangram’s newest model, Pangram 4, released in July 2026, claims a false-negative rate of just 0.34% overall, rising to 2.9% against style-imitation attacks — though performance still drops for AI passages under 50 words.

The Real Challenge: Mixed Human-AI Writing

The sharpest limitation isn’t detecting fully AI-generated slop — it’s judging text where human and AI contributions intertwine, which is now the norm for many writers. Both Pangram and GPTZero segment documents and estimate what proportion is AI-influenced, but the results can be misleading.

A telling case: an essay defending AI-assisted peer review, published on The Scholarly Kitchen, scored 100% AI under Pangram 3.3 and 96% AI under Pangram 4 — despite the author insisting her draft and ideas were “entirely human-created,” with AI used only to polish the text. Pangram’s Spero explains that a 100% score doesn’t mean every word was AI-written; it means every text segment was judged probably-AI, even if that segment contains human content. Pangram 4 improves granularity by analysing chunks as small as 30-40 words, rather than the older 200-300 word segments, aiming to better separate light AI polishing from heavy AI rewriting.

Similar issues surfaced when Nature examined articles from Nature India and Nature Africa: some flagged as “100% AI” were human-written pieces that used AI only for transcription, translation or copy-editing, with multiple rounds of human review afterward — a use permitted under Nature Portfolio’s AI guidelines.

Bias Against Non-Native English Writers

A recurring concern is that translation and light editing by non-native English speakers gets misread as full AI generation. Evanko reports that AACR reviews translated from non-Indo-European languages are disproportionately flagged as “100% AI” by older Pangram versions — though early Pangram 4 trials have reduced this misclassification for about one-fifth of affected reviews. Spero maintains that translation alone shouldn’t trigger a flag, attributing the issue to simultaneous AI editing during translation.

A Cat-and-Mouse Game

The tools’ rising visibility hasn’t been welcomed universally. When Substack began surfacing Pangram scores, some authors objected. AI-literacy researcher Sam Illingworth called it a “witch hunt” after his heavily AI-edited post scored 100% AI — a score that dropped to 100% human once he ran the same text through a humanizer tool. Pangram 4 later rated his original post 95% AI and the humanized version 60% AI, suggesting improved (but imperfect) resistance to humanizing tricks. As Nikhil Garg, a computer scientist at Cornell Tech, puts it: “This cat-and-mouse game on both sides is going to continue.”

What Detection Can and Cannot Prove

Experts converge on one point: even a near-perfect AI detector cannot resolve the deeper question of what counts as ethically acceptable AI use. As Renée DiResta, a researcher at Georgetown University, writes, “We end up surfacing what is easiest to detect rather than addressing the deeper underlying concern.”

For research publishers, the practical takeaway is that detection scores should function as an investigative signal, not a verdict. GPTZero’s Alex Cui argues that a pattern of heavy AI use across a body of work is far more revealing than any single flagged article. Meanwhile, adoption is spreading: Proofig AI and Clearskies now offer Pangram to clients, Science journals have adopted iThenticate’s AI-detection feature, MDPI has built an in-house detector called Binoculars, and Springer Nature is evaluating both in-house and third-party options.

With Anthropic also introducing watermarking into its Claude models to comply with the EU AI Act, the direction is clear: as detection and watermarking tools mature, researchers and institutions will face growing pressure to disclose AI use transparently — and to define, collectively, where assistance ends and authorship begins.

Source: Adapted from Nature news, doi: https://doi.org/10.1038/d41586-026-02569-3

References:

1. She, R. Preprint at bioRxiv https://doi.org/10.64898/2026.01.01.697311 (2026).

2. Emi, B. & Spero, M. Preprint at arXiv https://doi.org/10.48550/arXiv.2402.14873 (2024).

3. Russel, J., Karpinska, M. & Iyyer, M. In Proc. 63rd Annu. Meet. Assoc. Comput. Linguist. (Vol. 1: Long Papers), 5342–5373 (Association for Computational Linguistics, 2025).

4. Glickenhaus, B. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2607.27183 (2026).

5. Alexandru Adam, G. et al. Preprint at arXiv https://doi.org/10.48550/arXiv.2602.13042 (2026).