Up to 22% of papers and 9% of the Nature portfolio show LLM modification. These are the papers your evidence reviews are already citing in M&E frameworks.

Liang et al. (Nature Human Behaviour, 2025) analyzed 1,121,912 papers. The signal is clear: AI-assisted text modification is not a niche practice. In computer science, the estimated share reaches 22.5%. Across Nature portfolio journals — among the most selective in the world — it sits at 9.4%.

This is not a future trend. This is the composition of the current literature.

────

For M&E directors, foundation program officers, and impact investors, this raises a question that rarely appears in evidence synthesis protocols: what does AI-modified evidence mean for the validity of what you’re reading?

The short answer is: it depends on what was modified and how.

LLM assistance in editing prose for clarity or grammar does not automatically compromise a finding. But LLM involvement in structuring arguments, framing results, or generating abstract language that obscures methodological limitations — that’s a different matter. And the current literature does not reliably distinguish between these cases.

────

Three concrete implications for evidence-based practice:

Your theory of change rests on a body of literature that is changing faster than your review cycles. A systematic review conducted 18 months ago has a different evidence base than the same review conducted today — not because new RCTs were published, but because the volume, framing, and authorship composition of available papers has shifted.

Transparency about AI use is not yet standard. Most journals implemented disclosure policies in 2023–2024. Compliance is uneven. A paper that doesn’t declare LLM use may have used it — or may not have. That ambiguity is now part of the evidence environment.

The quality signal has moved. Citation counts, journal prestige, and abstract clarity are all being inflated by AI-assisted production at scale. The indicators your teams use to triage evidence need recalibration.

None of this means evidence-based decision-making is broken. It means the protocols for evaluating evidence quality need to catch up with the tools that are changing how evidence is produced.

#MEL #ImpactInvesting #EvidenceSynthesis #DevelopmentResearch