Large language models, or LLMs, are powerful artificial intelligence systems like ChatGPT that can generate and revise text with remarkable, human-like fluency. While these tools offer clear benefits, they also have significant limitations, including the potential to produce factually incorrect information or reinforce existing societal biases. This has sparked a debate in the scientific community about the appropriate use of AI in scholarly writing and raised a critical question: Just how many academic papers have been touched by AI?

To investigate this, a team of researchers developed a novel method to detect the influence of LLMs on scientific literature. Instead of trying to identify AI-generated text directly, which can be unreliable, they analyzed vocabulary trends over time. The study examined more than 15 million biomedical abstracts — the concise summaries of research papers — indexed in the public PubMed database between 2010 and 2024. By establishing a baseline of typical word usage before LLMs became widely available, the researchers could pinpoint any sudden, unusual shifts in writing style that occurred afterward.

The analysis uncovered a distinct and abrupt change in the language of scientific abstracts following the release of popular LLMs. The researchers found a significant increase in the use of certain “stylistic” or “flowery” words that are characteristic of LLM-generated text, such as “pivotal,” “showcasing,” and “meticulous.” Based on this excess word analysis, the team calculated that at least 13.5% of biomedical abstracts published in 2024 were likely written with assistance from an LLM. This figure represents a conservative lower estimate and varies significantly, climbing as high as 40% in specific scientific disciplines and journals.

These findings, published in Science Advances, suggest that LLMs have had an unprecedented impact on the style of scientific writing, one that surpasses even the linguistic shifts caused by major global events like the COVID-19 pandemic. The study highlights the rapid and widespread adoption of AI by scientists, a trend that is quietly reshaping how research is communicated. As AI tools become further integrated into the scientific process, this research underscores the growing importance of transparency and ethical guidelines to ensure the integrity of the academic record.

References

• Blue, C. & Phys.org. (2025, July 6). Massive study detects AI fingerprints in millions of scientific papers. Phys.Org; Phys.org. https://phys.org/news/2025-07-massive-ai-fingerprints-millions-scientific.html

• Kobak, D., González-Márquez, R., Horvát, E.-Á., & Lause, J. (2025). Delving into LLM-assisted writing in biomedical publications through excess vocabulary. Science Advances, 11(27), eadt3813. https://doi.org/10.1126/sciadv.adt3813