At a Glance
- A major study analyzed over 1.1 million papers to estimate the frequency with which scientists have used large language models for academic writing from 2020 to 2024.
- Researchers found a steady increase in AI-modified content, with the abstracts and introductions of research papers being the most commonly affected sections.
- The use of AI was highest in computer science, where up to 22% of papers showed evidence of LLM modification, a dramatic rise since late 2022.
- In contrast, fields such as mathematics and papers in prestigious Nature portfolio journals showed significantly lower levels of AI adoption, with usage estimated at around 9%.
- Factors such as high publication frequency, shorter paper length, and working in a competitive research area were all correlated with a greater use of AI writing assistance.
Since the public release of advanced artificial intelligence tools, such as ChatGPT, in late 2022, scientists and academics have debated their role in research. A new large-scale study, published in Nature Human Behaviour, provides the first concrete estimate of how widely these large language models, or LLMs, are being used to write scientific papers. By analyzing over 1.1 million research articles, the study confirms that AI assistance is not just a theoretical possibility but a rapidly growing practice across many fields of study.
To measure this trend, researchers developed a novel method that avoids flagging individual papers. Instead, they created a “population-level framework” that tracks subtle, widespread shifts in word choices over time. By comparing word frequencies in papers published before and after ChatGPT’s release, they could estimate the percentage of text likely modified by an LLM. The analysis covered preprints and published papers from January 2020 to September 2024, sourced from the popular archives arXiv and bioRxiv, as well as the prestigious Nature portfolio of journals.
The results show a clear and steady increase in LLM-assisted writing, though the rate of adoption varies significantly by discipline. Computer science, a field closely related to AI development, saw the most dramatic growth, with an estimated 22.5% of paper abstracts showing signs of LLM modification by September 2024. In contrast, fields like mathematics and the highly vetted Nature portfolio journals showed a much lower prevalence, with usage estimated at around 9%. The study also found that AI was most commonly used for writing abstracts and introductions, likely due to the models’ strength in summarizing information.
These findings highlight a significant shift in how science is communicated and raise important questions for the research community. The study noted that authors who publish more frequently and those in more competitive fields were more likely to use AI, suggesting it may be a tool to accelerate publication. As these technologies become more integrated into the scientific process, the study’s authors call for further research and open discussion on how to ensure that AI-assisted science remains transparent, original, and reliable.
References
- Kasal, K. & Phys.org. (2025, August 5). More scientific papers being written with help of ChatGPT—especially in computer science. Phys.Org; Phys.org. https://phys.org/news/2025-08-scientific-papers-written-chatgpt-science.html
- Liang, W., Zhang, Y., Wu, Z., Lepp, H., Ji, W., Zhao, X., Cao, H., Liu, S., He, S., Huang, Z., Yang, D., Potts, C., Manning, C. D., & Zou, J. (2025). Quantifying large language model usage in scientific papers. Nature Human Behaviour. https://doi.org/10.1038/s41562-025-02273-8
