A study of 14 million research papers uncovers a sudden and dramatic shift that occurred shortly after the introduction of ChatGPT. The language of science is constantly evolving. Over the past decade, numerous words and phrases have transitioned from obscurity to common usage within the scientific community, including terms like Zika, Ebola, and ChatGPT. These changes reflect the shifting focus of scientific research as well as broader societal and scientific trends.

These linguistic shifts are evident in the papers, reviews, and articles that scientists continuously produce. Researchers have even attempted to map the evolution of science by tracking changes in the language used in scientific literature. This raises an intriguing question about the impact of artificial intelligence on science. Since the public release of ChatGPT in November 2022, scientists have had the capability to use Large Language Models to revise, edit, and sometimes even write scientific papers from scratch. However, the extent to which scientists are actually utilizing this AI assistance remains unclear.

Figure 1. How ChatGPT Has Revolutionized Scientific Paper Writing

Historic Transformation

Enter Dmitry Kobak and his team at the Hertie Institute for AI in Brain Health in Tübingen, Germany. They have developed a method to measure the impact of AI systems on scientific literature since 2022 and compare it to other significant events in science. They assert that Large Language Models are reshaping scientific discourse on an unprecedented scale. Figure 1 shows How ChatGPT Has Revolutionized Scientific Paper Writing.

Kobak and his colleagues began by downloading the abstracts from over 14 million scientific papers published on the PubMed biomedical database since 2010. They then removed common words and phrases unrelated to the authors' content, such as “copyright” or “How to cite this article.” Next, they calculated the annual frequency of each word longer than three letters and examined how the frequency of the 800 most popular words changed each year.

The results quickly highlighted some clear trends in scientific literature. For instance, the frequency of the word "Ebola" peaked in 2015, and "Zika" in 2017. One of the most significant shifts occurred in 2020, with a substantial rise in terms like "lockdown," "pandemic," "respiratory," and "remdesivir" during the COVID-19 outbreak, an event recognized for its profound impact on scientific publishing.

To the researchers' surprise, an even more significant change emerged in 2024, with an increase in words such as "delves," "crucial," "important," and "potential." Interestingly, these words are more related to writing style rather than the scientific content of the papers.

Indeed, the researchers propose that these words are precisely the types favored by Large Language Models. “The unprecedented surge in stylistic words in 2024 enables us to use them as indicators of ChatGPT usage,” Kobak and his team assert. The change has been significant. “Hundreds of words have suddenly increased in frequency following the availability of ChatGPT,” they note.

Language Assistance

Kobak and his team established a minimum estimate for the number of papers influenced by Large Language Models. The data indicates that at least 10 percent of the papers on PubMed in 2024 were impacted in this manner. “With approximately 1.5 million papers currently indexed in PubMed each year, this suggests that LLMs assist in writing at least 150 thousand papers annually,” the researchers conclude.

The team noted that AI assistance was more prevalent in papers from countries where English is not the primary language. This could imply that non-English speakers are using AI assistance to level the playing field in scientific writing. Alternatively, it might suggest that English speakers also use AI assistance but are more skilled at disguising its influence in their papers before publication. Regardless, the use of LLMs appears to be widespread.

This intriguing study sheds rare light on how AI is transforming not only the way scientists write but also the nature of scientific research itself. “The impact of LLM usage on scientific writing is truly unprecedented, even surpassing the significant vocabulary changes brought about by the COVID-19 pandemic,” Kobak and his team observe.

What’s needed now is a clearer understanding and acknowledgment of these trends so that the scientific community can establish guidelines for LLM use that serve the best interests of scientists, scientific publishers, and the broader society that science aims to benefit. This work represents an important step in that direction. However, the rapid pace of change in LLM usage indicates that scientists and publishers must act swiftly to keep up. If the trends in scientific publishing are any indication, other areas of publishing are likely facing similar challenges.

Source: DISCOVER

Cite this article:

Janani R (2024), How ChatGPT Has Transformed Scientific Paper Writing,AnaTechmaz, pp. 170