The Utility of AI in Writing a Scientific Review Article on the Impacts of COVID-19 on Musculoskeletal Health

• Published in: Current osteoporosis reports Vol. 22; no. 1; pp. 146 - 151
• Main Authors: Awosanya, Olatundun D., Harris, Alexander, Creecy, Amy, Qiao, Xian, Toepp, Angela J., McCune, Thomas, Kacena, Melissa A., Ozanne, Marie V.
• Format: Journal Article
• Language: English
• Published: New York Springer US 01-02-2024
• Subjects: Epidemiology; Medicine; Medicine & Public Health; Orthopedics; Topical Collection on AI Uses in Musculoskeletal Research and Review; COVID-19; Osteoporosis; SARS-CoV-2; Query; ChatGPT; AI; Scientific review article; Bone Loss; Fracture
• ISSN: 1544-1873; 1544-2241
• DOI: 10.1007/s11914-023-00855-x

Purpose of Review
There were two primary purposes to our reviews. First, to provide an update to the scientific community about the impacts of COVID-19 on musculoskeletal health. Second, was to determine the value of using a large language model, ChatGPT 4.0, in the process of writing a scientific review article. To accomplish these objectives, we originally set out to write three review articles on the topic using different methods to produce the initial drafts of the review articles. The first review article was written in the traditional manner by humans, the second was to be written exclusively using ChatGPT (AI-only or AIO), and the third approach was to input the outline and references selected by humans from approach 1 into ChatGPT, using the AI to assist in completing the writing (AI-assisted or AIA). All review articles were extensively fact-checked and edited by all co-authors leading to the final drafts of the manuscripts, which were significantly different from the initial drafts.
Recent Findings
Unfortunately, during this process, it became clear that approach 2 was not feasible for a very recent topic like COVID-19 as at the time, ChatGPT 4.0 had a cutoff date of September 2021 and all articles published after this date had to be provided to ChatGPT, making approaches 2 and 3 virtually identical. Therefore, only two approaches and two review articles were written (human and AI-assisted). Here we found that the human-only approach took less time to complete than the AI-assisted approach. This was largely due to the number of hours required to fact-check and edit the AI-assisted manuscript. Of note, the AI-assisted approach resulted in inaccurate attributions of references (about 20%) and had a higher similarity index suggesting an increased risk of plagiarism.
Summary
The main aim of this project was to determine whether the use of AI could improve the process of writing a scientific review article. Based on our experience, with the current state of technology, it would not be advised to solely use AI to write a scientific review article, especially on a recent topic.