Utility of Artificial Intelligence in Orthopedic Surgery Literature Review: A Comparative Pilot Study

Utility of Artificial Intelligence in Orthopedic Surgery Literature Review: A Comparative Pilot Study

Literature reviews are essential to the scientific process and allow clinician researchers to advance general knowledge. The purpose of this study was to evaluate if the artificial intelligence (AI) programs ChatGPT and Perplexity.AI can perform an orthopedic surgery literature review. Five differen...

Full description

Saved in:
Bibliographic Details
Published in:Orthopedics (Thorofare, N.J.) Vol. 47; no. 3; pp. e125 - e130
Main Authors: Sanii, Ryan Y, Kasto, Johnny K, Wines, Wade B, Mahylis, Jared M, Muh, Stephanie J
Format: Journal Article
Language:English
Published: United States Slack, Inc 01-05-2024
SLACK INCORPORATED
Subjects:
Artificial Intelligence
Automation
Bone surgery
Chatbots
Computational linguistics
Humans
Joint replacement surgery
Language processing
Literature reviews
Natural language interfaces
Orthopedic Procedures
Orthopedic surgery
Orthopedics
Patient satisfaction
Pilot Projects
Review Literature as Topic
Search strategies
Shoulder
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:Literature reviews are essential to the scientific process and allow clinician researchers to advance general knowledge. The purpose of this study was to evaluate if the artificial intelligence (AI) programs ChatGPT and Perplexity.AI can perform an orthopedic surgery literature review. Five different search topics of varying specificity within orthopedic surgery were chosen for each search arm to investigate. A consolidated list of unique articles for each search topic was recorded for the experimental AI search arms and compared with the results of the control arm of two independent reviewers. Articles in the experimental arms were examined by the two independent reviewers for relevancy and validity. ChatGPT was able to identify a total of 61 unique articles. Four articles were not relevant to the search topic and 51 articles were deemed to be fraudulent, resulting in 6 valid articles. Perplexity.AI was able to identify a total of 43 unique articles. Nineteen were not relevant to the search topic but all articles were able to be verified, resulting in 24 valid articles. The control arm was able to identify 132 articles. Success rates for ChatGPT and Perplexity. AI were 4.6% (6 of 132) and 18.2% (24 of 132), respectively. The current iteration of ChatGPT cannot perform a reliable literature review, and Perplexity.AI is only able to perform a limited review of the medical literature. Any utilization of these open AI programs should be done with caution and human quality assurance to promote responsible use and avoid the risk of using fabricated search results. [ . 2024;47(3):e125-e130.].
Bibliography:ObjectType-Article-2
SourceType-Scholarly Journals-1
ObjectType-Feature-1
content type line 23
ISSN:0147-7447
1938-2367
DOI:10.3928/01477447-20231220-02