ChatGPT in Occupational Medicine: A Comparative Study with Human Experts

ChatGPT in Occupational Medicine: A Comparative Study with Human Experts

The objective of this study is to evaluate ChatGPT's accuracy and reliability in answering complex medical questions related to occupational health and explore the implications and limitations of AI in occupational health medicine. The study also provides recommendations for future research in...

Full description

Saved in:
Bibliographic Details
Published in:Bioengineering (Basel) Vol. 11; no. 1; p. 57
Main Authors: Padovan, Martina, Cosci, Bianca, Petillo, Armando, Nerli, Gianluca, Porciatti, Francesco, Scarinci, Sergio, Carlucci, Francesco, Dell'Amico, Letizia, Meliani, Niccolò, Necciari, Gabriele, Lucisano, Vincenzo Carmelo, Marino, Riccardo, Foddis, Rudy, Palla, Alessandro
Format: Journal Article
Language:English
Published: Switzerland MDPI AG 01-01-2024
Subjects:
Accuracy
Application programming interface
Artificial intelligence
Chatbots
ChatGPT
Comparative studies
Decision making
digital health
health promotion
large language model
Legislation
Medical personnel
Medical screening
Medicine
Medicine, Industrial
Natural language
Occupational health
occupational health and safety
Physicians
Questions
R&D
Research & development
Semantics
Teams
Technology application
Usability
United States
Italy
health promotion
digital health
large language model
occupational health and safety
ChatGPT
artificial intelligence
Online Access:Get full text
Tags: Add Tag
No Tags, Be the first to tag this record!
Description
Summary:The objective of this study is to evaluate ChatGPT's accuracy and reliability in answering complex medical questions related to occupational health and explore the implications and limitations of AI in occupational health medicine. The study also provides recommendations for future research in this area and informs decision-makers about AI's impact on healthcare. A group of physicians was enlisted to create a dataset of questions and answers on Italian occupational medicine legislation. The physicians were divided into two teams, and each team member was assigned a different subject area. ChatGPT was used to generate answers for each question, with/without legislative context. The two teams then evaluated human and AI-generated answers blind, with each group reviewing the other group's work. Occupational physicians outperformed ChatGPT in generating accurate questions on a 5-point Likert score, while the answers provided by ChatGPT with access to legislative texts were comparable to those of professional doctors. Still, we found that users tend to prefer answers generated by humans, indicating that while ChatGPT is useful, users still value the opinions of occupational medicine professionals.
Bibliography:ObjectType-Article-1
SourceType-Scholarly Journals-1
ObjectType-Feature-2
content type line 23
ISSN:2306-5354
2306-5354
DOI:10.3390/bioengineering11010057