ChatGPT for assessment writing

• Published in: Medical teacher Vol. 45; no. 11; pp. 1224 - 1227
• Main Authors: Zuckerman, Matthew, Flood, Ryan, Tan, Rachael J B, Kelp, Nicole, Ecker, David J, Menke, Jonathan, Lockspeiser, Tai
• Format: Journal Article
• Language: English
• Published: England Taylor & Francis Ltd 01-11-2023
• Subjects: Chatbots; Drafts; Educational Status; Evaluation; Faculty; Humans; Medical schools; Schools, Medical; Time use; Videotape Recording; Writing; AI; medical education; ChatGPT; artificial intelligence; Assessment
• ISSN: 0142-159X; 1466-187X
• DOI: 10.1080/0142159X.2023.2249239

Medical schools invest significant resources into the creation of multiple-choice items for assessments. This process is costly and requires faculty training. Recently ChatGPT has been used in various areas to improve content creation efficiency, and it has otherwise been used to answer USMLE-style assessment items.
We proposed the use of ChatGPT to create initial drafts of multiple-choice items.
The use of ChatGPT to generate assessment items can decrease resources required, allowing for the creation of more items, and freeing-up faculty time to perform higher level assessment activities. ChatGPT is also able to consistently produce items using a standard format while adhering to item writing guidelines, which can be very challenging for faculty teams.
We plan to pilot ChatGPT drafted questions and compare item statistics for those written by ChatGPT with those written by our content experts. We also plan to further identify the types of questions that ChatGPT is most appropriate for, and incorporate media into assessment items (e.g. images, videos).