Creating synthetic patient data to support the design and evaluation of novel health information technology

• Published in: Journal of biomedical informatics Vol. 95; p. 103201
• Main Authors: Pollack, Ari H., Simon, Tamara D., Snyder, Jaime, Pratt, Wanda
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01-07-2019
• Subjects: Adolescent; Child; Child, Preschool; Cognitive burden; Computer Simulation; Electronic Health Records; Female; Health information technology; Human-centered design; Humans; Male; Medical Informatics - education; Medical Informatics - methods; Software; System evaluation; Human-centered design; System evaluation; HIT; Cognitive burden; EHR; Health information technology
• ISSN: 1532-0464; 1532-0480
• DOI: 10.1016/j.jbi.2019.103201

[Display omitted]
•User-centered design can significantly improve health information technology.•Need realistic clinical scenarios to effectively evaluate early stage prototypes.•Processes to generate synthetic testing data require rigorous methods.•Synthetic data needs specific structure to support cognitive evaluations.•User-centered evaluations should occur throughout the design process.
To ensure that new health information technology supports its intended users, researchers and developers need to follow human-centered methods during all stages of the software development lifecycle, including early stage evaluations. These evaluations need to include realistic testing scenarios to ensure that they provide valuable and accurate feedback to system developers. However, obtaining realistic patient data to support these evaluations has many challenges, including the risk of re-identifying anonymized patients as well as the costs associated with connecting test systems with production ready clinical databases. Here we present a novel five-step process to create highly structured and realistic synthetic patient data to support the evaluation and comparison of early to middle stage health information technology prototypes. We applied this method to evaluate and compare three novel health information technology prototypes designed to support clinicians during the identification of high-priority patients when answering the question: “What patient should I see first?” Our novel approach fills an important gap in the evaluation of health information technology and assists designers in creating high-quality software that best supports its end users.