Natural language processing to extract follow-up provider information from hospital discharge summaries

• Published in: AMIA ... Annual Symposium proceedings Vol. 2010; pp. 872 - 876
• Main Authors: Were, Martin C, Gorbachev, Sergey, Cadwallader, Jason, Kesterson, Joe, Li, Xiaochun, Overhage, J Marc, Friedlin, Jeff
• Format: Journal Article
• Language: English
• Published: United States American Medical Informatics Association 13-11-2010
• Subjects: Electronic Health Records; Follow-Up Studies; Humans; Natural Language Processing; Patient Discharge
• ISSN: 1559-4076

We evaluate the performance of a Natural Language Processing (NLP) application designed to extract follow-up provider information from free-text discharge summaries at two hospitals.
We compare performance by the NLP application, called the Regenstrief EXtracion tool (REX), to performance by three physician reviewers at extracting follow-up provider names, phone/fax numbers and location information. Precision, recall, and F-measures are reported, with 95% CI for pairwise comparisons.
Of 556 summaries with follow-up information, REX performed as follows in precision, recall, F-measure respectively: Provider Name 0.96, 0.92, 0.94; Phone/Fax 0.99, 0.92, 0.96; Location 0.83, 0.82, 0.82. REX was as good as all physician-reviewers in identifying follow-up provider names and phone/fax numbers, and slightly inferior to two physicians at identifying location information. REX took about four seconds (vs. 3-5 minutes for physician-reviewers) to extract follow-up information.
A NLP program had physician-like performance at extracting provider follow-up information from discharge summaries.