On the creation of a clinical gold standard corpus in Spanish: Mining adverse drug reactions

• Published in: Journal of biomedical informatics Vol. 56; pp. 318 - 332
• Main Authors: Oronoz, Maite, Gojenola, Koldo, Pérez, Alicia, de Ilarraza, Arantza Díaz, Casillas, Arantza
• Format: Journal Article
• Language: English
• Published: United States Elsevier Inc 01-08-2015
• Subjects: Adverse drug reaction; Adverse Drug Reaction Reporting Systems; Algorithms; Annotations; Automation; Clinical text; Data Mining - methods; Diseases; Drug-Related Side Effects and Adverse Reactions; Drugs; Electronic Health Records - standards; Ethics; Gold standard; Language; Linguistics; Machine Learning; Medical; Natural Language Processing; Pharmaceutical Preparations; Pharmacovigilance; Predictive Value of Tests; Reproducibility of Results; Text mining; Translating; Text mining; Gold standard; Adverse drug reaction; Clinical text
• ISSN: 1532-0464; 1532-0480
• DOI: 10.1016/j.jbi.2015.06.016

[Display omitted]
•Creation of a gold standard of electronic health records in Spanish.•Annotation of diseases, drugs and adverse drug reaction (ADR) events.•Quality: inter-annotator agreement of 90.53% for entities and 82.86% for events.•Development and assessment of linguistic analyzer for medical texts.•Automatic ADR extraction with machine learning shows the potential of the corpus.
The advances achieved in Natural Language Processing make it possible to automatically mine information from electronically created documents. Many Natural Language Processing methods that extract information from texts make use of annotated corpora, but these are scarce in the clinical domain due to legal and ethical issues. In this paper we present the creation of the IxaMed-GS gold standard composed of real electronic health records written in Spanish and manually annotated by experts in pharmacology and pharmacovigilance. The experts mainly annotated entities related to diseases and drugs, but also relationships between entities indicating adverse drug reaction events. To help the experts in the annotation task, we adapted a general corpus linguistic analyzer to the medical domain. The quality of the annotation process in the IxaMed-GS corpus has been assessed by measuring the inter-annotator agreement, which was 90.53% for entities and 82.86% for events. In addition, the corpus has been used for the automatic extraction of adverse drug reaction events using machine learning.