Performance Evaluation of Various Storage Formats for Clinical Data Repositories

• Published in: IEEE transactions on instrumentation and measurement Vol. 60; no. 10; pp. 3244 - 3252
• Main Authors: Gilchrist, Jeff, Frize, Monique, Ennett, Colleen M., Bariciak, Erika
• Format: Journal Article
• Language: English
• Published: New York IEEE 01-10-2011; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Biomedical imaging; Biomedical monitoring; Blood; Data acquisition; Data points; data storage systems; decision support systems; Format; Instrumentation; Laboratories; medical information systems; Monitoring; Pediatrics; Performance evaluation; Product introduction; Queries; Real time; Repositories
• ISSN: 0018-9456; 1557-9662
• DOI: 10.1109/TIM.2011.2122850

Fast access to clinical data is necessary when performing real-time predictions of medical events. A clinical data repository (CDR) therefore requires an efficient format for storing data so it can meet the access demands of prediction algorithms for clinical decision support. We have developed a new hybrid entity-attribute-value (EAV) storage format for CDRs that is compared with the common simple and multidata-type EAV formats. Their performance is evaluated with regard to querying millions of data points from clinical sources to assess the amount of storage space the data use, the speed with which the data can be obtained, and the complexity of the queries required to retrieve the data. Performance results showed that our hybrid EAV approach provides a nice balance of the simple and multidata-type formats that scales well as the amount of data in the CDR increases. The hybrid format is faster, uses less complex queries, and allows the data to remain in its original data type while using the same amount of storage space as the simple and multidata-type formats.