Project IMPACT: Results from a Pilot Validity Study of a New Observational Database

• Published in: Critical care medicine Vol. 30; no. 12; pp. 2765 - 2770
• Main Authors: Cook, Suzanne F, Visscher, Wendy A, Hobbs, Connie L, Williams, Rick L
• Format: Journal Article
• Language: English
• Published: Hagerstown, MD by the Society of Critical Care Medicine and Lippincott Williams & Wilkins 01-12-2002; Lippincott
• Subjects: Biological and medical sciences; Computerized management (medical records, files, hospital management); Computerized, statistical medical data processing and models in biomedicine; Database Management Systems; Databases, Factual; Decision Support Systems, Clinical; Humans; Intensive Care Units - standards; Medical Records Systems, Computerized; Medical sciences; Quality Assurance, Health Care; Reproducibility of Results; United States; United States; Human; Intensive care; critical care; data abstraction; Hospitalization; hospital records; patient outcomes; Intensive care unit; Validity; Database; Pilot test; Project IMPACT; Collection; Medical record; validity study; medical records; Computer aid
• ISSN: 0090-3493; 1530-0293
• DOI: 10.1097/00003246-200212000-00024

OBJECTIVEThe objective of this study was to evaluate the accuracy of the information contained in the Project IMPACT database. Project IMPACT is a comprehensive database system developed to measure and describe the care of intensive care patients. This database is being used by a large group of hospitals to help clinicians improve the care of these patients. Data on patient demographics, diagnoses, treatment, and outcomes are entered into the Project IMPACT database by staff at participating hospitals. This pilot study was a first step in assessing the accuracy of these data to determine the usefulness of the Project IMPACT database for measuring intensive care unit (ICU) performance and patient outcomes.
DESIGNThe design of the pilot study was the independent abstraction of selected data items from a random sample of ICU patient records from two hospitals participating in Project IMPACT. The abstracted data were compared with the data existing in the Project IMPACT database for agreement.
SETTINGAbstraction was performed onsite at the two pilot hospitals by a trained abstractor who was not affiliated with either hospital.
PATIENTSPatients whose records were abstracted included 45 randomly selected ICU patients at each of the two pilot hospitals.
MEASUREMENTS AND MAIN RESULTSComparison of the Project IMPACT data with the independently abstracted data indicated good agreement (80% or above) on discrete items, such as type of ICU patient. Poorer agreement (under 80%) was seen for continuous items (e.g., 24-hr urine output) and coded items requiring judgment (e.g., reason for ICU admission).
CONCLUSIONSThe pilot study showed good internal validity for most of the abstracted variables. High agreement rates were observed, regardless of method of original data capture (electronic download or manual entry), although agreement was higher for some data items that had been electronically downloaded into the Project IMPACT database. The results suggest that Project IMPACT is a valuable resource for ICUs to collect and evaluate information about treatment and patient outcomes.