Dataset Information

Developing and evaluating an automated appendicitis risk stratification algorithm for pediatric patients in the emergency department.

ABSTRACT:

Objective

To evaluate a proposed natural language processing (NLP) and machine-learning based automated method to risk stratify abdominal pain patients by analyzing the content of the electronic health record (EHR).

Methods

We analyzed the EHRs of a random sample of 2100 pediatric emergency department (ED) patients with abdominal pain, including all with a final diagnosis of appendicitis. We developed an automated system to extract relevant elements from ED physician notes and lab values and to automatically assign a risk category for acute appendicitis (high, equivocal, or low), based on the Pediatric Appendicitis Score. We evaluated the performance of the system against a manually created gold standard (chart reviews by ED physicians) for recall, specificity, and precision.

Results

The system achieved an average F-measure of 0.867 (0.869 recall and 0.863 precision) for risk classification, which was comparable to physician experts. Recall/precision were 0.897/0.952 in the low-risk category, 0.855/0.886 in the high-risk category, and 0.854/0.766 in the equivocal-risk category. The information that the system required as input to achieve high F-measure was available within the first 4 h of the ED visit.

Conclusions

Automated appendicitis risk categorization based on EHR content, including information from clinical notes, shows comparable performance to physician chart reviewers as measured by their inter-annotator agreement and represents a promising new approach for computerized decision support to promote application of evidence-based medicine at the point of care.

SUBMITTER: Deleger L

PROVIDER: S-EPMC3861926 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BACKGROUND: While Canadian ED physicians discharge most syncope patients with no specific further follow-up, approximately 5% will suffer serious outcomes after ED discharge. The goal of this study is to prospectively identify risk factors and to derive a clinical decision tool to accurately predict those at risk for serious outcomes after ED discharge within 30 days. METHODS/DESIGN: We will conduct a prospective cohort study at 6 Canadian EDs to include adults with syncope and exclude patients with loss of consciousness>5 minutes, mental status changes from baseline, obvious witnessed seizure, or head trauma prior to syncope. Emergency physicians will collect standardized clinical variables including historical features, physical findings, and results of immediately available tests (blood, ECG, and ED cardiac monitoring) prior to ED discharge/hospital admission. A second emergency physician will evaluate approximately 10% of study patients for interobserver agreement calculation of predictor variables. The primary outcome will be a composite serious outcome occurring within 30 days of ED discharge and includes three distinct categories: serious adverse events (death, arrhythmia); identification of serious underlying disease (structural heart disease, aortic dissection, pulmonary embolism, severe pulmonary hypertension, subarachnoid hemorrhage, significant hemorrhage, myocardial infarction); or procedures to treat the cause of syncope. The secondary outcome will be any of the above serious outcomes either suspected or those occurring in the ED. A blinded Adjudication Committee will confirm all serious outcomes. Univariate analysis will be performed to compare the predictor variables in patients with and without primary outcome. Variables with p-values <0.2 and kappa values ? 0.60 will be selected for stepwise logistic regression to identify the risk factors and to develop the clinical decision tool. We will enroll 5,000 patients (with 125 positive for primary outcome) for robust identification of risk factors and clinical decision tool development. DISCUSSION: Once successfully developed, this tool will accurately risk-stratify adult syncope patients; however, validation and implementation will still be required. This program of research should lead to standardized care of syncope patients, and improve patient safety.