Dataset Information

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

ABSTRACT:

Objectives

Hospitalisations for serious infections are common among middle age and older adults and frequently used as study outcomes. Yet, few studies have evaluated the performance of diagnosis codes to identify serious infections in this population. We sought to determine the positive predictive value (PPV) of diagnosis codes for identifying hospitalisations due to serious infections among middle age and older adults.

Setting and participants

We identified hospitalisations for possible infection among adults >=50 years enrolled in the Tennessee Medicaid healthcare programme (2008-2012) using International Classifications of Diseases, Ninth Revision diagnosis codes for pneumonia, meningitis/encephalitis, bacteraemia/sepsis, cellulitis/soft-tissue infections, endocarditis, pyelonephritis and septic arthritis/osteomyelitis.

Design

Medical records were systematically obtained from hospitals randomly selected from a stratified sampling framework based on geographical region and hospital discharge volume.

Measures

Two trained clinical reviewers used a standardised extraction form to abstract information from medical records. Predefined algorithms served as reference to adjudicate confirmed infection-specific hospitalisations. We calculated the PPV of diagnosis codes using confirmed hospitalisations as reference. Sensitivity analyses determined the robustness of the PPV to definitions that required radiological or microbiological confirmation. We also determined inter-rater reliability between reviewers.

Results

The PPV of diagnosis codes for hospitalisations for infection (n=716) was 90.2% (95% CI 87.8% to 92.2%). The PPV was highest for pneumonia (96.5% (95% CI 93.9% to 98.0%)) and cellulitis (91.1% (95% CI 84.7% to 94.9%)), and lowest for meningitis/encephalitis (50.0% (95% CI 23.7% to 76.3%)). The adjudication reliability was excellent (92.7% agreement; first agreement coefficient: 0.91). The overall PPV was lower when requiring microbiological confirmation (45%) and when requiring radiological confirmation for pneumonia (79%).

Conclusions

Discharge diagnosis codes have a high PPV for identifying hospitalisations for common, serious infections among middle age and older adults. PPV estimates for rare infections were imprecise.

SUBMITTER: Wiese AD

PROVIDER: S-EPMC6009457 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

Wiese Andrew D AD Griffin Marie R MR Stein C Michael CM Schaffner William W Greevy Robert A RA Mitchel Edward F EF Grijalva Carlos G CG

BMJ open 20180619 6

<h4>Objectives</h4>Hospitalisations for serious infections are common among middle age and older adults and frequently used as study outcomes. Yet, few studies have evaluated the performance of diagnosis codes to identify serious infections in this population. We sought to determine the positive predictive value (PPV) of diagnosis codes for identifying hospitalisations due to serious infections among middle age and older adults.<h4>Setting and participants</h4>We identified hospitalisations for ...[more]

PMID: 29921683

Similar Datasets

Project description:BackgroundThe use of real-world data to generate evidence requires careful assessment and validation of critical variables before drawing clinical conclusions. Prospective clinical trial data suggest that anatomic origin of colon cancer impacts prognosis and treatment effectiveness. As an initial step in validating this observation in routine clinical settings, we explored the feasibility and accuracy of obtaining information on tumor sidedness from electronic health records (EHR) billing codes.MethodsNine thousand four hundred three patients with metastatic colorectal cancer (mCRC) were selected from the Flatiron Health database, which is derived from de-identified EHR data. This study included a random sample of 200 mCRC patients. Tumor site data derived from International Classification of Diseases (ICD) codes were compared with data abstracted from unstructured documents in the EHR (e.g. surgical and pathology notes). Concordance was determined via observed agreement and Cohen's kappa coefficient (?). Accuracy of ICD codes for each tumor site (left, right, transverse) was determined by calculating the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and corresponding 95% confidence intervals, using abstracted data as the gold standard.ResultsStudy patients had similar characteristics and side of colon distribution compared with the full mCRC dataset. The observed agreement between the ICD codes and abstracted data for tumor site for all sampled patients was 0.58 (??=?0.41). When restricting to the 62% of patients with a side-specific ICD code, the observed agreement was 0.84 (??=?0.79). The specificity (92-98%) of structured data for tumor location was high, with lower sensitivity (49-63%), PPV (64-92%) and NPV (72-97%). Demographic and clinical characteristics were similar between patients with specific and non-specific side of colon ICD codes.ConclusionsICD codes are a highly reliable indicator of tumor location when the specific location code is entered in the EHR. However, non-specific side of colon ICD codes are present for a sizable minority of patients, and structured data alone may not be adequate to support testing of some research hypotheses. Careful assessment of key variables is required before determining the need for clinical abstraction to supplement structured data in generating real-world evidence from EHRs.

Project description:BackgroundAutomated disease code classification using free-text medical information is important for public health surveillance. However, traditional natural language processing (NLP) pipelines are limited, so we propose a method combining word embedding with a convolutional neural network (CNN).ObjectiveOur objective was to compare the performance of traditional pipelines (NLP plus supervised machine learning models) with that of word embedding combined with a CNN in conducting a classification task identifying International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes in discharge notes.MethodsWe used 2 classification methods: (1) extracting from discharge notes some features (terms, n-gram phrases, and SNOMED CT categories) that we used to train a set of supervised machine learning models (support vector machine, random forests, and gradient boosting machine), and (2) building a feature matrix, by a pretrained word embedding model, that we used to train a CNN. We used these methods to identify the chapter-level ICD-10-CM diagnosis codes in a set of discharge notes. We conducted the evaluation using 103,390 discharge notes covering patients hospitalized from June 1, 2015 to January 31, 2017 in the Tri-Service General Hospital in Taipei, Taiwan. We used the receiver operating characteristic curve as an evaluation measure, and calculated the area under the curve (AUC) and F-measure as the global measure of effectiveness.ResultsIn 5-fold cross-validation tests, our method had a higher testing accuracy (mean AUC 0.9696; mean F-measure 0.9086) than traditional NLP-based approaches (mean AUC range 0.8183-0.9571; mean F-measure range 0.5050-0.8739). A real-world simulation that split the training sample and the testing sample by date verified this result (mean AUC 0.9645; mean F-measure 0.9003 using the proposed method). Further analysis showed that the convolutional layers of the CNN effectively identified a large number of keywords and automatically extracted enough concepts to predict the diagnosis codes.ConclusionsWord embedding combined with a CNN showed outstanding performance compared with traditional methods, needing very little data preprocessing. This shows that future studies will not be limited by incomplete dictionaries. A large amount of unstructured information from free-text medical writing will be extracted by automated approaches in the future, and we believe that the health care field is about to enter the age of big data.

Dataset Information

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

Objectives

Setting and participants

Design

Measures

Results

Conclusions

Publications

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets