Dataset Information

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

ABSTRACT:

Objectives

Hospitalisations for serious infections are common among middle age and older adults and frequently used as study outcomes. Yet, few studies have evaluated the performance of diagnosis codes to identify serious infections in this population. We sought to determine the positive predictive value (PPV) of diagnosis codes for identifying hospitalisations due to serious infections among middle age and older adults.

Setting and participants

We identified hospitalisations for possible infection among adults >=50 years enrolled in the Tennessee Medicaid healthcare programme (2008-2012) using International Classifications of Diseases, Ninth Revision diagnosis codes for pneumonia, meningitis/encephalitis, bacteraemia/sepsis, cellulitis/soft-tissue infections, endocarditis, pyelonephritis and septic arthritis/osteomyelitis.

Design

Medical records were systematically obtained from hospitals randomly selected from a stratified sampling framework based on geographical region and hospital discharge volume.

Measures

Two trained clinical reviewers used a standardised extraction form to abstract information from medical records. Predefined algorithms served as reference to adjudicate confirmed infection-specific hospitalisations. We calculated the PPV of diagnosis codes using confirmed hospitalisations as reference. Sensitivity analyses determined the robustness of the PPV to definitions that required radiological or microbiological confirmation. We also determined inter-rater reliability between reviewers.

Results

The PPV of diagnosis codes for hospitalisations for infection (n=716) was 90.2% (95% CI 87.8% to 92.2%). The PPV was highest for pneumonia (96.5% (95% CI 93.9% to 98.0%)) and cellulitis (91.1% (95% CI 84.7% to 94.9%)), and lowest for meningitis/encephalitis (50.0% (95% CI 23.7% to 76.3%)). The adjudication reliability was excellent (92.7% agreement; first agreement coefficient: 0.91). The overall PPV was lower when requiring microbiological confirmation (45%) and when requiring radiological confirmation for pneumonia (79%).

Conclusions

Discharge diagnosis codes have a high PPV for identifying hospitalisations for common, serious infections among middle age and older adults. PPV estimates for rare infections were imprecise.

SUBMITTER: Wiese AD

PROVIDER: S-EPMC6009457 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

Wiese Andrew D AD Griffin Marie R MR Stein C Michael CM Schaffner William W Greevy Robert A RA Mitchel Edward F EF Grijalva Carlos G CG

BMJ open 20180619 6

<h4>Objectives</h4>Hospitalisations for serious infections are common among middle age and older adults and frequently used as study outcomes. Yet, few studies have evaluated the performance of diagnosis codes to identify serious infections in this population. We sought to determine the positive predictive value (PPV) of diagnosis codes for identifying hospitalisations due to serious infections among middle age and older adults.<h4>Setting and participants</h4>We identified hospitalisations for ...[more]

PMID: 29921683

Similar Datasets

Project description:BackgroundThe use of real-world data to generate evidence requires careful assessment and validation of critical variables before drawing clinical conclusions. Prospective clinical trial data suggest that anatomic origin of colon cancer impacts prognosis and treatment effectiveness. As an initial step in validating this observation in routine clinical settings, we explored the feasibility and accuracy of obtaining information on tumor sidedness from electronic health records (EHR) billing codes.MethodsNine thousand four hundred three patients with metastatic colorectal cancer (mCRC) were selected from the Flatiron Health database, which is derived from de-identified EHR data. This study included a random sample of 200 mCRC patients. Tumor site data derived from International Classification of Diseases (ICD) codes were compared with data abstracted from unstructured documents in the EHR (e.g. surgical and pathology notes). Concordance was determined via observed agreement and Cohen's kappa coefficient (?). Accuracy of ICD codes for each tumor site (left, right, transverse) was determined by calculating the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV), and corresponding 95% confidence intervals, using abstracted data as the gold standard.ResultsStudy patients had similar characteristics and side of colon distribution compared with the full mCRC dataset. The observed agreement between the ICD codes and abstracted data for tumor site for all sampled patients was 0.58 (??=?0.41). When restricting to the 62% of patients with a side-specific ICD code, the observed agreement was 0.84 (??=?0.79). The specificity (92-98%) of structured data for tumor location was high, with lower sensitivity (49-63%), PPV (64-92%) and NPV (72-97%). Demographic and clinical characteristics were similar between patients with specific and non-specific side of colon ICD codes.ConclusionsICD codes are a highly reliable indicator of tumor location when the specific location code is entered in the EHR. However, non-specific side of colon ICD codes are present for a sizable minority of patients, and structured data alone may not be adequate to support testing of some research hypotheses. Careful assessment of key variables is required before determining the need for clinical abstraction to supplement structured data in generating real-world evidence from EHRs.

Project description:Automated disease code classification using free-text medical information is important for public health surveillance. However, traditional natural language processing (NLP) pipelines are limited, so we propose a method combining word embedding with a convolutional neural network (CNN).Our objective was to compare the performance of traditional pipelines (NLP plus supervised machine learning models) with that of word embedding combined with a CNN in conducting a classification task identifying International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes in discharge notes.We used 2 classification methods: (1) extracting from discharge notes some features (terms, n-gram phrases, and SNOMED CT categories) that we used to train a set of supervised machine learning models (support vector machine, random forests, and gradient boosting machine), and (2) building a feature matrix, by a pretrained word embedding model, that we used to train a CNN. We used these methods to identify the chapter-level ICD-10-CM diagnosis codes in a set of discharge notes. We conducted the evaluation using 103,390 discharge notes covering patients hospitalized from June 1, 2015 to January 31, 2017 in the Tri-Service General Hospital in Taipei, Taiwan. We used the receiver operating characteristic curve as an evaluation measure, and calculated the area under the curve (AUC) and F-measure as the global measure of effectiveness.In 5-fold cross-validation tests, our method had a higher testing accuracy (mean AUC 0.9696; mean F-measure 0.9086) than traditional NLP-based approaches (mean AUC range 0.8183-0.9571; mean F-measure range 0.5050-0.8739). A real-world simulation that split the training sample and the testing sample by date verified this result (mean AUC 0.9645; mean F-measure 0.9003 using the proposed method). Further analysis showed that the convolutional layers of the CNN effectively identified a large number of keywords and automatically extracted enough concepts to predict the diagnosis codes.Word embedding combined with a CNN showed outstanding performance compared with traditional methods, needing very little data preprocessing. This shows that future studies will not be limited by incomplete dictionaries. A large amount of unstructured information from free-text medical writing will be extracted by automated approaches in the future, and we believe that the health care field is about to enter the age of big data.

Project description:Purpose: Previous studies identified the study cohort or outcome of carbon monoxide poisoning (COP) by using the relevant disease diagnosis codes in secondary databases, but the validity of diagnosis codes of COP is unclear in such secondary databases. This study aimed to evaluate the accuracy of case definitions for COP using the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and Tenth Revision, Clinical Modification (ICD-10-CM) diagnosis codes in Taiwan's health insurance claims data. Methods: We selected a 10% simple random sample from an original cohort of patients newly undergoing carboxyhemoglobin (COHb) testing under any clinical diagnosis at four Chang Gung Memorial Hospitals in Taiwan during 2011-2020. Two clinical doctors independently ascertained the COP diagnosis by reviewing the medical records as the reference standard. We estimated the sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) of various case definitions (e.g., ICD-9-CM code, ICD-10-CM code and the uses of hyperbaric oxygen therapy) in the claims data. Results: We randomly selected 557 cases from the original cohort of 5,571 cases newly receiving COHb test in the study hospitals. We found 90, 35, and 9 cases were true-positive, false-positive, and false-negative episodes of COP, respectively, among 278 cases with an ICD-9-CM code of 986. A further 111, 34, and 6 cases were true-positive, false-positive, and false-negative episodes of COP, respectively, among 279 cases with an ICD-10-CM code of T58. Using ICD-9-CM codes, the sensitivity, specificity, PPV and NPV for COP were 90.9, 80.4, 72, and 94.1%, respectively. Using ICD-10-CM codes they were 94.9, 79, 76.6, and 95.5%, respectively. PPV typically increased when COP was the primary diagnosis and could reach 100% if patients with ICD-CM code 986 or T58 also received hyperbaric oxygen therapy during hospitalization. Conclusion: The COP-related ICD-CM codes alone did not accurately identify COP patients, but accuracy improved after including oxygen therapy data with the ICD-CM codes in Taiwan's claims data.

Project description:Background/objectivesDue to high rates of multimorbidity, polypharmacy, and hazardous alcohol and opioid use, middle-aged Veterans are at risk for serious falls (those prompting a visit with a healthcare provider), posing significant risk to their forthcoming geriatric health and quality of life. We developed and validated a predictive model of the 6-month risk of serious falls among middle-aged Veterans.DesignCohort study.SettingVeterans Health Administration (VA).ParticipantsVeterans, aged 45 to 65 years, who presented for care within the VA between 2012 and 2015 (N = 275,940).ExposuresThe exposures of primary interest were substance use (including alcohol and prescription opioid use), multimorbidity, and polypharmacy. Hazardous alcohol use was defined as an Alcohol Use Disorders Identification Test - Consumption (AUDIT-C) score of 3 or greater for women and 4 or greater for men. We used International Classification of Diseases, Ninth Revision (ICD-9), codes to identify alcohol and illicit substance use disorders and identified prescription opioid use from pharmacy fill-refill data. We included counts of chronic medications and of physical and mental health comorbidities.MeasurementsWe identified serious falls using external cause of injury codes and a machine-learning algorithm that identified serious falls in radiology reports. We used multivariable logistic regression with general estimating equations to calculate risk. We used an integrated predictiveness curve to identify intervention thresholds.ResultsMost of our sample (54%) was aged 60 years or younger. Duration of follow-up was up to 4 years. Veterans who fell were more likely to be female (11% vs 7%) and White (72% vs 68%). They experienced 43,641 serious falls during follow-up. We identified 16 key predictors of serious falls and five interaction terms. Model performance was enhanced by addition of opioid use, as evidenced by overall category-free net reclassification improvement of 0.32 (P < .001). Discrimination (C-statistic = 0.76) and calibration were excellent for both development and validation data sets.ConclusionWe developed and internally validated a model to predict 6-month risk of serious falls among middle-aged Veterans with excellent discrimination and calibration.

Dataset Information

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

Objectives

Setting and participants

Design

Measures

Results

Conclusions

Publications

Validation of discharge diagnosis codes to identify serious infections among middle age and older adults.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure