Project description:PURPOSE:We previously developed and validated informatic algorithms that used International Classification of Diseases 9th revision (ICD9)-based diagnostic and procedure codes to detect the presence and timing of cancer recurrence (the RECUR Algorithms). In 2015, ICD10 replaced ICD9 as the worldwide coding standard. To understand the impact of this transition, we evaluated the performance of the RECUR Algorithms after incorporating ICD10 codes. METHODS:Using publicly available translation tables along with clinician and other expertise, we updated the algorithms to include ICD10 codes as additional input variables. We evaluated the performance of the algorithms using gold standard recurrence measures associated with a contemporary cohort of patients with stage I to III breast, colorectal, and lung (excluding IIIB) cancer and derived performance measures, including the area under the receiver operating curve, average absolute prediction error, and correct classification rate. These values were compared with the performance measures derived from the validation of the original algorithms. RESULTS:A total of 659 colorectal, 280 lung, and 2,053 breast cancer cases were identified. Area under the receiver operating curve derived from the updated algorithms was 89.0% (95% CI, 82.3% to 95.7%), 88.9% (95% CI, 79.3% to 98.2%), and 80.5% (95% CI, 72.8% to 88.2%) for the colorectal, lung, and breast cancer algorithms, respectively. Average absolute prediction errors for recurrence timing were 2.7 (SE, 11.3%), 2.4 (SE, 10.4%), and 5.6 months (SE, 21.8%), respectively, and timing estimates were within 6 months of actual recurrence for more than 80% of colorectal, more than 90% of lung, and more than 50% of breast cancer cases using the updated algorithm. CONCLUSION:Performance measures derived from the updated and original algorithms had overlapping confidence intervals, suggesting that the ICD9 to ICD10 transition did not affect the RECUR Algorithm performance.
Project description:The Faurot frailty index (FFI) is a validated algorithm that uses enrollment and International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM)-based billing information from Medicare claims data as a proxy for frailty. In October 2015, the US health-care system transitioned from the ICD-9-CM to the International Classification of Diseases, Tenth Revision, Clinical Modification (ICD-10-CM). Applying the Centers for Medicare and Medicaid Services General Equivalence Mappings, we translated diagnosis-based frailty indicator codes from the ICD-9-CM to the ICD-10-CM, followed by manual review. We used interrupted time-series analysis of Medicare data to assess the comparability of the pre- and posttransition FFI scores. In cohorts of beneficiaries enrolled in January 2015-2017 with 8-month frailty look-back periods, we estimated associations between the FFI and 1-year risk of aging-related outcomes (mortality, hospitalization, and admission to a skilled nursing facility). Updated indicators had similar prevalences as pretransition definitions. The median FFI scores and interquartile ranges (IQRs) for the predicted probability of frailty were similar before and after the International Classification of Diseases transition (pretransition: median, 0.034 (IQR, 0.02-0.07); posttransition: median, 0.038 (IQR, 0.02-0.09)). The updated FFI was associated with increased risks of mortality, hospitalization, and skilled nursing facility admission, similar to findings from the ICD-9-CM era. Studies of medical interventions in older adults using administrative claims should use validated indices, like the FFI, to mitigate confounding or assess effect-measure modification by frailty.
Project description:ObjectivesTo evaluate the impact of International Classification of Disease, 10th revision, Clinical Modification (ICD-10-CM) implementation on pneumonia hospitalizations rates, which had declined following pneumococcal conjugate vaccine introduction for infants in 2000.MethodsWe randomly selected records from a single hospital 1 year before (n = 500) and after (n = 500) October 2015 implementation of ICD-10-CM coding. We used a validated ICD-9-CM algorithm and translation of that algorithm to ICD-10-CM to identify pneumonia hospitalizations pre- and post-implementation, respectively. We recoded ICD-10-CM records to ICD-9-CM and vice versa. We calculated sensitivity and positive predictive value (PPV) of the ICD-10-CM algorithm using ICD-9-CM coding as the reference. We used sensitivity and PPV values to calculate an adjustment factor to apply to ICD-10 era rates to enable comparison with ICD-9-CM rates. We reviewed primary diagnoses of charts not meeting the pneumonia definition when recoded.ResultsSensitivity and PPV of the ICD-10-CM algorithm were 94% and 92%, respectively, for young children and 74% and 79% for older adults. The estimated adjustment factor for ICD-10-CM period rates was -2.09% (95% credible region [CR], -7.71% to +3.0%) for children and +6.76% (95% CR, -3.06% to +16.7%) for older adults. We identified a change in coding adult charts that met the ICD-9-CM pneumonia definition that led to recoding in ICD-10-CM as chronic obstructive pulmonary disease (COPD) exacerbation.ConclusionsThe ICD-10-CM algorithm derived from a validated ICD-9-CM algorithm should not introduce substantial bias for evaluating pneumonia trends in children. However, changes in coding of pneumonia associated with COPD in adults warrant further study.
Project description:BackgroundThe Functional Comorbidity Index (FCI) was developed for community-based adult populations, with function as the outcome. The original FCI was a survey tool, but several International Classification of Diseases (ICD) code lists-for calculating the FCI using administrative data-have been published. However, compatible International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) and ICD-10-CM versions have not been available.ObjectiveWe developed ICD-9-CM and ICD-10-CM diagnosis code lists to optimize FCI concordance across ICD lexicons.Research designWe assessed concordance and frequency distributions across ICD lexicons for the FCI and individual comorbidities. We used length of stay and discharge disposition to assess continuity of FCI criterion validity across lexicons.SubjectsState Inpatient Databases from Arizona, Colorado, Michigan, New Jersey, New York, Utah, and Washington State (calendar year 2015) were obtained from the Healthcare Cost and Utilization Project. State Inpatient Databases contained ICD-9-CM diagnoses for the first 3 calendar quarters of 2015 and ICD-10-CM diagnoses for the fourth quarter of 2015. Inpatients under 18 years old were excluded.MeasuresLength of stay and discharge disposition outcomes were assessed in separate regression models. Covariates included age, sex, state, ICD lexicon, and FCI/lexicon interaction.ResultsThe FCI demonstrated stability across lexicons, despite small discrepancies in prevalence for individual comorbidities. Under ICD-9-CM, each additional comorbidity was associated with an 8.9% increase in mean length of stay and an 18.5% decrease in the odds of a routine discharge, compared with an 8.4% increase and 17.4% decrease, respectively, under ICD-10-CM.ConclusionThis study provides compatible ICD-9-CM and ICD-10-CM diagnosis code lists for the FCI.
Project description:PurposeAn International Classification of Disease (ICD-10) Charlson Comorbidity Index (CCI) adaptation had not been previously developed and validated for United States (US) healthcare claims data. Many researchers use the Canadian adaption by Quan et al (2005), not validated in US data. We sought to evaluate the predictive validity of a US ICD-10 CCI adaptation in US claims and compare it with the Canadian standard.MethodsDiverse patient cohorts (rheumatoid arthritis, hip/knee replacement, lumbar spine surgery, acute myocardial infarction [AMI], stroke, pneumonia) in the IBM® MarketScan® Research Databases were linked with the IBM MarketScan Mortality file. Predictive performance was measured using c-statistics for binary outcomes (1-year and postoperative mortality, in-hospital complications) and root mean square prediction error (RMSE) for continuous outcomes (1-year all-cause medical costs, index hospitalization costs, length of stay [LOS]), after adjusting for age and sex. C-statistics were compared by the method of DeLong and colleagues (1988); RMSEs, by resampling.ResultsC-statistics were generally high (≥ ~ 0.8) for mortality but lower for in-hospital complications (~0.6-0.7). RMSEs for costs and hospitalization LOS were relatively large and comparable to standard deviations. Results were similar overall between the US and Canadian adaptations, with relative differences typically <1%.ConclusionsThis US-based coding adaptation and a previously published Canadian adaptation resulted in similar predictive ability for all outcomes evaluated but may have different construct validity (not evaluated in our study). We recommend using adaptations specific to the country of data origin based on good research practice.
Project description:BackgroundThe automatic coding of clinical text documents by using the International Classification of Diseases, 10th Revision (ICD-10) can be performed for statistical analyses and reimbursements. With the development of natural language processing models, new transformer architectures with attention mechanisms have outperformed previous models. Although multicenter training may increase a model's performance and external validity, the privacy of clinical documents should be protected. We used federated learning to train a model with multicenter data, without sharing data per se.ObjectiveThis study aims to train a classification model via federated learning for ICD-10 multilabel classification.MethodsText data from discharge notes in electronic medical records were collected from the following three medical centers: Far Eastern Memorial Hospital, National Taiwan University Hospital, and Taipei Veterans General Hospital. After comparing the performance of different variants of bidirectional encoder representations from transformers (BERT), PubMedBERT was chosen for the word embeddings. With regard to preprocessing, the nonalphanumeric characters were retained because the model's performance decreased after the removal of these characters. To explain the outputs of our model, we added a label attention mechanism to the model architecture. The model was trained with data from each of the three hospitals separately and via federated learning. The models trained via federated learning and the models trained with local data were compared on a testing set that was composed of data from the three hospitals. The micro F1 score was used to evaluate model performance across all 3 centers.ResultsThe F1 scores of PubMedBERT, RoBERTa (Robustly Optimized BERT Pretraining Approach), ClinicalBERT, and BioBERT (BERT for Biomedical Text Mining) were 0.735, 0.692, 0.711, and 0.721, respectively. The F1 score of the model that retained nonalphanumeric characters was 0.8120, whereas the F1 score after removing these characters was 0.7875-a decrease of 0.0245 (3.11%). The F1 scores on the testing set were 0.6142, 0.4472, 0.5353, and 0.2522 for the federated learning, Far Eastern Memorial Hospital, National Taiwan University Hospital, and Taipei Veterans General Hospital models, respectively. The explainable predictions were displayed with highlighted input words via the label attention architecture.ConclusionsFederated learning was used to train the ICD-10 classification model on multicenter clinical text while protecting data privacy. The model's performance was better than that of models that were trained locally.
Project description:Administrative databases are increasingly used in research studies to capture clinical outcomes such as sepsis. This systematic review and meta-analysis examines the accuracy of International Classification of Diseases, 10th revision (ICD-10), codes for identifying sepsis in adult and pediatric patients.Data sourcesWe searched MEDLINE, EMBASE, Web of Science, CENTRAL, Epistemonikos, and McMaster Superfilters from inception to September 7, 2021.Study selectionWe included studies that validated the accuracy of sepsis ICD-10 codes against any reference standard.Data extractionThree authors, working in duplicate, independently extracted data. We conducted meta-analysis using a random effects model to pool sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We evaluated individual study risk of bias using the Quality Assessment of Diagnostic Accuracy Studies tool and assessed certainty in pooled diagnostic effect measures using the Grading of Recommendations Assessment, Development, and Evaluation framework.Data synthesisThirteen eligible studies were included in the qualitative synthesis and the meta-analysis. Eleven studies used manual chart review as the reference standard, and four studies used registry databases. Only one study evaluated pediatric patients exclusively. Compared with the reference standard of detailed chart review and/or registry databases, the pooled sensitivity for sepsis ICD-10 codes was 35% (95% CI, 22-48, low certainty), whereas the pooled specificity was 98% (95% CI: 98-99, low certainty). The PPV for ICD-10 codes ranged from 9.8% to 100% (median, 72.0%; interquartile range [IQR], 50.0-84.7%). NPV ranged from 54.7% to 99.1% (median, 95.9%; interquartile range, 85.5-98.3%).ConclusionsSepsis is undercoded in administrative databases. Future research is needed to explore if greater consistency in ICD-10 code definitions and enhanced quality measures for ICD-10 coders can improve the coding accuracy of sepsis in large databases.
Project description:International Classification of Diseases, 10th Revision codes (ICD-10) for autosomal dominant polycystic kidney disease (ADPKD) is used within several administrative health care databases. It is unknown whether these codes identify patients who meet strict clinical criteria for ADPKD.The objective of this study is (1) to determine whether different ICD-10 coding algorithms identify adult patients who meet strict clinical criteria for ADPKD as assessed through medical chart review and (2) to assess the number of patients identified with different ADPKD coding algorithms in Ontario.Validation study of health care database codes, and prevalence.Ontario, Canada.For the chart review, 201 adult patients with hospital encounters between April 1, 2002, and March 31, 2014, assigned either ICD-10 codes Q61.2 or Q61.3.This study measured positive predictive value of the ICD-10 coding algorithms and the number of Ontarians identified with different coding algorithms.We manually reviewed a random sample of medical charts in London, Ontario, Canada, and determined whether or not ADPKD was present according to strict clinical criteria.The presence of either ICD-10 code Q61.2 or Q61.3 in a hospital encounter had a positive predictive value of 85% (95% confidence interval [CI], 79%-89%) and identified 2981 Ontarians (0.02% of the Ontario adult population). The presence of ICD-10 code Q61.2 in a hospital encounter had a positive predictive value of 97% (95% CI, 86%-100%) and identified 394 adults in Ontario (0.003% of the Ontario adult population).(1) We could not calculate other measures of validity; (2) the coding algorithms do not identify patients without hospital encounters; and (3) coding practices may differ between hospitals.Most patients with ICD-10 code Q61.2 or Q61.3 assigned during their hospital encounters have ADPKD according to the clinical criteria. These codes can be used to assemble cohorts of adult patients with ADPKD and hospital encounters.