Dataset Information

Performance of Cancer Recurrence Algorithms After Coding Scheme Switch From International Classification of Diseases 9th Revision to International Classification of Diseases 10th Revision.

ABSTRACT: PURPOSE:We previously developed and validated informatic algorithms that used International Classification of Diseases 9th revision (ICD9)-based diagnostic and procedure codes to detect the presence and timing of cancer recurrence (the RECUR Algorithms). In 2015, ICD10 replaced ICD9 as the worldwide coding standard. To understand the impact of this transition, we evaluated the performance of the RECUR Algorithms after incorporating ICD10 codes. METHODS:Using publicly available translation tables along with clinician and other expertise, we updated the algorithms to include ICD10 codes as additional input variables. We evaluated the performance of the algorithms using gold standard recurrence measures associated with a contemporary cohort of patients with stage I to III breast, colorectal, and lung (excluding IIIB) cancer and derived performance measures, including the area under the receiver operating curve, average absolute prediction error, and correct classification rate. These values were compared with the performance measures derived from the validation of the original algorithms. RESULTS:A total of 659 colorectal, 280 lung, and 2,053 breast cancer cases were identified. Area under the receiver operating curve derived from the updated algorithms was 89.0% (95% CI, 82.3% to 95.7%), 88.9% (95% CI, 79.3% to 98.2%), and 80.5% (95% CI, 72.8% to 88.2%) for the colorectal, lung, and breast cancer algorithms, respectively. Average absolute prediction errors for recurrence timing were 2.7 (SE, 11.3%), 2.4 (SE, 10.4%), and 5.6 months (SE, 21.8%), respectively, and timing estimates were within 6 months of actual recurrence for more than 80% of colorectal, more than 90% of lung, and more than 50% of breast cancer cases using the updated algorithm. CONCLUSION:Performance measures derived from the updated and original algorithms had overlapping confidence intervals, suggesting that the ICD9 to ICD10 transition did not affect the RECUR Algorithm performance.

SUBMITTER: Carroll NM

PROVIDER: S-EPMC6706070 | biostudies-literature | 2019 Mar

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThe automatic coding of clinical text documents by using the International Classification of Diseases, 10th Revision (ICD-10) can be performed for statistical analyses and reimbursements. With the development of natural language processing models, new transformer architectures with attention mechanisms have outperformed previous models. Although multicenter training may increase a model's performance and external validity, the privacy of clinical documents should be protected. We used federated learning to train a model with multicenter data, without sharing data per se.ObjectiveThis study aims to train a classification model via federated learning for ICD-10 multilabel classification.MethodsText data from discharge notes in electronic medical records were collected from the following three medical centers: Far Eastern Memorial Hospital, National Taiwan University Hospital, and Taipei Veterans General Hospital. After comparing the performance of different variants of bidirectional encoder representations from transformers (BERT), PubMedBERT was chosen for the word embeddings. With regard to preprocessing, the nonalphanumeric characters were retained because the model's performance decreased after the removal of these characters. To explain the outputs of our model, we added a label attention mechanism to the model architecture. The model was trained with data from each of the three hospitals separately and via federated learning. The models trained via federated learning and the models trained with local data were compared on a testing set that was composed of data from the three hospitals. The micro F1 score was used to evaluate model performance across all 3 centers.ResultsThe F1 scores of PubMedBERT, RoBERTa (Robustly Optimized BERT Pretraining Approach), ClinicalBERT, and BioBERT (BERT for Biomedical Text Mining) were 0.735, 0.692, 0.711, and 0.721, respectively. The F1 score of the model that retained nonalphanumeric characters was 0.8120, whereas the F1 score after removing these characters was 0.7875-a decrease of 0.0245 (3.11%). The F1 scores on the testing set were 0.6142, 0.4472, 0.5353, and 0.2522 for the federated learning, Far Eastern Memorial Hospital, National Taiwan University Hospital, and Taipei Veterans General Hospital models, respectively. The explainable predictions were displayed with highlighted input words via the label attention architecture.ConclusionsFederated learning was used to train the ICD-10 classification model on multicenter clinical text while protecting data privacy. The model's performance was better than that of models that were trained locally.

Project description:Administrative databases are increasingly used in research studies to capture clinical outcomes such as sepsis. This systematic review and meta-analysis examines the accuracy of International Classification of Diseases, 10th revision (ICD-10), codes for identifying sepsis in adult and pediatric patients.Data sourcesWe searched MEDLINE, EMBASE, Web of Science, CENTRAL, Epistemonikos, and McMaster Superfilters from inception to September 7, 2021.Study selectionWe included studies that validated the accuracy of sepsis ICD-10 codes against any reference standard.Data extractionThree authors, working in duplicate, independently extracted data. We conducted meta-analysis using a random effects model to pool sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV). We evaluated individual study risk of bias using the Quality Assessment of Diagnostic Accuracy Studies tool and assessed certainty in pooled diagnostic effect measures using the Grading of Recommendations Assessment, Development, and Evaluation framework.Data synthesisThirteen eligible studies were included in the qualitative synthesis and the meta-analysis. Eleven studies used manual chart review as the reference standard, and four studies used registry databases. Only one study evaluated pediatric patients exclusively. Compared with the reference standard of detailed chart review and/or registry databases, the pooled sensitivity for sepsis ICD-10 codes was 35% (95% CI, 22-48, low certainty), whereas the pooled specificity was 98% (95% CI: 98-99, low certainty). The PPV for ICD-10 codes ranged from 9.8% to 100% (median, 72.0%; interquartile range [IQR], 50.0-84.7%). NPV ranged from 54.7% to 99.1% (median, 95.9%; interquartile range, 85.5-98.3%).ConclusionsSepsis is undercoded in administrative databases. Future research is needed to explore if greater consistency in ICD-10 code definitions and enhanced quality measures for ICD-10 coders can improve the coding accuracy of sepsis in large databases.

Project description:To examine the validity of the International Classification of Diseases, 10th revision (ICD-10) codes for hyponatraemia in the nationwide population-based Danish National Registry of Patients (DNRP) among inpatients of all ages.Population-based validation study.All somatic hospitals in the North and Central Denmark Regions from 2006 through 2011.Patients of all ages admitted to hospital (n=819 701 individual patients) during the study period. The patient could be included in the study more than once, and our study did not restrict to patients with serum sodium measurements (total of n=2 186 642 hospitalisations).We validated ICD-10 discharge diagnoses of hyponatraemia recorded in the DNRP, using serum sodium measurements obtained from the laboratory information systems (LABKA) research database as the gold standard. One sodium value <135 mmol/L measured at any time during hospitalisation confirmed the diagnosis. We estimated sensitivity, specificity, positive predictive value (PPV) and negative predictive value (NPV) for ICD-10 codes for hyponatraemia overall and for cut-off points for increasing hyponatraemia severity.An ICD-10 code for hyponatraemia was recorded in the DNRP in 5850 of the 2 186 642 hospitalisations identified. According to laboratory measurements, however, hyponatraemia was present in 306 418 (14%) hospitalisations. Sensitivity of hyponatraemia diagnoses was 1.8% (95% CI 1.7% to 1.8%). For sodium values <115 mmol/L, sensitivity was 34.3% (95% CI 32.6% to 35.9%). The overall PPV was 92.5% (95% CI 91.8% to 93.1%) and decreased with increasing hyponatraemia severity. Specificity and NPV were high for all cut-off points (?99.8% and ?86.2%, respectively). Patients with hyponatraemia without a corresponding ICD-10 discharge diagnosis were younger and had higher Charlson Comorbidity Index scores than patients with hyponatraemia with a hyponatraemia code in the DNRP.ICD-10 codes for hyponatraemia in the DNRP have high specificity but very low sensitivity. Laboratory test results, not discharge diagnoses, should be used to ascertain hyponatraemia.

Dataset Information

Performance of Cancer Recurrence Algorithms After Coding Scheme Switch From International Classification of Diseases 9th Revision to International Classification of Diseases 10th Revision.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets