Dataset Information

Accuracy of algorithms to identify patients with a diagnosis of major cancers and cancer-related adverse events in an administrative database: a validation study in an acute care hospital in Japan.

ABSTRACT:

Objectives

Validation studies in oncology are limited in Japan. This study was conducted to evaluate the accuracy of diagnosis and adverse event (AE) definitions for specific cancers in a Japanese health administrative real-world database (RWD).

Design and setting

Retrospective observational validation study to assess the diagnostic accuracy of electronic medical records (EMRs) and claim coding regarding oncology diagnosis and AEs based on medical record review in the RWD. The sensitivity and positive predictive value (PPV) with 95% CIs were calculated.

Participants

The validation cohort included patients with lung (n=2257), breast (n=1121), colorectal (n=1773), ovarian (n=216) and bladder (n=575) cancer who visited the hospital between January 2014 and December 2018, and those with prostate cancer (n=3491) visiting between January 2009 and December 2018, who were identified using EMRs.

Outcomes

Key outcomes included primary diagnosis, deaths and AEs.

Results

For primary diagnosis, sensitivity and PPV for the respective cancers were as follows: lung, 100.0% (96.6 to 100.0) and 81.0% (74.9 to 86.2); breast, 100.0% (96.3 to 100.0) and 74.0% (67.3 to 79.9); colorectal, 100.0% (96.6 to 100.0) and 80.5% (74.3 to 85.8); ovarian, 89.8% (77.8 to 96.6) and 75.9% (62.8 to 86.1); bladder, 78.6% (63.2 to 89.7) and 67.3% (52.5 to 0.1); prostate, 100.0% (93.2 to 100.0) and 79.0% (69.7 to 86.5). Sensitivity and PPV for death were as follows: lung, 97.0% (84.2 to 99.9) and 100.0% (84.2 to 100.0); breast, 100.0% (1.3 to 100.0) and 100.0% (1.3 to 100.0); colorectal, 100.0% (28.4 to 100.0) and 100.0% (28.4 to 100.0); ovarian, 100.0% (35.9 to 100.0) and 100.0% (35.9 to 100.0); bladder, 100.0% (9.4-100.0) and 100.0% (9.4 to 100.0); prostate, 75.0% (19.4 to 99.4) and 100.0% (19.4 to 100.0). Overall, PPV tended to be low, with the definition based on International Classification of Diseases, 10th revision alone for AEs.

Conclusion

Diagnostic accuracy was not so high, and therefore needs to be further investigated.

Trial registration number

University Hospital Medical Information Network (UMIN) Clinical Trials Registry (UMIN000039345).

SUBMITTER: Fujiwara T

PROVIDER: S-EPMC9280899 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundAdministrative data algorithms (ADAs) to identify pneumonia cases are commonly used in the analysis of pneumonia burden, trends, etiology, processes of care, outcomes, health care utilization, cost, and response to preventative and therapeutic interventions. However, without a good understanding of the validity of ADAs for pneumonia case identification, an adequate appreciation of this literature is difficult. We systematically reviewed the quality and accuracy of published ADAs to identify adult hospitalized pneumonia cases.MethodsWe reviewed the Medline, EMBase, and Cochrane Central databases through May 2020. All studies describing ADAs for adult hospitalized pneumonia and at least one accuracy statistic were included. Investigators independently extracted information about the sampling frame, reference standard, ADA composition, and ADA accuracy.ResultsThirteen studies involving 24 ADAs were analyzed. Compliance with a 38-item study-quality assessment tool ranged from 17 to 29 (median, 23; interquartile range [IQR], 20 to 25). Study setting, design, and ADA composition varied extensively. Inclusion criteria of most studies selected for high-risk populations and/or increased pneumonia likelihood. Reference standards with explicit criteria (clinical, laboratorial, and/or radiographic) were used in only 4 ADAs. Only 2 ADAs were validated (one internally and one externally). ADA positive predictive values ranged from 35.0 to 96.5% (median, 84.8%; IQR, 65.3 to 89.1%). However, these values are exaggerated for an unselected patient population because pneumonia prevalences in the study cohorts were very high (median, 66%; IQR, 46 to 86%). ADA sensitivities ranged from 31.3 to 97.8% (median, 65.1%; IQR 52.5-72.4).DiscussionADAs for identification of adult pneumonia hospitalizations are highly heterogeneous, poorly validated, and at risk for misclassification bias. Greater standardization in reporting ADA accuracy is required in studies using pneumonia ADA for case identification so that results can be properly interpreted.

Project description:BackgroundLack of using a validated algorithm to select patients is a source of selection bias in oncology studies using administrative claims. The objective of this study to evaluate published algorithms to identify patients with soft tissue sarcoma (STS) in administrative claims and to evaluate new algorithms to improved performance.MethodsTwo cancer populations including STS cases and non-STS controls were selected from the MarketScan Explorys Linked Claims-Electronic Medical Record (EMR) Database between January 1, 2000 and July 31, 2018. Eligible cases had a diagnosis on a clinical record for STS in the EMR while controls had no evidence of STS on any EMR records. Both cases and controls were enrolled in administrative claims during a period of observation and were aged???18 years. A split sample was used to test and validate algorithms using data from administrative claims. Values for sensitivity, specificity, and positive predictive value (PPV) were calculated for 14 algorithms. Prior literature validating algorithms in administrative claims across other cancer types report both sensitivity and specificity ranging from as low as 73% to as high as 95%. This was used as a benchmark for defining algorithm success.ResultsThere were 784 STS cases and 249,062 non-STS cancer controls eligible for analysis. Requiring at least two claims with an ICD-CM diagnosis code for STS achieved a sensitivity of 67% but had a specificity of 72%. Algorithms that required NCCN-recommended systemic treatment for STS improved the specificity to over 90% but dropped the sensitivity to below 20%. Other combinations of diagnostic tests, symptoms, and procedures did not improve performance.ConclusionsThe algorithms tested in this study sample did not achieve sufficient performance and suggest the ability to accurately identify the STS population in administrative data is problematic. Difficulties are likely due to the origin of STS in a variety of locations, the non-specific symptoms of STS, and the common diagnostic tests recommended to diagnose the disease. Future research applying machine learning to examine timing and patterns of variables that comprise the diagnostic process may further investigate the ability to accurately identify STS cases in claims databases.

Project description:Background and purpose: Testosterone is an essential sex hormone in maintaining masculine characteristics, which is prescribed for male hypogonadism as testosterone replacement treatment (TRT). Herein, we investigated long-standing controversies about the association between TRT and major adverse cardiovascular events (MACEs), based on real world adverse event (AE) reports, registered in the Food and Drug Administration Adverse Event Reporting System (FAERS). Methods: Publicly available FAERS data from 1 January 2004 to 31 December 2022 were retrieved from the Food and Drug Administration (FDA) website. The data mining protocol including the reporting odds ratio (ROR) and the Bayesian confidence propagation neural network (BCPNN) was applied to analyze overreporting caused by risk factors and MACEs, including TRT, morbidities, and ages. The ROR and the BCPNN were also applied to investigate the annually developing trend of pharmacovigilance (PV) signals in the real world, retrospectively. Results: A total of 3,057 cases referring to MACEs, with a median age of 57 years old (yo), were identified from 28,921 cases of testosterone users. MACEs related to PV signals have emerged since 2014, including cardiac death, non-fatal myocardial infarction, and non-fatal stroke. Myocardial infarction (MI) (ROR: 9.46; IC025: 3.08), acute myocardial infarction (AMI) (ROR: 16.20; IC025: 3.72), ischemic cardiomyopathy (ROR: 11.63; IC025: 2.20), and cardiomyopathy (ROR: 5.98; IC025: 1.96) were the most significant signals generated, and weaker signals included cardiac failure acute (ROR: 4.01; IC025: 0.71), cardiac arrest (ROR: 1.88; IC025: 0.56), and ventricular fibrillation (VF) (ROR: 2.38; IC025: 0.38). The time-to-onset (TTO) of MACEs was calculated with a median of 246 days for AMI. Conclusion: For myocardial infarction and cardiomyopathy, TRT statistically tended to increase the risk of MACEs, while for cardiac arrhythmia, cardiac failure, and stroke, TRT demonstrated beneficial effects among the population with morbidities, such as testosterone deficiency (TD), diabetes mellitus (DM), and hypertension. MACEs were rare but led to serious outcomes including significant increase in death and disability. Since 2018, and before 2014, reports referring to TRT associated with MACEs were relatively scarce, which indicated that there might be a considerable number of cases that went unrecorded, due to neglection. Health workers and testosterone users might pay more attention to testosterone-induced MACEs.