Dataset Information

Intra-database validation of case-identifying algorithms using reconstituted electronic health records from healthcare claims data.

ABSTRACT:

Background

Diagnosis performances of case-identifying algorithms developed in healthcare database are usually assessed by comparing identified cases with an external data source. When this is not feasible, intra-database validation can present an appropriate alternative.

Objectives

To illustrate through two practical examples how to perform intra-database validations of case-identifying algorithms using reconstituted Electronic Health Records (rEHRs).

Methods

Patients with 1) multiple sclerosis (MS) relapses and 2) metastatic castration-resistant prostate cancer (mCRPC) were identified in the French nationwide healthcare database (SNDS) using two case-identifying algorithms. A validation study was then conducted to estimate diagnostic performances of these algorithms through the calculation of their positive predictive value (PPV) and negative predictive value (NPV). To that end, anonymized rEHRs were generated based on the overall information captured in the SNDS over time (e.g. procedure, hospital stays, drug dispensing, medical visits) for a random selection of patients identified as cases or non-cases according to the predefined algorithms. For each disease, an independent validation committee reviewed the rEHRs of 100 cases and 100 non-cases in order to adjudicate on the status of the selected patients (true case/ true non-case), blinded with respect to the result of the corresponding algorithm.

Results

Algorithm for relapses identification in MS showed a 95% PPV and 100% NPV. Algorithm for mCRPC identification showed a 97% PPV and 99% NPV.

Conclusion

The use of rEHRs to conduct an intra-database validation appears to be a valuable tool to estimate the performances of a case-identifying algorithm and assess its validity, in the absence of alternative.

SUBMITTER: Thurin NH

PROVIDER: S-EPMC8088022 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundMusculoskeletal (MSK) conditions are a common presentation in primary care. This study sought to determine the prevalence of MSK conditions in primary care in Ontario and to validate the extent to which health administrative date billing codes accurately represent MSK diagnoses.MethodsDe-identified electronic medical records (EMR) from the DELPHI database in southwestern Ontario, which contains 2493 patients (55.6% female, mean age 50.3 years (SD = 22.2)) and 21,964 encounters (July 1, 2006-June 30, 2010) were used for the analyses. Outcomes included: validation measures of agreement between International Classification of Diseases (ICD-9) diagnostic codes (health administrative data) and International Classification of Primary Care (ICPC) diagnoses defined as the reference standard, time to first ICD-9 code, prevalence, and healthcare utilization.ResultsThere were 2940 true positive MSK encounters with primary care practitioners for 998 patients. Performance of the ICD-9 diagnostic codes included sensitivity = 76.5%, specificity = 95.2%, PPV = 94.6%, and NPV = 78.7%, compared to the ICPC reference standard. The majority of 998 patients were coded with both an ICPC and ICD-9 MSK code at their first or second encounter (67.4%). However, 23.5% of patients with the ICPC reference standard MSK were never coded with ICD-9. Four-year prevalence of MSK was 52.3% and varied by age (4.5% 0-17 years, 20.1% 18-44, 42.7% 45-64, and 32.7% 65+). Patients at MSK encounters had a higher number of: investigations (17.9% compared to 9.1%, p < .0001); referrals (17.6% compared to 14.3%, p < .0001); and prescriptions for opioids (17.2% compared to 5.3%, p < .0001).ConclusionsThis study determined the prevalence of musculoskeletal conditions in primary care in Ontario using a reference standard definition. The study highlighted the value of using primary care ICPC codes to validate a definition for musculoskeletal conditions. Health administrative data can be used to ascertain the presence of musculoskeletal conditions; however, ICD-9 codes may underrepresent the prevalence of MSK conditions.

Project description:BackgroundThe analysis of statutory health insurance (SHI) data is a little-used approach for understanding treatment and care as well as resource use of lung cancer (LC) patients in Germany. The aims of this observational, retrospective, longitudinal analysis of structured data were to analyze the healthcare situation of LC patients in Germany based on routine data from SHI funds, to develop an algorithm that sheds light on LC types (non-small cell / NSCLC vs. small cell / SCLC), and to gain new knowledge to improve needs-based care.MethodsAnonymized billing data of approximately four million people with SHI were analyzed regarding ICD-10 (German modification), documented medical interventions based on the outpatient SHI Uniform Assessment Standard Tariff (EBM) or the inpatient Operations and Procedure Code (OPS), and the dispensing of prescription drugs to outpatients (ATC classification). The study included patients who were members of 64 SHI funds between Jan-1st, 2015 and Dec-31st, 2016 and who received the initial diagnosis of LC in 2015 and 2016.ResultsThe analysis shows that neither the cancer type nor the cancer stage can be unambiguously described by the ICD-10 coding. Furthermore, an assignment based on the prescribed medication provides only limited information: many of the drugs are either approved for both LC types or are used off-label, making it difficult to assign them to a specific LC type. Overall, 25% of the LC patients were unambiguously identifiable as NSCLC vs SCLC based on the ICD-10 code, the drug therapy, and the billing data.ConclusionsThe current coding system appears to be of limited suitability for drawing conclusions about LC and therefore the SHI patient population. This makes it difficult to analyze the healthcare data with the aim of gathering new knowledge to improve needs-based care. The approach chosen for this study did not allow for development of a LC differentiation algorithm based on the available healthcare data. However, a better overview of patient specific needs could make it possible to modify the range of services provided by the SHI funds. From this perspective, it makes sense, in a first step, to refine the ICD-10 system to facilitate NSCLC vs. SCLC classification.

Project description:BackgroundMuscular dystrophies (MDs) are a group of inherited conditions characterized by progressive muscle degeneration and weakness. The rarity and heterogeneity of the population with MD have hindered therapeutic developments as well as epidemiological and health outcomes research. The objective of the study was to develop and validate a case-finding algorithm utilizing administrative claims data to identify and characterize patients with MD.MethodsThis retrospective cohort study used medical chart validation to evaluate an ICD-9/10 coding algorithm in a large commercial claims database. Patients were identified who had ≥2 office visits with a diagnosis of hereditary progressive MDs from January 1, 2013 through December 31, 2016, were male, and younger than 18 years at the time of first MD diagnosis. Cases who met the algorithm were then validated against medical charts. Diagnoses of MD and specific type (Duchenne, Becker, or other MD) were confirmed by medical chart review by trained reviewers. Positive predictive value (PPV) and 95% confidence intervals (CI) were calculated using a 2 × 2 contingence table. Patient demographic, clinical, and health utilization characteristics were summarized using basic descriptive statistics.ResultsCharts were obtained and reviewed for 109 patients who met the algorithm. The PPV of the case-identifying algorithm for MD was 95% (95% CI 88-98%). Of the 103 confirmed MD cases, 87 patients (85%, 95% CI 76-91%) had Duchenne or Becker MD; 76 patients (74%, 95% CI 64-82%) had Duchenne MD, and 11 patients (11%, 95% CI 5-18%) had Becker MD. A total of 74 (67.9%) patients had ≥1 pediatric complex chronic condition (other than neurologic/neuromuscular disease); 54 (49.5%) had cardiovascular conditions; 14 (12.8%) had respiratory conditions; 50 (45.9%) had bone-related issues; 11 (10.1%) had impaired growth; and 6 (5.5%) had puberty delay.ConclusionsThe results of this study demonstrate that the case-finding algorithm accurately identified patients with MD, primarily Duchenne MD, within a large administrative database. The algorithm, which was constructed using a few items easily accessible from claims, can be used to facilitate epidemiological and health outcomes research in the Duchenne patient population.

Project description:BackgroundDrug-related adverse events remain an important cause of morbidity and mortality and impose huge burden on healthcare costs. Routinely collected electronic healthcare data give a good snapshot of how drugs are being used in 'real-world' settings.ObjectiveTo describe a strategy that identifies potentially drug-induced acute myocardial infarction (AMI) from a large international healthcare data network.MethodsPost-marketing safety surveillance was conducted in seven population-based healthcare databases in three countries (Denmark, Italy, and the Netherlands) using anonymised demographic, clinical, and prescription/dispensing data representing 21,171,291 individuals with 154,474,063 person-years of follow-up in the period 1996-2010. Primary care physicians' medical records and administrative claims containing reimbursements for filled prescriptions, laboratory tests, and hospitalisations were evaluated using a three-tier triage system of detection, filtering, and substantiation that generated a list of drugs potentially associated with AMI. Outcome of interest was statistically significant increased risk of AMI during drug exposure that has not been previously described in current literature and is biologically plausible.ResultsOverall, 163 drugs were identified to be associated with increased risk of AMI during preliminary screening. Of these, 124 drugs were eliminated after adjustment for possible bias and confounding. With subsequent application of criteria for novelty and biological plausibility, association with AMI remained for nine drugs ('prime suspects'): azithromycin; erythromycin; roxithromycin; metoclopramide; cisapride; domperidone; betamethasone; fluconazole; and megestrol acetate.LimitationsAlthough global health status, co-morbidities, and time-invariant factors were adjusted for, residual confounding cannot be ruled out.ConclusionA strategy to identify potentially drug-induced AMI from electronic healthcare data has been proposed that takes into account not only statistical association, but also public health relevance, novelty, and biological plausibility. Although this strategy needs to be further evaluated using other healthcare data sources, the list of 'prime suspects' makes a good starting point for further clinical, laboratory, and epidemiologic investigation.