Dataset Information

Accuracy of claims-based algorithms for epilepsy research: Revealing the unseen performance of claims-based studies.

ABSTRACT: OBJECTIVE:To evaluate published algorithms for the identification of epilepsy cases in medical claims data using a unique linked dataset with both clinical and claims data. METHODS:Using data from a large, regional health delivery system, we identified all patients contributing biologic samples to the health system's Biobank (n = 36K). We identified all subjects with at least one diagnosis potentially consistent with epilepsy, for example, epilepsy, convulsions, syncope, or collapse, between 2014 and 2015, or who were seen at the epilepsy clinic (n = 1,217), plus a random sample of subjects with neither claims nor clinic visits (n = 435); we then performed a medical chart review in a random subsample of 1,377 to assess the epilepsy diagnosis status. Using the chart review as the reference standard, we evaluated the test characteristics of six published algorithms. RESULTS:The best-performing algorithm used diagnostic and prescription drug data (sensitivity = 70%, 95% confidence interval [CI] 66-73%; specificity = 77%, 95% CI 73-81%; and area under the curve [AUC] = 0.73, 95%CI 0.71-0.76) when applied to patients age 18 years or older. Restricting the sample to adults aged 18-64 years resulted in a mild improvement in accuracy (AUC = 0.75,95%CI 0.73-0.78). Adding information about current antiepileptic drug use to the algorithm increased test performance (AUC = 0.78, 95%CI 0.76-0.80). Other algorithms varied in their included data types and performed worse. SIGNIFICANCE:Current approaches for identifying patients with epilepsy in insurance claims have important limitations when applied to the general population. Approaches incorporating a range of information, for example, diagnoses, treatments, and site of care/specialty of physician, improve the performance of identification and could be useful in epilepsy studies using large datasets.

SUBMITTER: Moura LM

PROVIDER: S-EPMC6592609 | biostudies-literature | 2017 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Accuracy of claims-based algorithms for epilepsy research: Revealing the unseen performance of claims-based studies.

Moura Lidia M V R LM Price Maggie M Cole Andrew J AJ Hoch Daniel B DB Hsu John J

Epilepsia 20170215 4

<h4>Objective</h4>To evaluate published algorithms for the identification of epilepsy cases in medical claims data using a unique linked dataset with both clinical and claims data.<h4>Methods</h4>Using data from a large, regional health delivery system, we identified all patients contributing biologic samples to the health system's Biobank (n = 36K). We identified all subjects with at least one diagnosis potentially consistent with epilepsy, for example, epilepsy, convulsions, syncope, or collap ...[more]

PMID: 28199007

Similar Datasets

Project description:ObjectiveTo evaluate the accuracy of ICD-10-CM claims-based definitions for epilepsy and classifying seizure types in the outpatient setting.MethodsWe reviewed electronic health records (EHR) for a cohort of adults aged 18+ years seen by six neurologists who had an outpatient visit at a level 4 epilepsy center between 01/2019-09/2019. The neurologists used a standardized documentation template to capture the diagnosis of epilepsy (yes/no/unsure), seizure type (focal/generalized/unknown), and seizure frequency in the EHR. Using linked ICD-10-CM codes assigned by the provider, we assessed the accuracy of claims-based definitions for epilepsy, focal seizure type, and generalized seizure type against the reference-standard EHR documentation by estimating sensitivity (Sn), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV).ResultsThere were 673 eligible outpatient encounters. After review of EHRs for standardized documentation, an analytic sample consisted of 520 encounters representing 402 unique patients. In the EHR documentation, 93.5 % (n = 486/520) of encounters were with patients with a diagnosis of epilepsy. Of those, 66.0 % (n = 321/486) had ≥1 focal seizure, 41.6 % (n = 202/486) had ≥1 generalized seizure, and 7% (n = 34/486) had ≥1 unknown seizure. An ICD-10-CM definition for epilepsy (i.e., ICD-10 G40.X) achieved Sn = 84.4 % (95 % CI 80.8-87.5%), Sp = 79.4 % (95 % CI 62.1-91.3%), PPV = 98.3 % (95 % CI 96.6-99.3%), and NPV = 26.2 % (95 % CI 18.0-35.8%). The classification of focal vs generalized/unknown seizures achieved Sn = 69.8 % (95 % CI 64.4-74.8%), Sp = 79.4 % (95 % CI 72.4-85.3%), PPV = 86.8 % (95 % CI 82.1-90.7%), and NPV = 57.5 % (95 % CI 50.8-64.0%).ConclusionsClaims-based definitions using groups of ICD-10-CM codes assigned by neurologists in routine outpatient clinic visits at a level 4 epilepsy center performed well in discriminating between patients with and without a diagnosis of epilepsy and between seizure types.

Project description:Administrative claims studies do not adequately distinguish pulmonary arterial hypertension (PAH) from other forms of pulmonary hypertension (PH). Our aim is to develop and validate a set of algorithms using International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) codes and electronic medical records (EMR), to identify patients with PAH. From January 2012 to August 2015, the EMRs of patients with ICD-9-CM codes for PH with an outpatient visit at the University of Texas Medical Branch were reviewed. Patients were divided into PAH or non-PAH groups according to EMR encounter diagnosis. Patient demographics, echocardiography, right heart catheterization (RHC) results, and PAH-specific therapies were assessed. RHC measurements were reviewed to categorize cases as hemodynamically determined PAH or not PAH. Weighted sensitivity, specificity, and positive and negative predictive values were calculated for the developed algorithms. A logistic regression analysis was conducted to determine how well the algorithms performed. External validation was performed at the University of Virginia Health System. The cohort for the development algorithms consisted of 683 patients with PH, PAH group (n = 191) and non-PAH group (n = 492). A hemodynamic diagnosis of PAH determined by RHC was recorded in the PAH (26%) and non-PAH (3%) groups. The positive predictive value for the algorithm that included ICD-9-CM and PAH-specific medications was 66.9% and sensitivity was 28.2% with a c-statistic of 0.66. The positive predictive value for the EMR-based algorithm that included ICD-9-CM, EMR encounter diagnosis, echocardiography, RHC, and PAH-specific medication was 69.4% and a c-statistic of 0.87. A validation cohort of 177 patients with PH examined from August 2015 to August 2016 using EMR-based algorithms yielded a similar positive predictive value of 62.5%. In conclusion, claims-based algorithms that included ICD-9-CM codes, EMR encounter diagnosis, echocardiography, RHC, and PAH-specific medications better-identified patients with PAH than ICD-9-CM codes alone.

Project description:BackgroundLack of using a validated algorithm to select patients is a source of selection bias in oncology studies using administrative claims. The objective of this study to evaluate published algorithms to identify patients with soft tissue sarcoma (STS) in administrative claims and to evaluate new algorithms to improved performance.MethodsTwo cancer populations including STS cases and non-STS controls were selected from the MarketScan Explorys Linked Claims-Electronic Medical Record (EMR) Database between January 1, 2000 and July 31, 2018. Eligible cases had a diagnosis on a clinical record for STS in the EMR while controls had no evidence of STS on any EMR records. Both cases and controls were enrolled in administrative claims during a period of observation and were aged???18 years. A split sample was used to test and validate algorithms using data from administrative claims. Values for sensitivity, specificity, and positive predictive value (PPV) were calculated for 14 algorithms. Prior literature validating algorithms in administrative claims across other cancer types report both sensitivity and specificity ranging from as low as 73% to as high as 95%. This was used as a benchmark for defining algorithm success.ResultsThere were 784 STS cases and 249,062 non-STS cancer controls eligible for analysis. Requiring at least two claims with an ICD-CM diagnosis code for STS achieved a sensitivity of 67% but had a specificity of 72%. Algorithms that required NCCN-recommended systemic treatment for STS improved the specificity to over 90% but dropped the sensitivity to below 20%. Other combinations of diagnostic tests, symptoms, and procedures did not improve performance.ConclusionsThe algorithms tested in this study sample did not achieve sufficient performance and suggest the ability to accurately identify the STS population in administrative data is problematic. Difficulties are likely due to the origin of STS in a variety of locations, the non-specific symptoms of STS, and the common diagnostic tests recommended to diagnose the disease. Future research applying machine learning to examine timing and patterns of variables that comprise the diagnostic process may further investigate the ability to accurately identify STS cases in claims databases.

Project description:Community clinics offer potential for timelier outbreak detection and monitoring than emergency departments. However, the accuracy of syndrome definitions used in surveillance has never been evaluated in community settings. This study's objective was to assess the accuracy of syndrome definitions based on diagnostic codes in physician claims for identifying 5 syndromes (fever, gastrointestinal, neurological, rash, and respiratory including influenza-like illness) in community clinics.We selected a random sample of 3,600 community-based primary care physicians who practiced in the fee-for-service system in the province of Quebec, Canada in 2005-2007. We randomly selected 10 visits per physician from their claims, stratifying on syndrome type and presence, diagnosis, and month. Double-blinded chart reviews were conducted by telephone with consenting physicians to obtain information on patient diagnoses for each sampled visit. The sensitivity, specificity, and positive predictive value (PPV) of physician claims were estimated by comparison to chart review.1,098 (30.5%) physicians completed the chart review. A chart entry on the date of the corresponding claim was found for 10,529 (95.9%) visits. The sensitivity of syndrome definitions based on diagnostic codes in physician claims was low, ranging from 0.11 (fever) to 0.44 (respiratory), the specificity was high, and the PPV was moderate to high, ranging from 0.59 (fever) to 0.85 (respiratory). We found that rarely used diagnostic codes had a higher probability of being false-positives, and that more commonly used diagnostic codes had a higher PPV.Future research should identify physician, patient, and encounter characteristics associated with the accuracy of diagnostic codes in physician claims. This would enable public health to improve syndromic surveillance, either by focusing on physician claims whose diagnostic code is more likely to be accurate, or by using all physician claims and weighing each according to the likelihood that its diagnostic code is accurate.

Dataset Information

Accuracy of claims-based algorithms for epilepsy research: Revealing the unseen performance of claims-based studies.

Publications

Accuracy of claims-based algorithms for epilepsy research: Revealing the unseen performance of claims-based studies.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets