Dataset Information

Improving Cervical Precancer Surveillance: Validity of Claims-Based Prediction Models in ICD-9 and ICD-10 Eras.

ABSTRACT:

Background

Human papillomavirus vaccine (HPV) impact on cervical precancer (cervical intraepithelial neoplasia grades 2+ [CIN2+]) is observable sooner than impact on cancer. Biopsy-confirmed CIN2+ is not included in most US cancer registries. Billing codes could provide surrogate metrics; however, the International Classification of Diseases, ninth (ICD-9) to tenth (ICD-10) transition disrupts trends. We built, validated, and compared claims-based models to identify CIN2+ events in both ICD eras.

Methods

A database of Davidson County (Nashville), Tennessee, pathology-confirmed CIN2+ from the HPV Vaccine Impact Monitoring Project (HPV-IMPACT) provided gold standard events. Using Tennessee Medicaid 2008-2017, cervical diagnostic procedures (N?=?8549) among Davidson County women aged 18-39?years were randomly split into 60% training and 40% testing sets. Relevant diagnosis, procedure, and screening codes were used to build models from CIN2+ tissue diagnosis codes alone, least absolute shrinkage and selection operator (LASSO), and random forest. Model-classified index events were counted to estimate incident events.

Results

HPV-IMPACT identified 983 incident CIN2+ events. Models identified 1007 (LASSO), 1245 (CIN2+ tissue diagnosis codes alone), and 957 (random forest) incident events. LASSO performed well in ICD-9 and ICD-10 eras: 77.3% (95% confidence interval [CI] = 72.5% to 81.5%) vs 81.1% (95% CI?=?71.5% to 88.6%) sensitivity, 93.0% (95% CI?=?91.9% to 94.0%) vs 90.2% (95% CI?=?87.2% to 92.7%) specificity, 61.3% (95% CI?=?56.6% to 65.8%) vs 60.3% (95% CI?=?51.0% to 69.1%) positive predictive value, 96.6% (95% CI?=?95.8% to 97.3%) vs 96.3% (95% CI?=?94.1% to 97.8%) negative predictive value, 91.0% (95% CI?=?89.9% to 92.1%) vs 88.8% (95% CI?=?85.9% to 91.2%) accuracy, and 85.1% (95% CI?=?82.9% to 87.4%) vs 85.6% (95% CI?=?81.4% to 89.9%) C-indices, respectively; performance did not statistically significantly differ between eras (95% confidence intervals all overlapped).

Conclusions

Results confirmed model utility with good performance across both ICD eras for CIN2+ surveillance. Validated claims-based models may be used in future CIN2+ trend analyses to estimate HPV vaccine impact where population-based biopsies are unavailable.

SUBMITTER: Shing JZ

PROVIDER: S-EPMC7853170 | biostudies-literature | 2021 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Improving Cervical Precancer Surveillance: Validity of Claims-Based Prediction Models in ICD-9 and ICD-10 Eras.

Shing Jaimie Z JZ Griffin Marie R MR Nguyen Linh D LD Slaughter James C JC Mitchel Edward F EF Pemmaraju Manideepthi M Rentuza Alyssa B AB Hull Pamela C PC

JNCI cancer spectrum 20201230 1

<h4>Background</h4>Human papillomavirus vaccine (HPV) impact on cervical precancer (cervical intraepithelial neoplasia grades 2+ [CIN2+]) is observable sooner than impact on cancer. Biopsy-confirmed CIN2+ is not included in most US cancer registries. Billing codes could provide surrogate metrics; however, the International Classification of Diseases, ninth (ICD-9) to tenth (ICD-10) transition disrupts trends. We built, validated, and compared claims-based models to identify CIN2+ events in both ...[more]

PMID: 33554035

Similar Datasets

Project description:BACKGROUND: Administrative data are widely used to study health systems and make important health policy decisions. Yet little is known about the influence of coder characteristics on administrative data validity in these studies. Our goal was to describe the relationship between several measures of validity in coded hospital discharge data and 1) coders' volume of coding (> or = 13,000 vs. <13,000 records), 2) coders' employment status (full- vs. part-time), and 3) hospital type. METHODS: This descriptive study examined 6 indicators of face validity in ICD-10 coded discharge records from 4 hospitals in Calgary, Canada between April 2002 and March 2007. Specifically, mean number of coded diagnoses, procedures, complications, Z-codes, and codes ending in 8 or 9 were compared by coding volume and employment status, as well as hospital type. The mean number of diagnoses was also compared across coder characteristics for 6 major conditions of varying complexity. Next, kappa statistics were computed to assess agreement between discharge data and linked chart data reabstracted by nursing chart reviewers. Kappas were compared across coder characteristics. RESULTS: 422,618 discharge records were coded by 59 coders during the study period. The mean number of diagnoses per record decreased from 5.2 in 2002/2003 to 3.9 in 2006/2007, while the number of records coded annually increased from 69,613 to 102,842. Coders at the tertiary hospital coded the most diagnoses (5.0 compared with 3.9 and 3.8 at other sites). There was no variation by coder or site characteristics for any other face validity indicator. The mean number of diagnoses increased from 1.5 to 7.9 with increasing complexity of the major diagnosis, but did not vary with coder characteristics. Agreement (kappa) between coded data and chart review did not show any consistent pattern with respect to coder characteristics. CONCLUSIONS: This large study suggests that coder characteristics do not influence the validity of hospital discharge data. Other jurisdictions might benefit from implementing similar employment programs to ours, e.g.: a requirement for a 2-year college training program, a single management structure across sites, and rotation of coders between sites. Limitations include few coder characteristics available for study due to privacy concerns.

Project description:ObjectiveTo evaluate the accuracy of ICD-10-CM claims-based definitions for epilepsy and classifying seizure types in the outpatient setting.MethodsWe reviewed electronic health records (EHR) for a cohort of adults aged 18+ years seen by six neurologists who had an outpatient visit at a level 4 epilepsy center between 01/2019-09/2019. The neurologists used a standardized documentation template to capture the diagnosis of epilepsy (yes/no/unsure), seizure type (focal/generalized/unknown), and seizure frequency in the EHR. Using linked ICD-10-CM codes assigned by the provider, we assessed the accuracy of claims-based definitions for epilepsy, focal seizure type, and generalized seizure type against the reference-standard EHR documentation by estimating sensitivity (Sn), specificity (Sp), positive predictive value (PPV), and negative predictive value (NPV).ResultsThere were 673 eligible outpatient encounters. After review of EHRs for standardized documentation, an analytic sample consisted of 520 encounters representing 402 unique patients. In the EHR documentation, 93.5 % (n = 486/520) of encounters were with patients with a diagnosis of epilepsy. Of those, 66.0 % (n = 321/486) had ≥1 focal seizure, 41.6 % (n = 202/486) had ≥1 generalized seizure, and 7% (n = 34/486) had ≥1 unknown seizure. An ICD-10-CM definition for epilepsy (i.e., ICD-10 G40.X) achieved Sn = 84.4 % (95 % CI 80.8-87.5%), Sp = 79.4 % (95 % CI 62.1-91.3%), PPV = 98.3 % (95 % CI 96.6-99.3%), and NPV = 26.2 % (95 % CI 18.0-35.8%). The classification of focal vs generalized/unknown seizures achieved Sn = 69.8 % (95 % CI 64.4-74.8%), Sp = 79.4 % (95 % CI 72.4-85.3%), PPV = 86.8 % (95 % CI 82.1-90.7%), and NPV = 57.5 % (95 % CI 50.8-64.0%).ConclusionsClaims-based definitions using groups of ICD-10-CM codes assigned by neurologists in routine outpatient clinic visits at a level 4 epilepsy center performed well in discriminating between patients with and without a diagnosis of epilepsy and between seizure types.

Project description:Background:The International Classification of Diseases, Ninth Edition, Clinical Modification (ICD-9-CM) Injury Severity Score (ICISS) is a risk adjustment model when injuries are recorded using ICD-9-CM coding. The trauma mortality prediction model (TMPM-ICD9) provides better calibration and discrimination compared with ICISS and injury severity score (ISS). Though TMPM-ICD9 is statistically rigorous, it is not precise enough mathematically and has the tendency to overestimate injury severity. The purpose of this study is to develop a new ICD-10-CM injury model which estimates injury severities for every injury in the ICD-10-CM lexicon by a combination of rigorous statistical probit models and mathematical properties and improves the prediction accuracy. Methods:We developed an injury mortality prediction (IMP-ICDX) using data of 794,098 patients admitted to 738 hospitals in the National Trauma Data Bank from 2015 to 2016. Empiric measures of severity for each of the trauma ICD-10-CM codes were estimated using a weighted median death probability (WMDP) measurement and then used as the basis for IMP-ICDX. ISS (version 2005) and the single worst injury (SWI) model were re-estimated. The performance of each of these models was compared by using the area under the receiver operating characteristic (AUC), the Hosmer-Lemeshow (HL) statistic, and the Akaike information criterion statistic. Results:IMP-ICDX exhibits significantly better discrimination (AUCIMP-ICDX, 0.893, and 95% confidence interval (CI), 0.887 to 0.898; AUCISS, 0.853, and 95% CI, 0.846 to 0.860; and AUCSWI, 0.886, and 95% CI, 0.881 to 0.892) and calibration (HLIMP-ICDX, 68, and 95% CI, 36 to 98; HLISS, 252, and 95% CI, 191 to 310; and HLSWI, 92, and 95% CI, 53 to 128) compared with ISS and SWI. All models were improved after the extension of age, gender, and injury mechanism, but the augmented IMP-ICDX still dominated ISS and SWI by every performance. Conclusions:The IMP-ICDX has a better discrimination and calibration compared to ISS. Therefore, we believe that IMP-ICDX could be a new viable trauma research assessment method.

Project description:BackgroundAccurate identification of new diagnoses of human papillomavirus-associated cancers and precancers is an important step toward the development of strategies that optimize the use of human papillomavirus vaccines. The diagnosis of human papillomavirus cancers hinges on a histopathologic report, which is typically stored in electronic medical records as free-form, or unstructured, narrative text. Previous efforts to perform surveillance for human papillomavirus cancers have relied on the manual review of pathology reports to extract diagnostic information, a process that is both labor- and resource-intensive. Natural language processing can be used to automate the structuring and extraction of clinical data from unstructured narrative text in medical records and may provide a practical and effective method for identifying patients with vaccine-preventable human papillomavirus disease for surveillance and research.ObjectiveThis study's objective was to develop and assess the accuracy of a natural language processing algorithm for the identification of individuals with cancer or precancer of the cervix and anus.MethodsA pipeline-based natural language processing algorithm was developed, which incorporated machine learning and rule-based methods to extract diagnostic elements from the narrative pathology reports. To test the algorithm's classification accuracy, we used a split-validation study design. Full-length cervical and anal pathology reports were randomly selected from 4 clinical pathology laboratories. Two study team members, blinded to the classifications produced by the natural language processing algorithm, manually and independently reviewed all reports and classified them at the document level according to 2 domains (diagnosis and human papillomavirus testing results). Using the manual review as the gold standard, the algorithm's performance was evaluated using standard measurements of accuracy, recall, precision, and F-measure.ResultsThe natural language processing algorithm's performance was validated on 949 pathology reports. The algorithm demonstrated accurate identification of abnormal cytology, histology, and positive human papillomavirus tests with accuracies greater than 0.91. Precision was lowest for anal histology reports (0.87, 95% CI 0.59-0.98) and highest for cervical cytology (0.98, 95% CI 0.95-0.99). The natural language processing algorithm missed 2 out of the 15 abnormal anal histology reports, which led to a relatively low recall (0.68, 95% CI 0.43-0.87).ConclusionsThis study outlines the development and validation of a freely available and easily implementable natural language processing algorithm that can automate the extraction and classification of clinical data from cervical and anal cytology and histology.

Dataset Information

Improving Cervical Precancer Surveillance: Validity of Claims-Based Prediction Models in ICD-9 and ICD-10 Eras.

Background

Methods

Results

Conclusions

Publications

Improving Cervical Precancer Surveillance: Validity of Claims-Based Prediction Models in ICD-9 and ICD-10 Eras.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets