Dataset Information

Challenges in risk estimation using routinely collected clinical data: The example of estimating cervical cancer risks from electronic health-records.

ABSTRACT: Electronic health-records (EHR) are increasingly used by epidemiologists studying disease following surveillance testing to provide evidence for screening intervals and referral guidelines. Although cost-effective, undiagnosed prevalent disease and interval censoring (in which asymptomatic disease is only observed at the time of testing) raise substantial analytic issues when estimating risk that cannot be addressed using Kaplan-Meier methods. Based on our experience analysing EHR from cervical cancer screening, we previously proposed the logistic-Weibull model to address these issues. Here we demonstrate how the choice of statistical method can impact risk estimates. We use observed data on 41,067 women in the cervical cancer screening program at Kaiser Permanente Northern California, 2003-2013, as well as simulations to evaluate the ability of different methods (Kaplan-Meier, Turnbull, Weibull and logistic-Weibull) to accurately estimate risk within a screening program. Cumulative risk estimates from the statistical methods varied considerably, with the largest differences occurring for prevalent disease risk when baseline disease ascertainment was random but incomplete. Kaplan-Meier underestimated risk at earlier times and overestimated risk at later times in the presence of interval censoring or undiagnosed prevalent disease. Turnbull performed well, though was inefficient and not smooth. The logistic-Weibull model performed well, except when event times didn't follow a Weibull distribution. We have demonstrated that methods for right-censored data, such as Kaplan-Meier, result in biased estimates of disease risks when applied to interval-censored data, such as screening programs using EHR data. The logistic-Weibull model is attractive, but the model fit must be checked against Turnbull non-parametric risk estimates.

SUBMITTER: Landy R

PROVIDER: S-EPMC5930038 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Challenges in risk estimation using routinely collected clinical data: The example of estimating cervical cancer risks from electronic health-records.

Landy Rebecca R Cheung Li C LC Schiffman Mark M Gage Julia C JC Hyun Noorie N Wentzensen Nicolas N Kinney Walter K WK Castle Philip E PE Fetterman Barbara B Poitras Nancy E NE Lorey Thomas T Sasieni Peter D PD Katki Hormuzd A HA

Preventive medicine 20171206

Electronic health-records (EHR) are increasingly used by epidemiologists studying disease following surveillance testing to provide evidence for screening intervals and referral guidelines. Although cost-effective, undiagnosed prevalent disease and interval censoring (in which asymptomatic disease is only observed at the time of testing) raise substantial analytic issues when estimating risk that cannot be addressed using Kaplan-Meier methods. Based on our experience analysing EHR from cervical ...[more]

PMID: 29222045

Similar Datasets

Project description:Importance:Inpatient violence remains a significant problem despite existing risk assessment methods. The lack of robustness and the high degree of effort needed to use current methods might be mitigated by using routinely registered clinical notes. Objective:To develop and validate a multivariable prediction model for assessing inpatient violence risk based on machine learning techniques applied to clinical notes written in patients' electronic health records. Design, Setting, and Participants:This prognostic study used retrospective clinical notes registered in electronic health records during admission at 2 independent psychiatric health care institutions in the Netherlands. No exclusion criteria for individual patients were defined. At site 1, all adults admitted between January 2013 and August 2018 were included, and at site 2 all adults admitted to general psychiatric wards between June 2016 and August 2018 were included. Data were analyzed between September 2018 and February 2019. Main Outcomes and Measures:Predictive validity and generalizability of prognostic models measured using area under the curve (AUC). Results:Clinical notes recorded during a total of 3189 admissions of 2209 unique individuals at site 1 (mean [SD] age, 34.0 [16.6] years; 1536 [48.2%] male) and 3253 admissions of 1919 unique individuals at site 2 (mean [SD] age, 45.9 [16.6] years; 2097 [64.5%] male) were analyzed. Violent outcome was determined using the Staff Observation Aggression Scale-Revised. Nested cross-validation was used to train and evaluate models that assess violence risk during the first 4 weeks of admission based on clinical notes available after 24 hours. The predictive validity of models was measured at site 1 (AUC = 0.797; 95% CI, 0.771-0.822) and site 2 (AUC = 0.764; 95% CI, 0.732-0.797). The validation of pretrained models in the other site resulted in AUCs of 0.722 (95% CI, 0.690-0.753) at site 1 and 0.643 (95% CI, 0.610-0.675) at site 2; the difference in AUCs between the internally trained model and the model trained on other-site data was significant at site 1 (AUC difference = 0.075; 95% CI, 0.045-0.105; P < .001) and site 2 (AUC difference = 0.121; 95% CI, 0.085-0.156; P < .001). Conclusions and Relevance:Internally validated predictions resulted in AUC values with good predictive validity, suggesting that automatic violence risk assessment using routinely registered clinical notes is possible. The validation of trained models using data from other sites corroborates previous findings that violence risk assessment generalizes modestly to different populations.

Project description:BackgroundHealth systems are undergoing widespread adoption of the collaborative chronic care model (CCM). Care structured around the CCM may reduce costly psychiatric hospitalizations. Little is known, however, about the time course or heterogeneity of treatment effects (HTE) for CCM on psychiatric hospitalization.RationaleAssessment of CCM implementation support on psychiatric hospitalization might be more efficient if the timing were informed by an expected time course. Further, understanding HTE could help determine who should be referred for intervention.Objectives(i) Estimate the trajectory of CCM effect on psychiatric hospitalization rates. (ii) Explore HTE for CCM across demographic and clinical characteristics.MethodsData from a stepped wedge CCM implementation trial were reanalyzed using 5 570 patients in CCM treatment and 46 443 patients receiving usual care. Time-to-event data was constructed from routine medical records. Effect trajectory of CCM on psychiatric hospitalization was simulated from an extended Cox model over one year of implementation support. Covariate risk contributions were estimated from subset stratified Cox models without using simulation. Ratios of hazard ratios (RHR) allowed comparison by trial arm for HTE analysis, also without simulation. No standard Cox proportional hazards models were used for either estimating the time-course or heterogeneity of treatment effect.ResultsThe effect of CCM implementation support increased most rapidly immediately after implementation start and grew more gradually throughout the rest of the study. On the final study day, psychiatric hospitalization rates in the treatment arm were 17% to 49% times lower than controls, with adjustment for all model covariates (HR 0.66; 95% CI 0.51-0.83). Our analysis of HTE favored usual care for those with a history of prior psychiatric hospitalization (RHR 4.92; 95% CI 3.15-7.7) but favored CCM for those with depression (RHR 0.61; 95% CI: 0.41-0.91). Having a single medical diagnosis, compared to having none, favored CCM (RHR 0.52; 95% CI 0.31-0.86).ConclusionReduction of psychiatric hospitalization is evident immediately after start of CCM implementation support, but assessments may be better timed once the effect size begins to stabilize, which may be as early as six months. HTE findings for CCM can guide future research on utility of CCM in specific populations.

Project description:IntroductionCardiovascular disease is a leading cause of death globally, responsible for nearly 18 million deaths worldwide in 2017. Medications to reduce the risk of cardiovascular events are prescribed based on evidence from clinical trials which explore treatment effects in an indicated sample of the general population. However, these results may not be fully generalisable because of trial eligibility criteria that generally restrict to younger patients with fewer comorbidities. Therefore, evidence of effectiveness of medications for groups underrepresented in clinical trials such as those aged ≥75 years, from ethnic minority backgrounds or with low kidney function may be limited.Using individual anonymised data from the Ongoing Telmisartan Alone and the Ramipril Global Endpoint Trial (ONTARGET) trial, in collaboration with the original trial investigators, we aim to investigate clinical trial replicability within a real-world setting in the area of cardiovascular disease. If the original trial results are replicable, we will estimate treatment effects and risk in groups underrepresented and excluded from the original clinical trial.Methods and analysisWe will develop a cohort analogous to the ONTARGET trial within the Clinical Practice Research Datalink between 1 January 2001 and 31 July 2019 using the trial eligibility criteria and propensity score matching. The primary outcome is a composite of cardiovascular death, non-fatal myocardial infarction, non-fatal stroke and hospitalisation for congestive heart failure. If results from the cohort study fall within pre-specified limits, we will expand the cohort to include under represented and excluded groups.Ethics and disseminationEthical approval has been granted by the London School of Hygiene & Tropical Medicine Ethics Committee (Ref: 22658). The study has been approved by the Independent Scientific Advisory Committee of the UK Medicines and Healthcare Products Regulatory Agency (protocol no. 20_012). Access to the individual patient data from the ONTARGET trial was obtained by the trial investigators. Findings will be submitted to peer-reviewed journals and presented at conferences.

Dataset Information

Challenges in risk estimation using routinely collected clinical data: The example of estimating cervical cancer risks from electronic health-records.

Publications

Challenges in risk estimation using routinely collected clinical data: The example of estimating cervical cancer risks from electronic health-records.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets