Dataset Information

Do population-level risk prediction models that use routinely collected health data reliably predict individual risks?

ABSTRACT: The objective of this study was to assess the reliability of individual risk predictions based on routinely collected data considering the heterogeneity between clinical sites in data and populations. Cardiovascular disease (CVD) risk prediction with QRISK3 was used as exemplar. The study included 3.6 million patients in 392 sites from the Clinical Practice Research Datalink. Cox models with QRISK3 predictors and a frailty (random effect) term for each site were used to incorporate unmeasured site variability. There was considerable variation in data recording between general practices (missingness of body mass index ranged from 18.7% to 60.1%). Incidence rates varied considerably between practices (from 0.4 to 1.3 CVD events per 100 patient-years). Individual CVD risk predictions with the random effect model were inconsistent with the QRISK3 predictions. For patients with QRISK3 predicted risk of 10%, the 95% range of predicted risks were between 7.2% and 13.7% with the random effects model. Random variability only explained a small part of this. The random effects model was equivalent to QRISK3 for discrimination and calibration. Risk prediction models based on routinely collected health data perform well for populations but with great uncertainty for individuals. Clinicians and patients need to understand this uncertainty.

SUBMITTER: Li Y

PROVIDER: S-EPMC6677736 | biostudies-literature | 2019 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Do population-level risk prediction models that use routinely collected health data reliably predict individual risks?

Li Yan Y Li Yan Y Sperrin Matthew M Belmonte Miguel M Pate Alexander A Ashcroft Darren M DM van Staa Tjeerd Pieter TP

Scientific reports 20190802 1

The objective of this study was to assess the reliability of individual risk predictions based on routinely collected data considering the heterogeneity between clinical sites in data and populations. Cardiovascular disease (CVD) risk prediction with QRISK3 was used as exemplar. The study included 3.6 million patients in 392 sites from the Clinical Practice Research Datalink. Cox models with QRISK3 predictors and a frailty (random effect) term for each site were used to incorporate unmeasured si ...[more]

PMID: 31375726

Similar Datasets

Project description:BackgroundFalls in older adults remain a pressing health concern. With advancements in data analytics and increasing uptake of electronic health records, developing comprehensive predictive models for fall risk is now possible. We aimed to systematically identify studies involving the development and implementation of predictive falls models which used routinely collected electronic health record data in home-based, community and residential aged care settings.MethodsA systematic search of entries in Cochrane Library, CINAHL, MEDLINE, Scopus, and Web of Science was conducted in July 2020 using search terms relevant to aged care, prediction, and falls. Selection criteria included English-language studies, published in peer-reviewed journals, had an outcome of falls, and involved fall risk modelling using routinely collected electronic health record data. Screening, data extraction and quality appraisal using the Critical Appraisal Skills Program for Clinical Prediction Rule Studies were conducted. Study content was synthesised and reported narratively.ResultsFrom 7,329 unique entries, four relevant studies were identified. All predictive models were built using different statistical techniques. Predictors across seven categories were used: demographics, assessments of care, fall history, medication use, health conditions, physical abilities, and environmental factors. Only one of the four studies had been validated externally. Three studies reported on the performance of the models.ConclusionsAdopting predictive modelling in aged care services for adverse events, such as falls, is in its infancy. The increased availability of electronic health record data and the potential of predictive modelling to document fall risk and inform appropriate interventions is making use of such models achievable. Having a dynamic prediction model that reflects the changing status of an aged care client is key to this moving forward for fall prevention interventions.

Project description:BackgroundAdvancing the use of genomic data with routinely collected health data holds great promise for health care and research. Increasing the use of these data is a high priority to understand and address the causes of disease.ObjectiveThis study aims to provide an outline of the use of genomic data alongside routinely collected data in health research to date. As this field prepares to move forward, it is important to take stock of the current state of play in order to highlight new avenues for development, identify challenges, and ensure that adequate data governance models are in place for safe and socially acceptable progress.MethodsWe conducted a literature review to draw information from past studies that have used genomic and routinely collected data and conducted interviews with individuals who use these data for health research. We collected data on the following: the rationale of using genomic data in conjunction with routinely collected data, types of genomic and routinely collected data used, data sources, project approvals, governance and access models, and challenges encountered.ResultsThe main purpose of using genomic and routinely collected data was to conduct genome-wide and phenome-wide association studies. Routine data sources included electronic health records, disease and death registries, health insurance systems, and deprivation indices. The types of genomic data included polygenic risk scores, single nucleotide polymorphisms, and measures of genetic activity, and biobanks generally provided these data. Although the literature search showed that biobanks released data to researchers, the case studies revealed a growing tendency for use within a data safe haven. Challenges of working with these data revolved around data collection, data storage, technical, and data privacy issues.ConclusionsUsing genomic and routinely collected data holds great promise for progressing health research. Several challenges are involved, particularly in terms of privacy. Overcoming these barriers will ensure that the use of these data to progress health research can be exploited to its full potential.

Project description:BackgroundIntensive care units (ICUs) face financial, bed management, and staffing constraints. Detailed data covering all aspects of patients' journeys into and through intensive care are now collected and stored in electronic health records: machine learning has been used to analyse such data in order to provide decision support to clinicians.MethodsSystematic review of the applications of machine learning to routinely collected ICU data. Web of Science and MEDLINE databases were searched to identify candidate articles: those on image processing were excluded. The study aim, the type of machine learning used, the size of dataset analysed, whether and how the model was validated, and measures of predictive accuracy were extracted.ResultsOf 2450 papers identified, 258 fulfilled eligibility criteria. The most common study aims were predicting complications (77 papers [29.8% of studies]), predicting mortality (70 [27.1%]), improving prognostic models (43 [16.7%]), and classifying sub-populations (29 [11.2%]). Median sample size was 488 (IQR 108-4099): 41 studies analysed data on > 10,000 patients. Analyses focused on 169 (65.5%) papers that used machine learning to predict complications, mortality, length of stay, or improvement of health. Predictions were validated in 161 (95.2%) of these studies: the area under the ROC curve (AUC) was reported by 97 (60.2%) but only 10 (6.2%) validated predictions using independent data. The median AUC was 0.83 in studies of 1000-10,000 patients, rising to 0.94 in studies of > 100,000 patients. The most common machine learning methods were neural networks (72 studies [42.6%]), support vector machines (40 [23.7%]), and classification/decision trees (34 [20.1%]). Since 2015 (125 studies [48.4%]), the most common methods were support vector machines (37 studies [29.6%]) and random forests (29 [23.2%]).ConclusionsThe rate of publication of studies using machine learning to analyse routinely collected ICU data is increasing rapidly. The sample sizes used in many published studies are too small to exploit the potential of these methods. Methodological and reporting guidelines are needed, particularly with regard to the choice of method and validation of predictions, to increase confidence in reported findings and aid in translating findings towards routine use in clinical practice.

Dataset Information

Do population-level risk prediction models that use routinely collected health data reliably predict individual risks?

Publications

Do population-level risk prediction models that use routinely collected health data reliably predict individual risks?

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets