Risk prediction for heterogeneous populations with application to hospital admission prediction.
Ontology highlight
ABSTRACT:
SUBMITTER: Huling JD
PROVIDER: S-EPMC6076875 | biostudies-literature | 2018 Jun
REPOSITORIES: biostudies-literature
Publications
Risk prediction for heterogeneous populations with application to hospital admission prediction.
Huling Jared D JD Yu Menggang M Liang Muxuan M Smith Maureen M
Biometrics 20171026 2
This article is motivated by the increasing need to model risk for large hospital and health care systems that provide services to diverse and complex patients. Often, heterogeneity across a population is determined by a set of factors such as chronic conditions. When these stratifying factors result in overlapping subpopulations, it is likely that the covariate effects for the overlapping groups have some similarity. We exploit this similarity by imposing structural constraints on the importanc ...[more]
Similar Datasets
Project description:BackgroundTelemonitoring of symptoms and physiological signs has been suggested as a means of early detection of chronic obstructive pulmonary disease (COPD) exacerbations, with a view to instituting timely treatment. However, algorithms to identify exacerbations result in frequent false-positive results and increased workload. Machine learning, when applied to predictive modelling, can determine patterns of risk factors useful for improving prediction quality.ObjectiveOur objectives were to (1) establish whether machine learning techniques applied to telemonitoring datasets improve prediction of hospital admissions and decisions to start corticosteroids, and (2) determine whether the addition of weather data further improves such predictions.MethodsWe used daily symptoms, physiological measures, and medication data, with baseline demography, COPD severity, quality of life, and hospital admissions from a pilot and large randomized controlled trial of telemonitoring in COPD. We linked weather data from the United Kingdom meteorological service. We used feature selection and extraction techniques for time series to construct up to 153 predictive patterns (features) from symptom, medication, and physiological measurements. We used the resulting variables to construct predictive models fitted to training sets of patients and compared them with common symptom-counting algorithms.ResultsWe had a mean 363 days of telemonitoring data from 135 patients. The two most practical traditional score-counting algorithms, restricted to cases with complete data, resulted in area under the receiver operating characteristic curve (AUC) estimates of 0.60 (95% CI 0.51-0.69) and 0.58 (95% CI 0.50-0.67) for predicting admissions based on a single day's readings. However, in a real-world scenario allowing for missing data, with greater numbers of patient daily data and hospitalizations (N=57,150, N+=55, respectively), the performance of all the traditional algorithms fell, including those based on 2 days' data. One of the most frequently used algorithms performed no better than chance. All considered machine learning models demonstrated significant improvements; the best machine learning algorithm based on 57,150 episodes resulted in an aggregated AUC of 0.74 (95% CI 0.67-0.80). Adding weather data measurements did not improve the predictive performance of the best model (AUC 0.74, 95% CI 0.69-0.79). To achieve an 80% true-positive rate (sensitivity), the traditional algorithms were associated with an 80% false-positive rate: our algorithm halved this rate to approximately 40% (specificity approximately 60%). The machine learning algorithm was moderately superior to the best symptom-counting algorithm (AUC 0.77, 95% CI 0.74-0.79 vs AUC 0.66, 95% CI 0.63-0.68) at predicting the need for corticosteroids.ConclusionsEarly detection and management of COPD remains an important goal given its huge personal and economic costs. Machine learning approaches, which can be tailored to an individual's baseline profile and can learn from experience of the individual patient, are superior to existing predictive algorithms and show promise in achieving this goal.Trial registrationInternational Standard Randomized Controlled Trial Number ISRCTN96634935; http://www.isrctn.com/ISRCTN96634935 (Archived by WebCite at http://www.webcitation.org/722YkuhAz).
| S-EPMC6231768 | biostudies-literature
Project description:Genomic prediction is a useful tool to accelerate genetic gain in selection using DNA marker information. However, this technology typically relies on standard prediction procedures, such as genomic BLUP, that are not designed to accommodate population heterogeneity resulting from differences in marker effects across populations. In this study, we assayed different prediction procedures to capture marker-by-population interactions in genomic prediction models. Prediction procedures included genomic BLUP and two kernel-based extensions of genomic BLUP which explicitly accounted for population heterogeneity. To model population heterogeneity, dissemblance between populations was either depicted by a unique coefficient (as previously reported), or a more flexible function of genetic distance between populations (proposed herein). Models under investigation were applied in a diverse switchgrass sample under two validation schemes: whole-sample calibration, where all individuals except selection candidates are included in the calibration set, and cross-population calibration, where the target population is entirely excluded from the calibration set. First, we showed that using fixed effects, from principal components or putative population groups, appeared detrimental to prediction accuracy, especially in cross-population calibration. Then we showed that modeling population heterogeneity by our proposed procedure resulted in highly significant improvements in model fit. In such cases, gains in accuracy were often positive. These results suggest that population heterogeneity may be parsimoniously captured by kernel methods. However, in cases where improvement in model fit by our proposed procedure is null-to-moderate, ignoring heterogeneity should probably be preferred due to the robustness and simplicity of the standard genomic BLUP model.
| S-EPMC6404615 | biostudies-other
Project description:BackgroundRisk prediction models have been developed to identify those at increased risk for emergency admissions, which could facilitate targeted interventions in primary care to prevent these events.ObjectiveSystematic review of validated risk prediction models for predicting emergency hospital admissions in community-dwelling adults.MethodsA systematic literature review and narrative analysis was conducted. Inclusion criteria were as follows;Populationcommunity-dwelling adults (aged 18 years and above); Risk: risk prediction models, not contingent on an index hospital admission, with a derivation and ?1 validation cohort;Primary outcomeemergency hospital admission (defined as unplanned overnight stay in hospital);Study designretrospective or prospective cohort studies.ResultsOf 18,983 records reviewed, 27 unique risk prediction models met the inclusion criteria. Eleven were developed in the United States, 11 in the United Kingdom, 3 in Italy, 1 in Spain, and 1 in Canada. Nine models were derived using self-report data, and the remainder (n=18) used routine administrative or clinical record data. Total study sample sizes ranged from 96 to 4.7 million participants. Predictor variables most frequently included in models were: (1) named medical diagnoses (n=23); (2) age (n=23); (3) prior emergency admission (n=22); and (4) sex (n=18). Eleven models included nonmedical factors, such as functional status and social supports. Regarding predictive accuracy, models developed using administrative or clinical record data tended to perform better than those developed using self-report data (c statistics 0.63-0.83 vs. 0.61-0.74, respectively). Six models reported c statistics of >0.8, indicating good performance. All 6 included variables for prior health care utilization, multimorbidity or polypharmacy, and named medical diagnoses or prescribed medications. Three predicted admissions regarded as being ambulatory care sensitive.ConclusionsThis study suggests that risk models developed using administrative or clinical record data tend to perform better. In applying a risk prediction model to a new population, careful consideration needs to be given to the purpose of its use and local factors.
| S-EPMC4219489 | biostudies-literature
Project description:BACKGROUND:Congestive heart failure is one of the most common reasons those aged 65 and over are hospitalized in the United States, which has caused a considerable economic burden. The precise prediction of hospitalization caused by congestive heart failure in the near future could prevent possible hospitalization, optimize the medical resources, and better meet the healthcare needs of patients. METHODS:To fully utilize the monthly-updated claim feed data released by The Centers for Medicare and Medicaid Services (CMS), we present a dynamic random survival forest model adapted for periodically updated data to predict the risk of adverse events. We apply our model to dynamically predict the risk of hospital admission among patients with congestive heart failure identified using the Accountable Care Organization Operational System Claim and Claim Line Feed data from Feb 2014 to Sep 2015. We benchmark the proposed model with two commonly used models in medical application literature: the cox proportional model and logistic regression model with L-1 norm penalty. RESULTS:Results show that our model has high Area-Under-the-ROC-Curve across time points and C-statistics. In addition to the high performance, it provides measures of variable importance and individual-level instant risk. CONCLUSION:We present an efficient model adapted for periodically updated data such as the monthly updated claim feed data released by CMS to predict the risk of hospitalization. In addition to processing big-volume periodically updated stream-like data, our model can capture event onset information and time-to-event information, incorporate time-varying features, provide insights of variable importance and have good prediction power. To the best of our knowledge, it is the first work combining sliding window technique with the random survival forest model. The model achieves remarkable performance and could be easily deployed to monitor patients in real time.
| S-EPMC6354329 | biostudies-literature
Project description:OBJECTIVES:Evaluating whether future studies to develop prediction models for early readmissions based on health insurance claims data available at the time of a hospitalisation are worthwhile. DESIGN:Retrospective cohort study of hospital admissions with discharge dates between 1 January 2014 and 31 December 2016. SETTING:All-cause acute care hospital admissions in the general population of Switzerland, enrolled in the Helsana Group, a large provider of Swiss mandatory health insurance. PARTICIPANTS:The mean age of 138 222 hospitalised adults included in the study was 60.5 years. Patients were included only with their first index hospitalisation. Patients who deceased during the follow-up period were excluded, as well as patients admitted from and/or discharged to nursing homes or rehabilitation clinics. MEASURES:The primary outcome was 30-day readmission rate. Area under the receiver operating characteristic curve (AUC) was used to measure the discrimination of the developed logistic regression prediction model. Candidate variables were theory based and derived from a systematic literature search. RESULTS:We observed a 30-day readmission rate of 7.5%. Fifty-five candidate variables were identified. The final model included pharmacy-based cost group (PCG) cancer, PCG cardiac disease, PCG pain, emergency index admission, number of emergency visits, costs specialists, costs hospital outpatient, costs laboratory, costs therapeutic devices, costs physiotherapy, number of outpatient visits, sex, age group and geographical region as predictors. The prediction model achieved an AUC of 0.60 (95% CI 0.60 to 0.61). CONCLUSIONS:Based on the results of our study, it is not promising to invest resources in large-scale studies for the development of prediction tools for hospital readmissions based on health insurance claims data available at admission. The data proved appropriate to investigate the occurrence of hospitalisations and subsequent readmissions, but we did not find evidence for the potential of a clinically helpful prediction tool based on patient-sided variables alone.
| S-EPMC6609042 | biostudies-literature
Project description:ImportanceDelirium at admission is associated with increased hospital morbidity and mortality, but it may be missed in up to 70% of cases. Use of a predictive algorithm in an electronic medical record (EMR) system could provide critical information to target assessment of those with delirium at admission.ObjectivesTo develop and assess a prediction rule for delirium using 2 populations of veterans and compare this rule with previously confirmed rules.Design, setting, and participantsIn a diagnostic study, randomly selected EMRs of hospitalized veterans from the Veterans Affairs (VA) External Peer Review Program at 118 VA medical centers with inpatient facilities were reviewed for delirium risk factors associated with the National Institute for Health and Clinical Excellence (NICE) delirium rule in a derivation cohort (October 1, 2012, to September 30, 2013) and a confirmation cohort (October 1, 2013, to March 31, 2014). Delirium within 24 hours of admission was identified using key word terms. A total of 39 377 veterans 65 years or older who were admitted to a VA medical center for congestive heart failure, acute coronary syndrome, community-acquired pneumonia, and chronic obstructive pulmonary disease were included in the study.ExposureThe EMR calculated delirium risk.Main outcomes and measuresDelirium at admission as identified by trained nurse reviewers was the main outcome measure. Random forest methods were used to identify accurate risk factors for prevalent delirium. A prediction rule for prevalent delirium was developed, and its diagnostic accuracy was tested in the confirmation cohort. This consolidated NICE rule was compared with previously confirmed scoring algorithms (electronic NICE and Pendlebury NICE).ResultsA total of 27?625 patients were included in the derivation cohort (28 118 [92.2%] male; mean [SD] age, 75.95 [8.61] years) and 11 752 in the confirmation cohort (11 536 [98.2%] male; mean [SD] age, 75.43 [8.55] years). Delirium at admission was identified in 2343 patients (8.5%) in the derivation cohort and 882 patients (7.0%) in the confirmation cohort. Modeling techniques identified cognitive impairment, infection, sodium level, and age of 80 years or older as the dominant risk factors. The consolidated NICE rule (area under the receiver operating characteristic [AUROC] curve, 0.91; 95% CI, 0.91-0.92; P?<?.001) had significantly higher discriminatory function than the eNICE rule (AUROC curve, 0.81; 95% CI, 0.80-0.82; P?<?.001) or Pendlebury NICE rule (AUROC curve, 0.87; 95% CI, 0.86-0.88; P?<?.001). These findings were confirmed in the confirmation cohort.Conclusions and relevanceThis analysis identified preexisting cognitive impairment, infection, sodium level, and age of 80 years or older as delirium screening targets. Use of this algorithm in an EMR system could direct clinical assessment efforts to patients with delirium at admission.
| S-EPMC6324279 | biostudies-literature
Project description:ObjectiveTo derive and validate a risk prediction algorithm to estimate hospital admission and mortality outcomes from coronavirus disease 2019 (covid-19) in adults.DesignPopulation based cohort study.Setting and participantsQResearch database, comprising 1205 general practices in England with linkage to covid-19 test results, Hospital Episode Statistics, and death registry data. 6.08 million adults aged 19-100 years were included in the derivation dataset and 2.17 million in the validation dataset. The derivation and first validation cohort period was 24 January 2020 to 30 April 2020. The second temporal validation cohort covered the period 1 May 2020 to 30 June 2020.Main outcome measuresThe primary outcome was time to death from covid-19, defined as death due to confirmed or suspected covid-19 as per the death certification or death occurring in a person with confirmed severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection in the period 24 January to 30 April 2020. The secondary outcome was time to hospital admission with confirmed SARS-CoV-2 infection. Models were fitted in the derivation cohort to derive risk equations using a range of predictor variables. Performance, including measures of discrimination and calibration, was evaluated in each validation time period.Results4384 deaths from covid-19 occurred in the derivation cohort during follow-up and 1722 in the first validation cohort period and 621 in the second validation cohort period. The final risk algorithms included age, ethnicity, deprivation, body mass index, and a range of comorbidities. The algorithm had good calibration in the first validation cohort. For deaths from covid-19 in men, it explained 73.1% (95% confidence interval 71.9% to 74.3%) of the variation in time to death (R2); the D statistic was 3.37 (95% confidence interval 3.27 to 3.47), and Harrell's C was 0.928 (0.919 to 0.938). Similar results were obtained for women, for both outcomes, and in both time periods. In the top 5% of patients with the highest predicted risks of death, the sensitivity for identifying deaths within 97 days was 75.7%. People in the top 20% of predicted risk of death accounted for 94% of all deaths from covid-19.ConclusionThe QCOVID population based risk algorithm performed well, showing very high levels of discrimination for deaths and hospital admissions due to covid-19. The absolute risks presented, however, will change over time in line with the prevailing SARS-C0V-2 infection rate and the extent of social distancing measures in place, so they should be interpreted with caution. The model can be recalibrated for different time periods, however, and has the potential to be dynamically updated as the pandemic evolves.
| S-EPMC7574532 | biostudies-literature
Project description:ObjectivesTo derive and validate risk prediction algorithms to estimate the risk of covid-19 related mortality and hospital admission in UK adults after one or two doses of covid-19 vaccination.DesignProspective, population based cohort study using the QResearch database linked to data on covid-19 vaccination, SARS-CoV-2 results, hospital admissions, systemic anticancer treatment, radiotherapy, and the national death and cancer registries.SettingsAdults aged 19-100 years with one or two doses of covid-19 vaccination between 8 December 2020 and 15 June 2021.Main outcome measuresPrimary outcome was covid-19 related death. Secondary outcome was covid-19 related hospital admission. Outcomes were assessed from 14 days after each vaccination dose. Models were fitted in the derivation cohort to derive risk equations using a range of predictor variables. Performance was evaluated in a separate validation cohort of general practices.ResultsOf 6 952 440 vaccinated patients in the derivation cohort, 5 150 310 (74.1%) had two vaccine doses. Of 2031 covid-19 deaths and 1929 covid-19 hospital admissions, 81 deaths (4.0%) and 71 admissions (3.7%) occurred 14 days or more after the second vaccine dose. The risk algorithms included age, sex, ethnic origin, deprivation, body mass index, a range of comorbidities, and SARS-CoV-2 infection rate. Incidence of covid-19 mortality increased with age and deprivation, male sex, and Indian and Pakistani ethnic origin. Cause specific hazard ratios were highest for patients with Down's syndrome (12.7-fold increase), kidney transplantation (8.1-fold), sickle cell disease (7.7-fold), care home residency (4.1-fold), chemotherapy (4.3-fold), HIV/AIDS (3.3-fold), liver cirrhosis (3.0-fold), neurological conditions (2.6-fold), recent bone marrow transplantation or a solid organ transplantation ever (2.5-fold), dementia (2.2-fold), and Parkinson's disease (2.2-fold). Other conditions with increased risk (ranging from 1.2-fold to 2.0-fold increases) included chronic kidney disease, blood cancer, epilepsy, chronic obstructive pulmonary disease, coronary heart disease, stroke, atrial fibrillation, heart failure, thromboembolism, peripheral vascular disease, and type 2 diabetes. A similar pattern of associations was seen for covid-19 related hospital admissions. No evidence indicated that associations differed after the second dose, although absolute risks were reduced. The risk algorithm explained 74.1% (95% confidence interval 71.1% to 77.0%) of the variation in time to covid-19 death in the validation cohort. Discrimination was high, with a D statistic of 3.46 (95% confidence interval 3.19 to 3.73) and C statistic of 92.5. Performance was similar after each vaccine dose. In the top 5% of patients with the highest predicted covid-19 mortality risk, sensitivity for identifying covid-19 deaths within 70 days was 78.7%.ConclusionThis population based risk algorithm performed well showing high levels of discrimination for identifying those patients at highest risk of covid-19 related death and hospital admission after vaccination.
| S-EPMC8446717 | biostudies-literature
Project description:Machine learning for hospital operations is under-studied. We present a prediction pipeline that uses live electronic health-records for patients in a UK teaching hospital's emergency department (ED) to generate short-term, probabilistic forecasts of emergency admissions. A set of XGBoost classifiers applied to 109,465 ED visits yielded AUROCs from 0.82 to 0.90 depending on elapsed visit-time at the point of prediction. Patient-level probabilities of admission were aggregated to forecast the number of admissions among current ED patients and, incorporating patients yet to arrive, total emergency admissions within specified time-windows. The pipeline gave a mean absolute error (MAE) of 4.0 admissions (mean percentage error of 17%) versus 6.5 (32%) for a benchmark metric. Models developed with 104,504 later visits during the Covid-19 pandemic gave AUROCs of 0.68-0.90 and MAE of 4.2 (30%) versus a 4.9 (33%) benchmark. We discuss how we surmounted challenges of designing and implementing models for real-time use, including temporal framing, data preparation, and changing operational conditions.
| S-EPMC9321296 | biostudies-literature
Project description:BackgroundDuring the pandemic, remote consultations have become the norm for assessing patients with signs and symptoms of COVID-19 to decrease the risk of transmission. This has intensified the clinical uncertainty already experienced by primary care clinicians when assessing patients with suspected COVID-19 and has prompted the use of risk prediction scores, such as the National Early Warning Score (NEWS2), to assess severity and guide treatment. However, the risk prediction tools available have not been validated in a community setting and are not designed to capture the idiosyncrasies of COVID-19 infection.ObjectiveThe objective of this study is to produce a multivariate risk prediction tool, RECAP-V1 (Remote COVID-19 Assessment in Primary Care), to support primary care clinicians in the identification of those patients with COVID-19 that are at higher risk of deterioration and facilitate the early escalation of their treatment with the aim of improving patient outcomes.MethodsThe study follows a prospective cohort observational design, whereby patients presenting in primary care with signs and symptoms suggestive of COVID-19 will be followed and their data linked to hospital outcomes (hospital admission and death). Data collection will be carried out by primary care clinicians in four arms: North West London Clinical Commissioning Groups (NWL CCGs), Oxford-Royal College of General Practitioners (RCGP) Research and Surveillance Centre (RSC), Covid Clinical Assessment Service (CCAS), and South East London CCGs (Doctaly platform). The study involves the use of an electronic template that incorporates a list of items (known as RECAP-V0) thought to be associated with disease outcome according to previous qualitative work. Data collected will be linked to patient outcomes in highly secure environments. We will then use multivariate logistic regression analyses for model development and validation.ResultsRecruitment of participants started in October 2020. Initially, only the NWL CCGs and RCGP RSC arms were active. As of March 24, 2021, we have recruited a combined sample of 3827 participants in these two arms. CCAS and Doctaly joined the study in February 2021, with CCAS starting the recruitment process on March 15, 2021. The first part of the analysis (RECAP-V1 model development) is planned to start in April 2021 using the first half of the NWL CCGs and RCGP RSC combined data set. Posteriorly, the model will be validated with the rest of the NWL CCGs and RCGP RSC data as well as the CCAS and Doctaly data sets. The study was approved by the Research Ethics Committee on May 27, 2020 (Integrated Research Application System number: 283024, Research Ethics Committee reference number: 20/NW/0266) and badged as National Institute of Health Research Urgent Public Health Study on October 14, 2020.ConclusionsWe believe the validated RECAP-V1 early warning score will be a valuable tool for the assessment of severity in patients with suspected COVID-19 in the community, either in face-to-face or remote consultations, and will facilitate the timely escalation of treatment with the potential to improve patient outcomes.Trial registrationISRCTN registry ISRCTN13953727; https://www.isrctn.com/ISRCTN13953727.International registered report identifier (irrid)DERR1-10.2196/29072.
| S-EPMC8153031 | biostudies-literature