Published models that predict hospital readmission: a critical appraisal.
Ontology highlight
ABSTRACT: Introduction
The number of readmission risk prediction models available has increased rapidly, and these models are used extensively for health decision-making. Unfortunately, readmission models can be subject to flaws in their development and validation, as well as limitations in their clinical usefulness.Objective
To critically appraise readmission models in the published literature using Delphi-based recommendations for their development and validation.Methods
We used the modified Delphi process to create Critical Appraisal of Models that Predict Readmission (CAMPR), which lists expert recommendations focused on development and validation of readmission models. Guided by CAMPR, two researchers independently appraised published readmission models in two recent systematic reviews and concurrently extracted data to generate reference lists of eligibility criteria and risk factors.Results
We found that published models (n=81) followed 6.8 recommendations (45%) on average. Many models had weaknesses in their development, including failure to internally validate (12%), failure to account for readmission at other institutions (93%), failure to account for missing data (68%), failure to discuss data preprocessing (67%) and failure to state the model's eligibility criteria (33%).Conclusions
The high prevalence of weaknesses in model development identified in the published literature is concerning, as these weaknesses are known to compromise predictive validity. CAMPR may support researchers, clinicians and administrators to identify and prevent future weaknesses in model development.
SUBMITTER: Grossman Liu L
PROVIDER: S-EPMC8336235 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Parkinson's disease, the most common movement disorder, has a strong neuroinflammatory aspect. This is evident by increased pro-inflammatory cytokines in the serum, and the presence of activated microglial cells, and inflammatory cytokines in the substantia nigra of post-mortem brains as well as cerebrospinal fluid of Parkinson's disease patients. The central and peripheral neuroinflammatory aspects of Parkinson's disease can be investigated in vivo via administration of the inflammagen lipopolysaccharide, a component of the cell wall of gram-negative bacteria. In this mini-review, we will critically evaluate different routes of lipopolysaccharide administration (including intranasal systemic and stereotasic), their relevance to clinical Parkinson's disease as well as the recent findings in lipopolysaccharide mouse models. We will also share our own experiences with systemic and intrastriatal lipopolysaccharide models in C57BL/6 mice and will discuss the usefulness of lipopolysaccharide mouse models for future research in the field.
| S-EPMC9120679 | biostudies-literature
Project description:ObjectivesIn the field of relapsed or refractory multiple myeloma (RRMM), between-trial or indirect comparisons are required to estimate relative treatment effects between competing interventions based on the available evidence. Two approaches are frequently used in RRMM: network meta-analysis (NMA) and unanchored matching-adjusted indirect comparison (MAIC). The objective of the current study was to evaluate the relevance and credibility of published NMA and unanchored MAIC studies aiming to estimate the comparative efficacy of treatment options for RRMM.MethodsTwelve relevant studies were identified in the published literature (n = 7) and from health technology assessment agencies (n = 5). Data from trials were extracted to identify between-trial differences that may have biased results. Credibility of the performed analyses and relevance of the research questions were critically appraised using the International Society for Pharmacoeconomics and Outcomes Research (ISPOR) checklist and feedback based on consultations with clinical experts.ResultsThe identified studies concerned NMAs of randomized controlled trials (RCTs; n = 7), unanchored MAICs (n = 4), or both types of analyses (n = 1). According to clinical expert consultation, the majority of the identified NMAs did not consider differences in prior therapies or treatment duration across the RCTs included in the analyses, thereby compromising the relevance.ConclusionBased on the results and feedback from clinicians, the majority of NMAs did not consider prior treatment history or treatment duration, which resulted in nonrelevant comparisons. Furthermore, it may have compromised the credibility of the estimates owing to differences in effect-modifiers between the different trials. Pairwise comparisons by means of unanchored MAICs require clear justification given the reliance on non-randomized comparisons.
| S-EPMC7480667 | biostudies-literature
Project description:IntroductionOver the last years, the number of systematic reviews published is steadily increasing due to the global interest in this type of evidence synthesis. However, little is known about the characteristics of this research published in Portuguese medical journals. This study aims to evaluate the publication trends and overall quality of these systematic reviews.Material and methodsThis was a methodological study. We aimed the most visible Portuguese medical journals indexed in MEDLINE. Systematic reviews were identified through an electronic search (through PUBMED). We included systematic reviews published up to August 2020. Systematic reviews selection and data extraction were done independently by three authors. The overall quality critical appraisal using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR-2) was independently assessed by three authors. Disagreements were solved by consensus.ResultsSixty-six systematic reviews published in 5 Portuguese medical journals were included. Most (n = 53; 80.3%) were systematic reviews without meta-analysis. Up to 2010 there was a steady increase in the number of systematic reviews published, followed by a period of great variability of publication, ranging from 1 to 10 in a given year. According to the systematic reviews' typology, most have been predominantly conducted to assess the effectiveness/efficacy of health interventions (n = 27; 40.9%). General and Internal Medicine (n = 20; 30.3%) was the most addressed field. Most systematic reviews (n = 46; 69.7%) were rated as being of "critically low-quality".ConclusionsThere were consistent flaws in the methodological quality report of the systematic reviews included, particularly in establishing a prior protocol and not assessing the potential impact of the risk of bias on the results. Through the years, the number of systematic reviews published increased, yet their quality is suboptimal. There is a need to improve the reporting of systematic reviews in Portuguese medical journals, which can be achieved by better adherence to quality checklists/tools.
| S-EPMC8996638 | biostudies-literature
Project description:BackgroundPeople presenting with first-episode psychosis (FEP) have heterogenous outcomes. More than 40% fail to achieve symptomatic remission. Accurate prediction of individual outcome in FEP could facilitate early intervention to change the clinical trajectory and improve prognosis.AimsWe aim to systematically review evidence for prediction models developed for predicting poor outcome in FEP.MethodA protocol for this study was published on the International Prospective Register of Systematic Reviews, registration number CRD42019156897. Following Preferred Reporting Items for Systematic Reviews and Meta-Analyses guidance, we systematically searched six databases from inception to 28 January 2021. We used the Checklist for Critical Appraisal and Data Extraction for Systematic Reviews of Prediction Modelling Studies and the Prediction Model Risk of Bias Assessment Tool to extract and appraise the outcome prediction models. We considered study characteristics, methodology and model performance.ResultsThirteen studies reporting 31 prediction models across a range of clinical outcomes met criteria for inclusion. Eleven studies used logistic regression with clinical and sociodemographic predictor variables. Just two studies were found to be at low risk of bias. Methodological limitations identified included a lack of appropriate validation, small sample sizes, poor handling of missing data and inadequate reporting of calibration and discrimination measures. To date, no model has been applied to clinical practice.ConclusionsFuture prediction studies in psychosis should prioritise methodological rigour and external validation in larger samples. The potential for prediction modelling in FEP is yet to be realised.
| S-EPMC7612705 | biostudies-literature
Project description:The use of real-time quantitative polymerase chain reaction (qPCR) in cancer research has become ubiquitous. The relative simplicity of qPCR experiments, which deliver fast and cost-effective results, means that each year an increasing number of papers utilizing this technique are being published. But how reliable are the published results? Since the validity of gene expression data is greatly dependent on appropriate normalisation to compensate for sample-to-sample and run-to-run variation, we have evaluated the adequacy of normalisation procedures in qPCR-based experiments. Consequently, we assessed all colorectal cancer publications that made use of qPCR from 2006 until August 2013 for the number of reference genes used and whether they had been validated. Using even these minimal evaluation criteria, the validity of only three percent (6/179) of the publications can be adequately assessed. We describe common errors, and conclude that the current state of reporting on qPCR in colorectal cancer research is disquieting. Extrapolated to the study of cancer in general, it is clear that the majority of studies using qPCR cannot be reliably assessed and that at best, the results of these studies may or may not be valid and at worst, pervasive incorrect normalisation is resulting in the wholesale publication of incorrect conclusions. This survey demonstrates that the existence of guidelines, such as MIQE, is necessary but not sufficient to address this problem and suggests that the scientific community should examine its responsibility and be aware of the implications of these findings for current and future research.
| S-EPMC5528534 | biostudies-literature
Project description:Background and aimsEsophageal cancer risk prediction models allow for risk-stratified endoscopic screening. We aimed to assess the quality of these models developed in the general population.MethodsA systematic search of the PubMed and Embase databases from January 2000 through May 2021 was performed. Studies that developed or validated a model of esophageal cancer in the general population were included. Screening, data extraction, and risk of bias (ROB) assessment by the Prediction model Risk Of Bias Assessment Tool (PROBAST) were performed independently by two reviewers.ResultsOf the 13 models included in the qualitative analysis, 8 were developed for esophageal squamous cell carcinoma (ESCC) and the other 5 were developed for esophageal adenocarcinoma (EAC). Only two models conducted external validation. In the ESCC models, cigarette smoking was included in each model, followed by age, sex, and alcohol consumption. For EAC models, cigarette smoking and body mass index were included in each model, and gastroesophageal reflux disease, uses of acid-suppressant medicine, and nonsteroidal anti-inflammatory drug were exclusively included. The discriminative performance was reported in all studies, with C statistics ranging from 0.71 to 0.88, whereas only six models reported calibration. For ROB, all the models had a low risk in participant and outcome, but all models showed high risk in analysis, and 60% of models showed a high risk in predictors, which resulted in all models being classified as having overall high ROB. For model applicability, about 60% of these models had an overall low risk, with 30% of models of high risk and 10% of models of unclear risk, concerning the assessment of participants, predictors, and outcomes.ConclusionsMost current risk prediction models of esophageal cancer have a high ROB. Prediction models need further improvement in their quality and applicability to benefit esophageal cancer screening.
| S-EPMC8525074 | biostudies-literature
Project description:Objective: This work aims to systematically identify, describe, and appraise all prognostic models for cervical cancer and provide a reference for clinical practice and future research. Methods: We systematically searched PubMed, EMBASE, and Cochrane library databases up to December 2020 and included studies developing, validating, or updating a prognostic model for cervical cancer. Two reviewers extracted information based on the CHecklist for critical Appraisal and data extraction for systematic Reviews of prediction Modeling Studies checklist and assessed the risk of bias using the Prediction model Risk Of Bias ASsessment Tool. Results: Fifty-six eligible articles were identified, describing the development of 77 prognostic models and 27 external validation efforts. The 77 prognostic models focused on three types of cervical cancer patients at different stages, i.e., patients with early-stage cervical cancer (n = 29; 38%), patients with locally advanced cervical cancer (n = 27; 35%), and all-stage cervical cancer patients (n = 21; 27%). Among the 77 models, the most frequently used predictors were lymph node status (n = 57; 74%), the International Federation of Gynecology and Obstetrics stage (n = 42; 55%), histological types (n = 38; 49%), and tumor size (n = 37; 48%). The number of models that applied internal validation, presented a full equation, and assessed model calibration was 52 (68%), 16 (21%), and 45 (58%), respectively. Twenty-four models were externally validated, among which three were validated twice. None of the models were assessed with an overall low risk of bias. The Prediction Model of Failure in Locally Advanced Cervical Cancer model was externally validated twice, with acceptable performance, and seemed to be the most reliable. Conclusions: Methodological details including internal validation, sample size, and handling of missing data need to be emphasized on, and external validation is needed to facilitate the application and generalization of models for cervical cancer.
| S-EPMC8137851 | biostudies-literature
Project description:Osteoporotic fractures (OF) are a global public health problem currently. Many risk prediction models for OF have been developed, but their performance and methodological quality are unclear. We conducted this systematic review to summarize and critically appraise the OF risk prediction models. Three databases were searched until April 2021. Studies developing or validating multivariable models for OF risk prediction were considered eligible. Used the prediction model risk of bias assessment tool to appraise the risk of bias and applicability of included models. All results were narratively summarized and described. A total of 68 studies describing 70 newly developed prediction models and 138 external validations were included. Most models were explicitly developed (n=31, 44%) and validated (n=76, 55%) only for female. Only 22 developed models (31%) were externally validated. The most validated tool was Fracture Risk Assessment Tool. Overall, only a few models showed outstanding (n=3, 1%) or excellent (n=32, 15%) prediction discrimination. Calibration of developed models (n=25, 36%) or external validation models (n=33, 24%) were rarely assessed. No model was rated as low risk of bias, mostly because of an insufficient number of cases and inappropriate assessment of calibration. There are a certain number of OF risk prediction models. However, few models have been thoroughly internally validated or externally validated (with calibration being unassessed for most of the models), and all models showed methodological shortcomings. Instead of developing completely new models, future research is suggested to validate, improve, and analyze the impact of existing models.
| S-EPMC9286920 | biostudies-literature
Project description:BackgroundFalls impose significant health and economic burdens on community-dwelling older persons. Decision modelling can inform commissioning of alternative falls prevention strategies. Several methodological challenges arise when modelling public health interventions including community-based falls prevention. This study aims to conduct a systematic review (SR) to: systematically identify community-based falls prevention economic models; synthesise and critically appraise how the models handled key methodological challenges associated with public health modelling; and suggest areas for further methodological research.MethodsThe SR followed the 2021 PRISMA reporting guideline and covered the period 2003-2020 and 12 academic databases and grey literature. The extracted methodological features of included models were synthesised by their relevance to the following challenges: (1) capturing non-health outcomes and societal intervention costs; (2) considering heterogeneity and dynamic complexity; (3) considering theories of human behaviour and implementation; and (4) considering equity issues. The critical appraisal assessed the prevalence of each feature across models, then appraised the methods used to incorporate the feature. The methodological strengths and limitations stated by the modellers were used as indicators of desirable modelling practice and scope for improvement, respectively. The methods were also compared against those suggested in the broader empirical and methodological literature. Areas of further methodological research were suggested based on appraisal results.Results46 models were identified. Comprehensive incorporation of non-health outcomes and societal intervention costs was infrequent. The assessments of heterogeneity and dynamic complexity were limited; subgroup delineation was confined primarily to demographics and binary disease/physical status. Few models incorporated heterogeneity in intervention implementation level, efficacy and cost. Few dynamic variables other than age and falls history were incorporated to characterise the trajectories of falls risk and general health/frailty. Intervention sustainability was frequently based on assumptions; few models estimated the economic/health returns from improved implementation. Seven models incorporated ethnicity- and severity-based subgroups but did not estimate the equity-efficiency trade-offs. Sixteen methodological research suggestions were made.ConclusionExisting community-based falls prevention models contain methodological limitations spanning four challenge areas relevant for public health modelling. There is scope for further methodological research to inform the development of falls prevention and other public health models.
| S-EPMC9287934 | biostudies-literature
Project description:BackgroundReadmission after colorectal surgery is common and often implies complications for patients and costs for hospitals. Previous works have created predictive models using logistic regression for this outcome but have shown limited accuracy. Machine learning has shown promise in improving predictions by identifying non-linear patterns in data. We sought to create a more accurate predictive model for readmission after colorectal surgery using machine learning.MethodsPatients who underwent colorectal surgery were identified in the National Quality Improvement Program (NSQIP) database including years 2012-2019 and split into training, validation, and test sets. The primary outcome was readmission within 30 days of surgery. Three types of machine learning models were created, including random forest (RF), gradient boosting (XGB), and neural network (NN). A logistic regression (LR) model was also created for comparison. Model performance was evaluated using area under the receiver operating characteristic curve (AUROC).ResultsThe dataset included 213,827 patients after application of exclusion criteria. A total of 23,083 (10.8%) of patients experienced readmission. NN obtained an AUROC of 0.751 (95% CI 0.743-0.759), compared with 0.684 (95% CI 0.676-0.693) for LR. RF and XGB performed similarly with AUROCs of 0.749 (95% CI 0.741-0.757) and 0.745 (95% CI 0.737-0.753) respectively. Ileus, index admission length of stay, organ-space surgical site infection present at time of surgery, and ostomy placement were identified as the most contributory variables.ConclusionsMachine learning approaches outperformed traditional statistical methods in the prediction of readmission after colorectal surgery. After external validation, this improved prediction model could be used to target interventions to reduce readmission rate.
| S-EPMC10081888 | biostudies-literature