Dataset Information

External Validation of a Prognostic Model for Predicting Nonresponse Following Knee Arthroplasty.

ABSTRACT: BACKGROUND:Instruments designed to predict the extent of pain and function following knee arthroplasty (KA) recovery has strong potential to guide patients and clinicians in shared decision making. Our purpose was to test the external validity of a recently developed prognostic instrument designed to estimate the probability of nonresponse following KA. METHODS:We used data from the Osteoarthritis Initiative (OAI), a 9-year multisite National Institutes of Health study designed to examine the natural history of knee osteoarthritis in 4796 subjects. A total of 427 subjects underwent KA over the study period. Dowsey et al examined the prognostic role of obesity, general mental health, pain and function, and Kellgren and Lawrence knee osteoarthritis grades. Calibration of the prognostic model was determined using a calibration curve. The c-statistic was used to indicate discrimination of the model. RESULTS:In the primary analysis, 63 (19.3%) of 326 subjects in OAI were classified as nonresponders. The calibration curve generated from OAI data indicated poor calibration relative to the recently developed instrument. Discrimination as measured by the c-statistic was 0.76. CONCLUSION:The external validity of the prognostic instrument was partially supported. While discrimination of the model was very similar to the recently developed instrument, calibration was poor indicating poor agreement between actual vs predicted probabilities of nonresponse. Western Ontario and McMaster Universities Arthritis Index and Kellgren and Lawrence grades show strong potential for use in future prognostic model development. Measurements of general mental health and obesity were not prognostic for nonresponse.

SUBMITTER: Riddle DL

PROVIDER: S-EPMC5362316 | biostudies-literature | 2017 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

External Validation of a Prognostic Model for Predicting Nonresponse Following Knee Arthroplasty.

Riddle Daniel L DL Golladay Gregory J GJ Jiranek William A WA Perera Robert A RA

The Journal of arthroplasty 20161115 4

<h4>Background</h4>Instruments designed to predict the extent of pain and function following knee arthroplasty (KA) recovery has strong potential to guide patients and clinicians in shared decision making. Our purpose was to test the external validity of a recently developed prognostic instrument designed to estimate the probability of nonresponse following KA.<h4>Methods</h4>We used data from the Osteoarthritis Initiative (OAI), a 9-year multisite National Institutes of Health study designed to ...[more]

PMID: 27919582

Similar Datasets

Project description:BackgroundSeizures are the main cause of maternal death in women with epilepsy, but there are no tools for predicting seizures in pregnancy. We set out to develop and validate a prognostic model, using information collected during the antenatal booking visit, to predict seizure risk at any time in pregnancy and until 6 weeks postpartum in women with epilepsy on antiepileptic drugs.Methods and findingsWe used datasets of a prospective cohort study (EMPiRE) of 527 pregnant women with epilepsy on medication recruited from 50 hospitals in the UK (4 November 2011-17 August 2014). The model development cohort comprised 399 women whose antiepileptic drug doses were adjusted based on clinical features only; the validation cohort comprised 128 women whose drug dose adjustments were informed by serum drug levels. The outcome was epileptic (non-eclamptic) seizure captured using diary records. We fitted the model using LASSO (least absolute shrinkage and selection operator) regression, and reported the performance using C-statistic (scale 0-1, values > 0.5 show discrimination) and calibration slope (scale 0-1, values near 1 show accuracy) with 95% confidence intervals (CIs). We determined the net benefit (a weighted sum of true positive and false positive classifications) of using the model, with various probability thresholds, to aid clinicians in making individualised decisions regarding, for example, referral to tertiary care, frequency and intensity of monitoring, and changes in antiepileptic medication. Seizures occurred in 183 women (46%, 183/399) in the model development cohort and in 57 women (45%, 57/128) in the validation cohort. The model included age at first seizure, baseline seizure classification, history of mental health disorder or learning difficulty, occurrence of tonic-clonic and non-tonic-clonic seizures in the 3 months before pregnancy, previous admission to hospital for seizures during pregnancy, and baseline dose of lamotrigine and levetiracetam. The C-statistic was 0.79 (95% CI 0.75, 0.84). On external validation, the model showed good performance (C-statistic 0.76, 95% CI 0.66, 0.85; calibration slope 0.93, 95% CI 0.44, 1.41) but with imprecise estimates. The EMPiRE model showed the highest net proportional benefit for predicted probability thresholds between 12% and 99%. Limitations of this study include the varied gestational ages of women at recruitment, retrospective patient recall of seizure history, potential variations in seizure classification, the small number of events in the validation cohort, and the clinical utility restricted to decision-making thresholds above 12%. The model findings may not be generalisable to low- and middle-income countries, or when information on all predictors is not available.ConclusionsThe EMPiRE model showed good performance in predicting the risk of seizures in pregnant women with epilepsy who are prescribed antiepileptic drugs. Integration of the tool within the antenatal booking visit, deployed as a simple nomogram, can help to optimise care in women with epilepsy.

Project description:BackgroundTotal joint replacements are high-volume and high-cost procedures that should be monitored for cost and quality control. Models that can identify patients at high risk of readmission might help reduce costs by suggesting who should be enrolled in preventive care programs. Previous models for risk prediction have relied on structured data of patients rather than clinical notes in electronic health records (EHRs). The former approach requires manual feature extraction by domain experts, which may limit the applicability of these models.ObjectiveThis study aims to develop and evaluate a machine learning model for predicting the risk of 30-day readmission following knee and hip arthroplasty procedures. The input data for these models come from raw EHRs. We empirically demonstrate that unstructured free-text notes contain a reasonably predictive signal for this task.MethodsWe performed a retrospective analysis of data from 7174 patients at Partners Healthcare collected between 2006 and 2016. These data were split into train, validation, and test sets. These data sets were used to build, validate, and test models to predict unplanned readmission within 30 days of hospital discharge. The proposed models made predictions on the basis of clinical notes, obviating the need for performing manual feature extraction by domain and machine learning experts. The notes that served as model inputs were written by physicians, nurses, pathologists, and others who diagnose and treat patients and may have their own predictions, even if these are not recorded.ResultsThe proposed models output readmission risk scores (propensities) for each patient. The best models (as selected on a development set) yielded an area under the receiver operating characteristic curve of 0.846 (95% CI 82.75-87.11) for hip and 0.822 (95% CI 80.94-86.22) for knee surgery, indicating reasonable discriminative ability.ConclusionsMachine learning models can predict which patients are at a high risk of readmission within 30 days following hip and knee arthroplasty procedures on the basis of notes in EHRs with reasonable discriminative power. Following further validation and empirical demonstration that the models realize predictive performance above that which clinical judgment may provide, such models may be used to build an automated decision support tool to help caretakers identify at-risk patients.

Project description:BackgroundSeveral prognostic models for outcomes after chronic subdural hematoma (CSDH) treatment have been published in recent years. However, these models are not sufficiently validated for use in daily clinical practice. We aimed to assess the performance of existing prediction models for outcomes in patients diagnosed with CSDH.MethodsWe systematically searched relevant literature databases up to February 2021 to identify prognostic models for outcome prediction in patients diagnosed with CSDH. For the external validation of prognostic models, we used a retrospective database, containing data of 2384 patients from three Dutch regions. Prognostic models were included if they predicted either mortality, hematoma recurrence, functional outcome, or quality of life. Models were excluded when predictors were absent in our database or available for < 150 patients in our database. We assessed calibration, and discrimination (quantified by the concordance index C) of the included prognostic models in our retrospective database.ResultsWe identified 1680 original publications of which 1656 were excluded based on title or abstract, mostly because they did not concern CSDH or did not define a prognostic model. Out of 18 identified models, three could be externally validated in our retrospective database: a model for 30-day mortality in 1656 patients, a model for 2 months, and another for 3-month hematoma recurrence both in 1733 patients. The models overestimated the proportion of patients with these outcomes by 11% (15% predicted vs. 4% observed), 1% (10% vs. 9%), and 2% (11% vs. 9%), respectively. Their discriminative ability was poor to modest (C of 0.70 [0.63-0.77]; 0.46 [0.35-0.56]; 0.59 [0.51-0.66], respectively).ConclusionsNone of the examined models showed good predictive performance for outcomes after CSDH treatment in our dataset. This study confirms the difficulty in predicting outcomes after CSDH and emphasizes the heterogeneity of CSDH patients. The importance of developing high-quality models by using unified predictors and relevant outcome measures and appropriate modeling strategies is warranted.

Project description:BackgroundThe evaluation of swelling is important for the outcome of total knee arthroplasty (TKA) surgery. The circumference or volume measurements are applicable at the bedside of the patient but are altered by muscular atrophy and the post-surgical dressing. Bioimpedance spectroscopy might overcome these limitations; however, it should be validated. This study aimed to explore the validity, the reliability and the responsiveness of bioimpedance spectroscopy for measuring swelling after TKA.MethodsThe degree of swelling in 25 patients undergoing TKA surgery was measured using bioimpedance spectroscopy (BIS R0), knee circumference and limb volume. The measurements were performed on D-1 (day before surgery), D + 2 (2 days after surgery) and D + 8 (8 days after surgery). The BIS R0 measurements were repeated twice, alternating between two evaluators. The percentage of the difference between the limbs was calculated for BIS R0, circumference and volume. The intra- and inter-observer intraclass correlation coefficients (ICCs), limits of agreement (LOA), effect size (Cohen's d), correlations between the methods and diagnostic sensitivity were calculated.ResultsBIS R0, circumference and volume detected swelling < 3.5% at D-1. The swelling at D2 and D8 was greater with BIS R0 [mean (SD) 29.9% (±9.8) and 38.27 (±7.8)] than with volume [14.7 (±9.5) and 14.9 (±8.2)] and circumference [11.1 (±5.7) and 11.7 (±4.1)]. The BIS R0 intra- and inter-evaluator ICCs ranged from 0.89 to 0.99, whereas the LOA were < 5.2%. The BIS R0 correlation was 0.73 with volume and 0.75 with circumference. The BIS R0 Cohen's d was 3.32 for the D-1-D2 evolution. The diagnostic sensitivity was 83% D2 and 96% at D8.ConclusionBioimpedance is a valid method for the evaluation of swelling following TKA. BIS R0 also demonstrated excellent intra- and inter-evaluator reliability. The diagnostic sensitivity and responsiveness is superior to that of concurrent methods. BIS R0 is an efficient method for post-surgical follow up at the bedside of the patient. The measurement of BIS R0 is a straightforward, valid, reliable and responsive method for lower limb swelling following TKA surgery that could be used in clinics and research.Trial registrationClinicalTrials.gov Identifier: NCT00627770.

Project description:ObjectiveIn the United Kingdom and other European Union countries guidelines for driving following a first unprovoked seizure require the risk of another seizure in the next year to be less than 20%. Using data from one clinical trial, we previously developed a prognostic model to inform driving guidelines. The objective of this work is to externally validate our published model and demonstrate its generalisability.MethodsA cohort of 620 people with a first unprovoked seizure was used to develop the original model which included variables for aetiology, first degree relative with epilepsy, seizures only while asleep, electroencephalogram, computed tomography or magnetic resonance scan result, and treatment policy. The validation cohorts consisted of 274 (United Kingdom), 305 (Italy), and 847 (Australia) people. The model was evaluated using discrimination and calibration methods. A covariate, missing from the Italian dataset, was handled via five imputation methods. Following external validation, the model was fitted to a pooled population comprising all validation datasets and the development dataset. The model was stratified by dataset.ResultsThe model generalised relatively well. All methods of imputation performed fairly similarly. At six months, the risk of a seizure recurrence following a first ever seizure, based on the pooled datasets, is 15% (95% CI: (12% to 18%)) for patients who are treated immediately and 18% (95% CI: (15 to 21%)) otherwise. Individuals can be reliably stratified into risk groups according to the clinical factors included in the model.SignificanceOur prognostic model, used to inform driving regulations, has been validated and consequently has been proven as a valuable tool for predicting risk of seizure recurrence following a first seizure in people with various combinations of risk factors. Additionally, there is evidence to support one worldwide overall prognostic model for risk of second seizure following a first.

Project description:BackgroundPre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately predict the condition when used within the UK healthcare setting.MethodsIPD from 11 UK cohort studies (217,415 pregnant women) within the International Prediction of Pregnancy Complications (IPPIC) pre-eclampsia network contributed to external validation of published prediction models, identified by systematic review. Cohorts that measured all predictor variables in at least one of the identified models and reported pre-eclampsia as an outcome were included for validation. We reported the model predictive performance as discrimination (C-statistic), calibration (calibration plots, calibration slope, calibration-in-the-large), and net benefit. Performance measures were estimated separately in each available study and then, where possible, combined across studies in a random-effects meta-analysis.ResultsOf 131 published models, 67 provided the full model equation and 24 could be validated in 11 UK cohorts. Most of the models showed modest discrimination with summary C-statistics between 0.6 and 0.7. The calibration of the predicted compared to observed risk was generally poor for most models with observed calibration slopes less than 1, indicating that predictions were generally too extreme, although confidence intervals were wide. There was large between-study heterogeneity in each model's calibration-in-the-large, suggesting poor calibration of the predicted overall risk across populations. In a subset of models, the net benefit of using the models to inform clinical decisions appeared small and limited to probability thresholds between 5 and 7%.ConclusionsThe evaluated models had modest predictive performance, with key limitations such as poor calibration (likely due to overfitting in the original development datasets), substantial heterogeneity, and small net benefit across settings. The evidence to support the use of these prediction models for pre-eclampsia in clinical decision-making is limited. Any models that we could not validate should be examined in terms of their predictive performance, net benefit, and heterogeneity across multiple UK settings before consideration for use in practice.Trial registrationPROSPERO ID: CRD42015029349 .

Dataset Information

External Validation of a Prognostic Model for Predicting Nonresponse Following Knee Arthroplasty.

Publications

External Validation of a Prognostic Model for Predicting Nonresponse Following Knee Arthroplasty.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets