Dataset Information

External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis.

ABSTRACT: BACKGROUND:Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately predict the condition when used within the UK healthcare setting. METHODS:IPD from 11 UK cohort studies (217,415 pregnant women) within the International Prediction of Pregnancy Complications (IPPIC) pre-eclampsia network contributed to external validation of published prediction models, identified by systematic review. Cohorts that measured all predictor variables in at least one of the identified models and reported pre-eclampsia as an outcome were included for validation. We reported the model predictive performance as discrimination (C-statistic), calibration (calibration plots, calibration slope, calibration-in-the-large), and net benefit. Performance measures were estimated separately in each available study and then, where possible, combined across studies in a random-effects meta-analysis. RESULTS:Of 131 published models, 67 provided the full model equation and 24 could be validated in 11 UK cohorts. Most of the models showed modest discrimination with summary C-statistics between 0.6 and 0.7. The calibration of the predicted compared to observed risk was generally poor for most models with observed calibration slopes less than 1, indicating that predictions were generally too extreme, although confidence intervals were wide. There was large between-study heterogeneity in each model's calibration-in-the-large, suggesting poor calibration of the predicted overall risk across populations. In a subset of models, the net benefit of using the models to inform clinical decisions appeared small and limited to probability thresholds between 5 and 7%. CONCLUSIONS:The evaluated models had modest predictive performance, with key limitations such as poor calibration (likely due to overfitting in the original development datasets), substantial heterogeneity, and small net benefit across settings. The evidence to support the use of these prediction models for pre-eclampsia in clinical decision-making is limited. Any models that we could not validate should be examined in terms of their predictive performance, net benefit, and heterogeneity across multiple UK settings before consideration for use in practice. TRIAL REGISTRATION:PROSPERO ID: CRD42015029349 .

SUBMITTER: Snell KIE

PROVIDER: S-EPMC7604970 | biostudies-literature | 2020 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis.

Snell Kym I E KIE Allotey John J Smuk Melanie M Hooper Richard R Chan Claire C Ahmed Asif A Chappell Lucy C LC Von Dadelszen Peter P Green Marcus M Kenny Louise L Khalil Asma A Khan Khalid S KS Mol Ben W BW Myers Jenny J Poston Lucilla L Thilaganathan Basky B Staff Anne C AC Smith Gordon C S GCS Ganzevoort Wessel W Laivuori Hannele H Odibo Anthony O AO Arenas Ramírez Javier J Kingdom John J Daskalakis George G Farrar Diane D Baschat Ahmet A AA Seed Paul T PT Prefumo Federico F da Silva Costa Fabricio F Groen Henk H Audibert Francois F Masse Jacques J Skråstad Ragnhild B RB Salvesen Kjell Å KÅ Haavaldsen Camilla C Nagata Chie C Rumbold Alice R AR Heinonen Seppo S Askie Lisa M LM Smits Luc J M LJM Vinter Christina A CA Magnus Per P Eero Kajantie K Villa Pia M PM Jenum Anne K AK Andersen Louise B LB Norman Jane E JE Ohkuchi Akihide A Eskild Anne A Bhattacharya Sohinee S McAuliffe Fionnuala M FM Galindo Alberto A Herraiz Ignacio I Carbillon Lionel L Klipstein-Grobusch Kerstin K Yeo Seon Ae SA Browne Joyce L JL Moons Karel G M KGM Riley Richard D RD Thangaratinam Shakila S

BMC medicine 20201102 1

<h4>Background</h4>Pre-eclampsia is a leading cause of maternal and perinatal mortality and morbidity. Early identification of women at risk during pregnancy is required to plan management. Although there are many published prediction models for pre-eclampsia, few have been validated in external data. Our objective was to externally validate published prediction models for pre-eclampsia using individual participant data (IPD) from UK studies, to evaluate whether any of the models can accurately ...[more]

PMID: 33131506

Similar Datasets

Project description:ObjectiveTo externally validate various prognostic models and scoring rules for predicting short term mortality in patients admitted to hospital for covid-19.DesignTwo stage individual participant data meta-analysis.SettingSecondary and tertiary care.Participants46 914 patients across 18 countries, admitted to a hospital with polymerase chain reaction confirmed covid-19 from November 2019 to April 2021.Data sourcesMultiple (clustered) cohorts in Brazil, Belgium, China, Czech Republic, Egypt, France, Iran, Israel, Italy, Mexico, Netherlands, Portugal, Russia, Saudi Arabia, Spain, Sweden, United Kingdom, and United States previously identified by a living systematic review of covid-19 prediction models published in The BMJ, and through PROSPERO, reference checking, and expert knowledge.Model selection and eligibility criteriaPrognostic models identified by the living systematic review and through contacting experts. A priori models were excluded that had a high risk of bias in the participant domain of PROBAST (prediction model study risk of bias assessment tool) or for which the applicability was deemed poor.MethodsEight prognostic models with diverse predictors were identified and validated. A two stage individual participant data meta-analysis was performed of the estimated model concordance (C) statistic, calibration slope, calibration-in-the-large, and observed to expected ratio (O:E) across the included clusters.Main outcome measures30 day mortality or in-hospital mortality.ResultsDatasets included 27 clusters from 18 different countries and contained data on 46 914patients. The pooled estimates ranged from 0.67 to 0.80 (C statistic), 0.22 to 1.22 (calibration slope), and 0.18 to 2.59 (O:E ratio) and were prone to substantial between study heterogeneity. The 4C Mortality Score by Knight et al (pooled C statistic 0.80, 95% confidence interval 0.75 to 0.84, 95% prediction interval 0.72 to 0.86) and clinical model by Wang et al (0.77, 0.73 to 0.80, 0.63 to 0.87) had the highest discriminative ability. On average, 29% fewer deaths were observed than predicted by the 4C Mortality Score (pooled O:E 0.71, 95% confidence interval 0.45 to 1.11, 95% prediction interval 0.21 to 2.39), 35% fewer than predicted by the Wang clinical model (0.65, 0.52 to 0.82, 0.23 to 1.89), and 4% fewer than predicted by Xie et al's model (0.96, 0.59 to 1.55, 0.21 to 4.28).ConclusionThe prognostic value of the included models varied greatly between the data sources. Although the Knight 4C Mortality Score and Wang clinical model appeared most promising, recalibration (intercept and slope updates) is needed before implementation in routine care.

Project description:BackgroundSeveral prognostic models for outcomes after chronic subdural hematoma (CSDH) treatment have been published in recent years. However, these models are not sufficiently validated for use in daily clinical practice. We aimed to assess the performance of existing prediction models for outcomes in patients diagnosed with CSDH.MethodsWe systematically searched relevant literature databases up to February 2021 to identify prognostic models for outcome prediction in patients diagnosed with CSDH. For the external validation of prognostic models, we used a retrospective database, containing data of 2384 patients from three Dutch regions. Prognostic models were included if they predicted either mortality, hematoma recurrence, functional outcome, or quality of life. Models were excluded when predictors were absent in our database or available for < 150 patients in our database. We assessed calibration, and discrimination (quantified by the concordance index C) of the included prognostic models in our retrospective database.ResultsWe identified 1680 original publications of which 1656 were excluded based on title or abstract, mostly because they did not concern CSDH or did not define a prognostic model. Out of 18 identified models, three could be externally validated in our retrospective database: a model for 30-day mortality in 1656 patients, a model for 2 months, and another for 3-month hematoma recurrence both in 1733 patients. The models overestimated the proportion of patients with these outcomes by 11% (15% predicted vs. 4% observed), 1% (10% vs. 9%), and 2% (11% vs. 9%), respectively. Their discriminative ability was poor to modest (C of 0.70 [0.63-0.77]; 0.46 [0.35-0.56]; 0.59 [0.51-0.66], respectively).ConclusionsNone of the examined models showed good predictive performance for outcomes after CSDH treatment in our dataset. This study confirms the difficulty in predicting outcomes after CSDH and emphasizes the heterogeneity of CSDH patients. The importance of developing high-quality models by using unified predictors and relevant outcome measures and appropriate modeling strategies is warranted.

Project description:BackgroundTo overcome the three delays in triage, transport and treatment that underlie adverse pregnancy outcomes, we aimed to reduce all-cause adverse outcomes with community-level interventions targeting women with pregnancy hypertension in three low-income countries.MethodsIn this individual participant-level meta-analysis, we de-identified and pooled data from the Community-Level Interventions for Pre-eclampsia (CLIP) cluster randomised controlled trials in Mozambique, Pakistan, and India, which were run in 2014-17. Consenting pregnant women, aged 12-49 years, were recruited in their homes. Clusters, defined by local administrative units, were randomly assigned (1:1) to intervention or control groups. The control groups continued local standard of care. The intervention comprised community engagement and existing community health worker-led mobile health-supported early detection, initial treatment, and hospital referral of women with hypertension. For this meta-analysis, as for the original studies, the primary outcome was a composite of maternal or perinatal outcome (either maternal, fetal, or neonatal death, or severe morbidity for the mother or baby), assessed by unmasked trial surveillance personnel. For this analysis, we included all consenting participants who were followed up with completed pregnancies at trial end. We analysed the outcome data with multilevel modelling and present data with the summary statistic of adjusted odds ratios (ORs) with 95% CIs (fixed effects for maternal age, parity, maternal education, and random effects for country and cluster). This meta-analysis is registered with PROSPERO, CRD42018102564.FindingsOverall, 44 clusters (69 330 pregnant women) were randomly assigned to intervention (22 clusters [36 008 pregnancies]) or control (22 clusters [33 322 pregnancies]) groups. 32 290 (89·7%) pregnancies in the intervention group and 29 698 (89·1%) in the control group were followed up successfully. Median maternal age of included women was 26 years (IQR 22-30). In the intervention clusters, 6990 group and 16 691 home-based community engagement sessions and 138 347 community health worker-led visits to 20 819 (57·8%) of 36 008 women (of whom 11 095 [53·3%] had a visit every 4 weeks) occurred. Blood pressure and dipstick proteinuria were assessed per protocol. Few women were eligible for methyldopa for severe hypertension (181 [1%] of 20 819) or intramuscular magnesium sulfate for pre-eclampsia (198 [1%]), of whom most accepted treatment (162 [89·5%] of 181 for severe hypertension and 133 [67·2%] of 198 for pre-eclampsia). 1255 (6%) were referred to a comprehensive emergency obstetric care facility, of whom 864 (82%) accepted the referral. The primary outcome was similar in the intervention (7871 [24%] of 32 290 pregnancies) and control clusters (6516 [22%] of 29 698; adjusted OR 1·17, 95% CI 0·90-1·51; p=0·24). No intervention-related serious adverse events occurred, and few adverse effects occurred after in-community treatment with methyldopa (one [2%] of 51; India only) and none occurred after in-community treatment with magnesium sulfate or during transport to facility.InterpretationThe CLIP intervention did not reduce adverse pregnancy outcomes. Future community-level interventions should expand the community health worker workforce, assess general (rather than condition-specific) messaging, and include health system strengthening.FundingUniversity of British Columbia, a grantee of the Bill & Melinda Gates Foundation.

Project description:BackgroundAdequate maternal thyroid function is important for an uncomplicated pregnancy. Although multiple observational studies have evaluated the association between thyroid dysfunction and hypertensive disorders of pregnancy, the methods and definitions of abnormalities in thyroid function tests were heterogeneous, and the results were conflicting. We aimed to examine the association between abnormalities in thyroid function tests and risk of gestational hypertension and pre-eclampsia.MethodsIn this systematic review and meta-analysis of individual-participant data, we searched MEDLINE (Ovid), Embase, Scopus, and the Cochrane Database of Systematic Reviews from date of inception to Dec 27, 2019, for prospective cohort studies with data on maternal concentrations of thyroid-stimulating hormone (TSH), free thyroxine (FT4), thyroid peroxidase (TPO) antibodies, individually or in combination, as well as on gestational hypertension, pre-eclampsia, or both. We issued open invitations to study authors to participate in the Consortium on Thyroid and Pregnancy and to share the individual-participant data. We excluded participants who had pre-existing thyroid disease or multifetal pregnancy, or were taking medications that affect thyroid function. The primary outcomes were documented gestational hypertension and pre-eclampsia. Individual-participant data were analysed using logistic mixed-effects regression models adjusting for maternal age, BMI, smoking, parity, ethnicity, and gestational age at blood sampling. The study protocol was registered with PROSPERO, CRD42019128585.FindingsWe identified 1539 published studies, of which 33 cohorts met the inclusion criteria and 19 cohorts were included after the authors agreed to participate. Our study population comprised 46 528 pregnant women, of whom 39 826 (85·6%) women had sufficient data (TSH and FT4 concentrations and TPO antibody status) to be classified according to their thyroid function status. Of these women, 1275 (3·2%) had subclinical hypothyroidism, 933 (2·3%) had isolated hypothyroxinaemia, 619 (1·6%) had subclinical hyperthyroidism, and 337 (0·8%) had overt hyperthyroidism. Compared with euthyroidism, subclinical hypothyroidism was associated with a higher risk of pre-eclampsia (2·1% vs 3·6%; OR 1·53 [95% CI 1·09-2·15]). Subclinical hyperthyroidism, isolated hypothyroxinaemia, or TPO antibody positivity were not associated with gestational hypertension or pre-eclampsia. In continuous analyses, both a higher and a lower TSH concentration were associated with a higher risk of pre-eclampsia (p=0·0001). FT4 concentrations were not associated with the outcomes measured.InterpretationCompared with euthyroidism, subclinical hypothyroidism during pregnancy was associated with a higher risk of pre-eclampsia. There was a U-shaped association of TSH with pre-eclampsia. These results quantify the risks of gestational hypertension or pre-eclampsia in women with thyroid function test abnormalities, adding to the total body of evidence on the risk of adverse maternal and fetal outcomes of thyroid dysfunction during pregnancy. These findings have potential implications for defining the optimal treatment target in women treated with levothyroxine during pregnancy, which needs to be assessed in future interventional studies.FundingArkansas Biosciences Institute and Netherlands Organization for Scientific Research.

Project description:ObjectiveTo evaluate the performance of a UK based prediction model for estimating fat-free mass (and indirectly fat mass) in children and adolescents in non-UK settings.DesignIndividual participant data meta-analysis.Setting19 countries.Participants5693 children and adolescents (49.7% boys) aged 4 to 15 years with complete data on the predictors included in the UK based model (weight, height, age, sex, and ethnicity) and on the independently assessed outcome measure (fat-free mass determined by deuterium dilution assessment).Main outcome measuresThe outcome of the UK based prediction model was natural log transformed fat-free mass (lnFFM). Predictive performance statistics of R2, calibration slope, calibration-in-the-large, and root mean square error were assessed in each of the 19 countries and then pooled through random effects meta-analysis. Calibration plots were also derived for each country, including flexible calibration curves.ResultsThe model showed good predictive ability in non-UK populations of children and adolescents, providing R2 values of >75% in all countries and >90% in 11 of the 19 countries, and with good calibration (ie, agreement) of observed and predicted values. Root mean square error values (on fat-free mass scale) were <4 kg in 17 of the 19 settings. Pooled values (95% confidence intervals) of R2, calibration slope, and calibration-in-the-large were 88.7% (85.9% to 91.4%), 0.98 (0.97 to 1.00), and 0.01 (-0.02 to 0.04), respectively. Heterogeneity was evident in the R2 and calibration-in-the-large values across settings, but not in the calibration slope. Model performance did not vary markedly between boys and girls, age, ethnicity, and national income groups. To further improve the accuracy of the predictions, the model equation was recalibrated for the intercept in each setting so that country specific equations are available for future use.ConclusionThe UK based prediction model, which is based on readily available measures, provides predictions of childhood fat-free mass, and hence fat mass, in a range of non-UK settings that explain a large proportion of the variability in observed fat-free mass, and exhibit good calibration performance, especially after recalibration of the intercept for each population. The model demonstrates good generalisability in both low-middle income and high income populations of healthy children and adolescents aged 4-15 years.

Project description:BackgroundPrognostic factors are associated with the risk of a subsequent outcome in people with a given disease or health condition. Meta-analysis using individual participant data (IPD), where the raw data are synthesised from multiple studies, has been championed as the gold-standard for synthesising prognostic factor studies. We assessed the feasibility and conduct of this approach.MethodsA systematic review to identify published IPD meta-analyses of prognostic factors studies, followed by detailed assessment of a random sample of 20 articles published from 2006. Six of these 20 articles were from the IMPACT (International Mission for Prognosis and Analysis of Clinical Trials in traumatic brain injury) collaboration, for which additional information was also used from simultaneously published companion papers.ResultsForty-eight published IPD meta-analyses of prognostic factors were identified up to March 2009. Only three were published before 2000 but thereafter a median of four articles exist per year, with traumatic brain injury the most active research field. Availability of IPD offered many advantages, such as checking modelling assumptions; analysing variables on their continuous scale with the possibility of assessing for non-linear relationships; and obtaining results adjusted for other variables. However, researchers also faced many challenges, such as large cost and time required to obtain and clean IPD; unavailable IPD for some studies; different sets of prognostic factors in each study; and variability in study methods of measurement. The IMPACT initiative is a leading example, and had generally strong design, methodological and statistical standards. Elsewhere, standards are not always as high and improvements in the conduct of IPD meta-analyses of prognostic factor studies are often needed; in particular, continuous variables are often categorised without reason; publication bias and availability bias are rarely examined; and important methodological details and summary results are often inadequately reported.ConclusionsIPD meta-analyses of prognostic factors are achievable and offer many advantages, as displayed most expertly by the IMPACT initiative. However such projects face numerous logistical and methodological obstacles, and their conduct and reporting can often be substantially improved.

Dataset Information

External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis.

Publications

External validation of prognostic models predicting pre-eclampsia: individual participant data meta-analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets