A novel approach for identifying and addressing case-mix heterogeneity in individual participant data meta-analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Vo TT
PROVIDER: S-EPMC6973268 | biostudies-literature | 2019 Dec
REPOSITORIES: biostudies-literature
Publications
A novel approach for identifying and addressing case-mix heterogeneity in individual participant data meta-analysis.
Vo Tat-Thang TT Porcher Raphael R Chaimani Anna A Vansteelandt Stijn S
Research synthesis methods 20191202 4
Case-mix heterogeneity across studies complicates meta-analyses. As a result of this, treatments that are equally effective on patient subgroups may appear to have different effectiveness on patient populations with different case mix. It is therefore important that meta-analyses be explicit for what patient population they describe the treatment effect. To achieve this, we develop a new approach for meta-analysis of randomized clinical trials, which use individual patient data (IPD) from all tr ...[more]
Similar Datasets
Project description:BackgroundStratified medicine seeks to identify patients most likely to respond to treatment. Individual participant data (IPD) network meta-analysis (NMA) models have greater power than individual trials to identify treatment-covariate interactions (TCIs). Treatment-covariate interactions contain "within" and "across" trial interactions, where the across-trial interaction is more susceptible to confounding and ecological bias.MethodsWe considered a network of IPD from 37 trials (5922 patients) for cervical cancer (2394 events), where previous research identified disease stage as a potential interaction covariate. We compare 2 models for NMA with TCIs: (1) 2 effects separating within- and across-trial interactions and (2) a single effect combining within- and across-trial interactions. We argue for a visual assessment of consistency of within- and across-trial interactions and consider more detailed aspects of interaction modelling, eg, common vs trial-specific effects of the covariate. This leads us to propose a practical framework for IPD NMA with TCIs.ResultsFollowing our framework, we found no evidence in the cervical cancer network for a treatment-stage interaction on the basis of the within-trial interaction. The NMA provided additional power for an across-trial interaction over and above the pairwise evidence. Following our proposed framework, we found that the within- and across-trial interactions should not be combined.ConclusionAcross-trial interactions are susceptible to confounding and ecological bias. It is important to separate the sources of evidence to check their consistency and identify which sources of evidence are driving the conclusion. Our framework provides practical guidance for researchers, reducing the risk of unduly optimistic interpretation of TCIs.
| S-EPMC6159880 | biostudies-literature
Project description:The diagnostic accuracy of a screening tool is often characterized by its sensitivity and specificity. An analysis of these measures must consider their intrinsic correlation. In the context of an individual participant data meta-analysis, heterogeneity is one of the main components of the analysis. When using a random-effects meta-analytic model, prediction regions provide deeper insight into the effect of heterogeneity on the variability of estimated accuracy measures across the entire studied population, not just the average. This study aimed to investigate heterogeneity via prediction regions in an individual participant data meta-analysis of the sensitivity and specificity of the Patient Health Questionnaire-9 for screening to detect major depression. From the total number of studies in the pool, four dates were selected containing roughly 25%, 50%, 75% and 100% of the total number of participants. A bivariate random-effects model was fitted to studies up to and including each of these dates to jointly estimate sensitivity and specificity. Two-dimensional prediction regions were plotted in ROC-space. Subgroup analyses were carried out on sex and age, regardless of the date of the study. The dataset comprised 17,436 participants from 58 primary studies of which 2322 (13.3%) presented cases of major depression. Point estimates of sensitivity and specificity did not differ importantly as more studies were added to the model. However, correlation of the measures increased. As expected, standard errors of the logit pooled TPR and FPR consistently decreased as more studies were used, while standard deviations of the random-effects did not decrease monotonically. Subgroup analysis by sex did not reveal important contributions for observed heterogeneity; however, the shape of the prediction regions differed. Subgroup analysis by age did not reveal meaningful contributions to the heterogeneity and the prediction regions were similar in shape. Prediction intervals and regions reveal previously unseen trends in a dataset. In the context of a meta-analysis of diagnostic test accuracy, prediction regions can display the range of accuracy measures in different populations and settings.
| S-EPMC10247712 | biostudies-literature
Project description:When combining results across related studies, a multivariate meta-analysis allows the joint synthesis of correlated effect estimates from multiple outcomes. Joint synthesis can improve efficiency over separate univariate syntheses, may reduce selective outcome reporting biases, and enables joint inferences across the outcomes. A common issue is that within-study correlations needed to fit the multivariate model are unknown from published reports. However, provision of individual participant data (IPD) allows them to be calculated directly. Here, we illustrate how to use IPD to estimate within-study correlations, using a joint linear regression for multiple continuous outcomes and bootstrapping methods for binary, survival and mixed outcomes. In a meta-analysis of 10 hypertension trials, we then show how these methods enable multivariate meta-analysis to address novel clinical questions about continuous, survival and binary outcomes; treatment-covariate interactions; adjusted risk/prognostic factor effects; longitudinal data; prognostic and multiparameter models; and multiple treatment comparisons. Both frequentist and Bayesian approaches are applied, with example software code provided to derive within-study correlations and to fit the models.
| S-EPMC4847645 | biostudies-literature
Project description:BackgroundDepression is a highly common and recurrent condition. Predicting who is at most risk of relapse or recurrence can inform clinical practice. Applying machine-learning methods to Individual Participant Data (IPD) can be promising to improve the accuracy of risk predictions.MethodsIndividual data of four Randomized Controlled Trials (RCTs) evaluating antidepressant treatment compared to psychological interventions with tapering ([Formula: see text]) were used to identify predictors of relapse and/or recurrence. Ten baseline predictors were assessed. Decision trees with and without gradient boosting were applied. To study the robustness of decision-tree classifications, we also performed a complementary logistic regression analysis.ResultsThe combination of age, age of onset of depression, and depression severity significantly enhances the prediction of relapse risk when compared to classifiers solely based on depression severity. The studied decision trees can (i) identify relapse patients at intake with an accuracy, specificity, and sensitivity of about 55% (without gradient boosting) and 58% (with gradient boosting), and (ii) slightly outperform classifiers that are based on logistic regression.ConclusionsDecision tree classifiers based on multiple-rather than single-risk indicators may be useful for developing treatment stratification strategies. These classification models have the potential to contribute to the development of methods aimed at effectively prioritizing treatment for those individuals who require it the most. Our results also underline the existing gaps in understanding how to accurately predict depressive relapse.
| S-EPMC10644580 | biostudies-literature
Project description:IntroductionMeta-analysis traditionally uses aggregate data from published reports. Individual Participant Data (IPD) meta-analysis, which obtains and synthesizes participant-level data, is potentially more informative, but resource-intensive. The impact on the findings of meta-analyses using IPD in comparison with aggregate data has rarely been formally evaluated.MethodsWe conducted a secondary analysis of a Cochrane systematic review of skincare interventions for preventing eczema and food allergy in infants to identify the impact of the analytical choice on the review's findings. We used aggregate data meta-analysis only and contrasted the results against those of the originally published IPD meta-analysis. All meta-analysis used random effects inverse variance models. Certainty of evidence was evaluated using GRADE.ResultsThe pooled treatment effects for the Cochrane systematic review's co-primary outcomes of eczema and food allergy were similar in IPD meta-analysis (eczema RR 1.03, 95% CI 0.81, 1.31; I2 41%, 7 studies 3075 participants), and aggregate meta-analysis (eczema RR 1.01 95% CI 0.77, 1.33; I2 53%, 7 studies, 3089 participants). In aggregate meta-analysis, the statistical heterogeneity could not be explained but using IPD it was explained by one trial which used a different, bathing intervention. For IPD meta-analysis, risk of bias was assessed as lower and more adverse event data were available compared with aggregate meta-analysis. This resulted in higher certainty of evidence, especially for adverse events. IPD meta-analysis enabled analysis of treatment interactions by age and hereditary eczema risk; and analysis of the effect of treatment adherence using pooled complier-adjusted-causal-effect analysis, none of which was possible in aggregate meta-analysis.ConclusionsFor this systematic review, IPD did not significantly change primary outcome risk ratios compared with aggregate data meta-analysis. However, certainty of evidence, safety outcomes, subgroup and adherence analyses were significantly different using IPD. This demonstrates benefits of adopting an IPD approach to meta-analysis.
| S-EPMC9302682 | biostudies-literature
Project description:ImportanceThe pharmaceutical industry has made substantial investments in developing processes for sharing individual-participant data (IPD) from clinical trials. However, the utility and completeness of shared IPD and supporting documents must be evaluated to ensure the potential for scientific advancements from the data sharing ecosystem can be realized.ObjectiveTo assess the utility and completeness of IPD and supporting documents provided from industry-sponsored clinical trials.Design, setting, and participantsFrom February 9, 2022, to February 9, 2023, 91 of 203 clinical trials supporting US Food and Drug Administration registrations of anticancer medicines for the treatment of solid tumors from the past decade were confirmed as eligible for IPD request. This quality improvement study performed a retrospective audit of the utility and completeness of the IPD and supporting documents provided from the 91 clinical trials for a planned meta-analysis.ExposuresRequest for IPD from 91 clinical oncology trials indicated as eligible for the request.Main outcomes and measuresThe utility and completeness of the IPD and supporting documents provided.ResultsThe IPD packages were obtained from 70 of 91 requested clinical trials (77%). The median time to data provision was 123 (range, 117-352) days. Redactions were observed in 18 of the acquired IPD packages (26%) for outcome data, 11 (16%) for assessment variables, and 19 (27%) for adjustment data. Additionally, 20 IPD packages (29%) lacked a clinical study report, 4 (6%) had incomplete or missing data dictionaries, and 20 (29%) were missing anonymization or redaction description files. Access to IPD from 21 eligible trials (23%) was not granted.Conclusions and relevanceIn this quality improvement study, there was substantial variability within the provided IPD packages regarding the completeness of key data variables and supporting documents. To improve the data sharing ecosystem, key areas for enhancement include (1) ensuring that clinical trials are eligible for IPD sharing, (2) making eligible IPD transparently accessible, and (3) ensuring that IPD packages meet a standard of utility and completeness.
| S-EPMC10557028 | biostudies-literature
Project description:For analysis of the main effects of SNPs, meta-analysis of summary results from individual studies has been shown to provide comparable results as "mega-analysis" that jointly analyzes the pooled participant data from the available studies. This fact revolutionized the genetic analysis of complex traits through large GWAS consortia. Investigations of gene-environment (G×E) interactions are on the rise since they can potentially explain a part of the missing heritability and identify individuals at high risk for disease. However, for analysis of gene-environment interactions, it is not known whether these methods yield comparable results. In this empirical study, we report that the results from both methods were largely consistent for all four tests; the standard 1 degree of freedom (df) test of main effect only, the 1 df test of the main effect (in the presence of interaction effect), the 1 df test of the interaction effect, and the joint 2 df test of main and interaction effects. They provided similar effect size and standard error estimates, leading to comparable P-values. The genomic inflation factors and the number of SNPs with various thresholds were also comparable between the two approaches. Mega-analysis is not always feasible especially in very large and diverse consortia since pooling of raw data may be limited by the terms of the informed consent. Our study illustrates that meta-analysis can be an effective approach also for identifying interactions. To our knowledge, this is the first report investigating meta-versus mega-analyses for interactions.
| S-EPMC4332385 | biostudies-literature
Project description:Meta-analytic methods for combining data from multiple intervention trials are commonly used to estimate the effectiveness of an intervention. They can also be extended to study comparative effectiveness, testing which of several alternative interventions is expected to have the strongest effect. This often requires network meta-analysis (NMA), which combines trials involving direct comparison of two interventions within the same trial and indirect comparisons across trials. In this paper, we extend existing network methods for main effects to examining moderator effects, allowing for tests of whether intervention effects vary for different populations or when employed in different contexts. In addition, we study how the use of individual participant data may increase the sensitivity of NMA for detecting moderator effects, as compared with aggregate data NMA that employs study-level effect sizes in a meta-regression framework. A new NMA diagram is proposed. We also develop a generalized multilevel model for NMA that takes into account within-trial and between-trial heterogeneity and can include participant-level covariates. Within this framework, we present definitions of homogeneity and consistency across trials. A simulation study based on this model is used to assess effects on power to detect both main and moderator effects. Results show that power to detect moderation is substantially greater when applied to individual participant data as compared with study-level effects. We illustrate the use of this method by applying it to data from a classroom-based randomized study that involved two sub-trials, each comparing interventions that were contrasted with separate control groups. Copyright © 2016 John Wiley & Sons, Ltd.
| S-EPMC4893003 | biostudies-literature
Project description:Multiple imputation is a strategy for the analysis of incomplete data such that the impact of the missingness on the power and bias of estimates is mitigated. When data from multiple studies are collated, we can propose both within-study and multilevel imputation models to impute missing data on covariates. It is not clear how to choose between imputation models or how to combine imputation and inverse-variance weighted meta-analysis methods. This is especially important as often different studies measure data on different variables, meaning that we may need to impute data on a variable which is systematically missing in a particular study. In this paper, we consider a simulation analysis of sporadically missing data in a single covariate with a linear analysis model and discuss how the results would be applicable to the case of systematically missing data. We find in this context that ensuring the congeniality of the imputation and analysis models is important to give correct standard errors and confidence intervals. For example, if the analysis model allows between-study heterogeneity of a parameter, then we should incorporate this heterogeneity into the imputation model to maintain the congeniality of the two models. In an inverse-variance weighted meta-analysis, we should impute missing data and apply Rubin's rules at the study level prior to meta-analysis, rather than meta-analyzing each of the multiple imputations and then combining the meta-analysis estimates using Rubin's rules. We illustrate the results using data from the Emerging Risk Factors Collaboration.
| S-EPMC3963448 | biostudies-literature
Project description:One-stage meta-analysis of individual participant data (IPD) poses several statistical and computational challenges. For time-to-event outcomes, the approach requires the estimation of complicated nonlinear mixed-effects models that are flexible enough to realistically capture the most important characteristics of the IPD. We present a model class that incorporates general normally distributed random effects into linear transformation models. We discuss extensions to model between-study heterogeneity in baseline risks and covariate effects and also relax the assumption of proportional hazards. Within the proposed framework, data with arbitrary random censoring patterns can be handled. The accompanying $\textsf{R}$ package tramME utilizes the Laplace approximation and automatic differentiation to perform efficient maximum likelihood estimation and inference in mixed-effects transformation models. We compare several variants of our model to predict the survival of patients with chronic obstructive pulmonary disease using a large data set of prognostic studies. Finally, a simulation study is presented that verifies the correctness of the implementation and highlights its efficiency compared to an alternative approach.
| S-EPMC9566326 | biostudies-literature