Dataset Information

Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysis.

ABSTRACT:

Background

Missing participant outcome data (MOD) are ubiquitous in systematic reviews with network meta-analysis (NMA) as they invade from the inclusion of clinical trials with reported participant losses. There are available strategies to address aggregate MOD, and in particular binary MOD, while considering the missing at random (MAR) assumption as a starting point. Little is known about their performance though regarding the meta-analytic parameters of a random-effects model for aggregate binary outcome data as obtained from trial-reports (i.e. the number of events and number of MOD out of the total randomised per arm).

Methods

We used four strategies to handle binary MOD under MAR and we classified these strategies to those modelling versus excluding/imputing MOD and to those accounting for versus ignoring uncertainty about MAR. We investigated the performance of these strategies in terms of core NMA estimates by performing both an empirical and simulation study using random-effects NMA based on electrical network theory. We used Bland-Altman plots to illustrate the agreement between the compared strategies, and we considered the mean bias, coverage probability and width of the confidence interval to be the frequentist measures of performance.

Results

Modelling MOD under MAR agreed with exclusion and imputation under MAR in terms of estimated log odds ratios and inconsistency factor, whereas accountability or not of the uncertainty regarding MOD affected intervention hierarchy and precision around the NMA estimates: strategies that ignore uncertainty about MOD led to more precise NMA estimates, and increased between-trial variance. All strategies showed good performance for low MOD (<5%), consistent evidence and low between-trial variance, whereas performance was compromised for large informative MOD (>?20%), inconsistent evidence and substantial between-trial variance, especially for strategies that ignore uncertainty due to MOD.

Conclusions

The analysts should avoid applying strategies that manipulate MOD before analysis (i.e. exclusion and imputation) as they implicate the inferences negatively. Modelling MOD, on the other hand, via a pattern-mixture model to propagate the uncertainty about MAR assumption constitutes both conceptually and statistically proper strategy to address MOD in a systematic review.

SUBMITTER: Spineli LM

PROVIDER: S-EPMC7049189 | biostudies-literature | 2020 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysis.

Spineli Loukia M LM Kalyvas Chrysostomos C

BMC medical research methodology 20200228 1

<h4>Background</h4>Missing participant outcome data (MOD) are ubiquitous in systematic reviews with network meta-analysis (NMA) as they invade from the inclusion of clinical trials with reported participant losses. There are available strategies to address aggregate MOD, and in particular binary MOD, while considering the missing at random (MAR) assumption as a starting point. Little is known about their performance though regarding the meta-analytic parameters of a random-effects model for aggr ...[more]

PMID: 32111167

Similar Datasets

Project description:BackgroundA number of strategies have been proposed to handle missing binary outcome data (MOD) in systematic reviews. However, none of these have been evaluated empirically in a series of published systematic reviews.MethodsUsing published systematic reviews with network meta-analysis (NMA) from a wide range of health-related fields, we evaluated comparatively the most frequently described Bayesian modelling strategies for MOD in terms of log odds ratio (log OR), between-trial variance, inconsistency factor (i.e. difference between direct and indirect estimates for a comparison), surface under the cumulative ranking (SUCRA) and rankings. We extended the Bayesian random-effects NMA model to incorporate the informative missingness odds ratio (IMOR) parameter, and applied the node-splitting approach to investigate inconsistency locally. We considered both pattern-mixture and selection models, different structures for prior distribution of log IMOR, and different scenarios for MOD. To illustrate level of agreement between different strategies and scenarios, we used Bland-Altman plots.ResultsAddressing MOD using extreme scenarios and ignoring the uncertainty about the scenarios led to systematically different and more precise log ORs compared to modelling MOD under the missing at random (MAR) assumption. Hierarchical structure of log IMORs led to lower between-trial variance, especially in the case of substantial MOD. Assuming common-within-network or trial-specific log IMORs yielded similar posterior results for all NMA estimates, whereas intervention-specific structure systematically inflated uncertainty around log ORs and SUCRAs. Pattern-mixture model agreed with selection model, particularly under the trial-specific structure; however, selection model systematically reduced precision around log IMORs. Overall, different strategies and scenarios mostly had good agreement in the case of low MOD.ConclusionsAddressing MOD using extreme scenarios and/or ignoring the uncertainty about the scenarios may negatively affect NMA estimates. Modelling MOD via the IMOR parameter can ensure bias-adjusted estimates and offer valuable insights into missingness mechanisms. The researcher should seek an expert opinion in order to decide on the structure of log IMOR that best aligns to the condition and interventions studied and to define a proper prior distribution for log IMOR. Our findings also apply to pairwise meta-analyses.

Project description:BackgroundTrials with binary outcomes can be synthesised using within-trial exact likelihood or approximate normal likelihood in one-stage or two-stage approaches, respectively. The performance of the one-stage and the two-stage approaches has been documented extensively in the literature. However, little is known about how these approaches behave in the presence of missing outcome data (MOD), which are ubiquitous in clinical trials. In this work, we compare the one-stage versus two-stage approach via a pattern-mixture model in the network meta-analysis using Bayesian methods to handle MOD appropriately.MethodsWe used 29 published networks to empirically compare the two approaches concerning the relative treatment effects of several competing interventions and the between-trial variance (?2), while considering the extent and level of balance of MOD in the included trials. We additionally conducted a simulation study to compare the competing approaches regarding the bias and width of the 95% credible interval of the (summary) log odds ratios (OR) and ?2 in the presence of moderate and large MOD.ResultsThe empirical study did not reveal any systematic bias between the compared approaches regarding the log OR, but showed systematically larger uncertainty around the log OR under the one-stage approach for networks with at least one small trial or low event risk and moderate MOD. For these networks, the simulation study revealed that the bias in log OR for comparisons with the reference intervention in the network was relatively higher in the two-stage approach. Contrariwise, the bias in log OR for the remaining comparisons was relatively higher in the one-stage approach. Overall, bias increased for large MOD. For these networks, the empirical results revealed slightly higher ?2 estimates under the one-stage approach irrespective of the extent of MOD. The one-stage approach also led to less precise log OR and ?2 when compared with the two-stage approach for large MOD.ConclusionsDue to considerable bias in the log ORs overall, especially for large MOD, none of the competing approaches was superior. Until a more competent model is developed, the researchers may prefer the one-stage approach to handle MOD, while acknowledging its limitations.

Project description:ObjectiveTo assess the risk of bias associated with missing outcome data in systematic reviews.DesignImputation study.SettingSystematic reviews.Population100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome.Main outcome measuresMedian percentage change in the relative effect estimate when applying each of the following assumption (four commonly discussed but implausible assumptions (best case scenario, none had the event, all had the event, and worst case scenario) and four plausible assumptions for missing data based on the informative missingness odds ratio (IMOR) approach (IMOR 1.5 (least stringent), IMOR 2, IMOR 3, IMOR 5 (most stringent)); percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. Sensitivity analyses based on the eight different methods of handling missing data were conducted.Results100 systematic reviews with 653 randomised controlled trials were included. When applying the implausible but commonly discussed assumptions, the median change in the relative effect estimate varied from 0% to 30.4%. The percentage of meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario), and 26% changed direction with the worst case scenario. When applying the plausible assumptions, the median percentage change in relative effect estimate varied from 1.4% to 7.0%. The percentage of meta-analyses crossing the threshold of the null effect varied from 6% (IMOR 1.5) to 22% (IMOR 5) of meta-analyses, and 2% changed direction with the most stringent (IMOR 5).ConclusionEven when applying plausible assumptions to the outcomes of participants with definite missing data, the average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. Systematic review authors should present the potential impact of missing outcome data on their effect estimates and use this to inform their overall GRADE (grading of recommendations assessment, development, and evaluation) ratings of risk of bias and their interpretation of the results.

Project description:BackgroundSeveral systemic agents have been approved for use in advanced hepatocellular carcinoma (aHCC). However, it is unclear which treatment is superior in either the first- or second-line settings due to the paucity of head-to-head comparative trials. Therefore, we have conducted a systematic review and network meta-analysis for the indirect comparison of the systemic agents in the first line and second line settings.MethodsRandomized clinical trials evaluating systemic agents in first and second line settings in aHCC from inception to April 2020 were identified by searching PubMed, EMBASE, and Cochrane Databases and the annual ASCO and ESMO conferences from 2017 to 2020. Studies in English reporting clinical outcomes including overall survival (OS), progression-free survival (PFS), and objective response rate (ORR) were included. The primary outcomes of interest were pooled hazard ratios (HR) of OS and pooled odds ratios (OR) of ORR in first line studies and pooled HR of PFS and OR of ORR for second line studies. Additionally, OS for second line agents were reported in the qualitative analysis.ResultsOverall, first line studies comprised 8335 patients (13 studies) and second line studies comprised 4612 patients (11 studies). In the first line setting, atezolizumab plus bevacizumab was associated with the highest OS benefit over sorafenib (HR 0.58, 95% CI, 0.42-0.80; P-score 0.993). Additionally, lenvatinib was associated with the greatest ORR benefit (OR 3.34, 95% CI, 2.17-5.14; P-score 0.080) in the first line setting. In the second line setting, cabozantinib was associated with the highest PFS benefit over placebo (HR 0.44, 95% CI, 0.29-0.66; P-score 0.854) as well as the highest ORR benefit (OR 9.40, 95% CI, 1.25-70.83, P-score, 0.266).ConclusionAtezolizumab plus bevacizumab appears to have superior efficacy among first line agents whereas cabozantinib appears to be superior in the second line setting. Further studies are warranted to determine whether the type of prior therapy received affects the efficacy of subsequent second line therapy.

Dataset Information

Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysis.

Background

Methods

Results

Conclusions

Publications

Comparison of exclusion, imputation and modelling of missing binary outcome data in frequentist network meta-analysis.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets