Ontology highlight
ABSTRACT: Objective
To assess the risk of bias associated with missing outcome data in systematic reviews.Design
Imputation study.Setting
Systematic reviews.Population
100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome.Main outcome measures
Median percentage change in the relative effect estimate when applying each of the following assumption (four commonly discussed but implausible assumptions (best case scenario, none had the event, all had the event, and worst case scenario) and four plausible assumptions for missing data based on the informative missingness odds ratio (IMOR) approach (IMOR 1.5 (least stringent), IMOR 2, IMOR 3, IMOR 5 (most stringent)); percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. Sensitivity analyses based on the eight different methods of handling missing data were conducted.Results
100 systematic reviews with 653 randomised controlled trials were included. When applying the implausible but commonly discussed assumptions, the median change in the relative effect estimate varied from 0% to 30.4%. The percentage of meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario), and 26% changed direction with the worst case scenario. When applying the plausible assumptions, the median percentage change in relative effect estimate varied from 1.4% to 7.0%. The percentage of meta-analyses crossing the threshold of the null effect varied from 6% (IMOR 1.5) to 22% (IMOR 5) of meta-analyses, and 2% changed direction with the most stringent (IMOR 5).Conclusion
Even when applying plausible assumptions to the outcomes of participants with definite missing data, the average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. Systematic review authors should present the potential impact of missing outcome data on their effect estimates and use this to inform their overall GRADE (grading of recommendations assessment, development, and evaluation) ratings of risk of bias and their interpretation of the results.
SUBMITTER: Kahale LA
PROVIDER: S-EPMC7448113 | biostudies-literature | 2020 Aug
REPOSITORIES: biostudies-literature
Kahale Lara A LA Khamis Assem M AM Diab Batoul B Chang Yaping Y Lopes Luciane Cruz LC Agarwal Arnav A Li Ling L Mustafa Reem A RA Koujanian Serge S Waziry Reem R Busse Jason W JW Dakik Abeer A Schünemann Holger J HJ Hooft Lotty L Scholten Rob Jpm RJ Guyatt Gordon H GH Akl Elie A EA
BMJ (Clinical research ed.) 20200826
<h4>Objective</h4>To assess the risk of bias associated with missing outcome data in systematic reviews.<h4>Design</h4>Imputation study.<h4>Setting</h4>Systematic reviews.<h4>Population</h4>100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome.<h4>Main outcome measures</h4>Median percentage change in the relative effect estimate when applying each of the following assumption (four commonly disc ...[more]