Estimation of Effect Heterogeneity in Rare Events Meta-Analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Holling H
PROVIDER: S-EPMC9433364 | biostudies-literature | 2022 Sep
REPOSITORIES: biostudies-literature
Publications
Estimation of Effect Heterogeneity in Rare Events Meta-Analysis.
Holling Heinz H Jansen Katrin K Böhning Walailuck W Böhning Dankmar D Martin Susan S Sangnawakij Patarawan P
Psychometrika 20220208 3
The paper outlines several approaches for dealing with meta-analyses of count outcome data. These counts are the accumulation of occurred events, and these events might be rare, so a special feature of the meta-analysis is dealing with low counts including zero-count studies. Emphasis is put on approaches which are state of the art for count data modelling including mixed log-linear (Poisson) and mixed logistic (binomial) regression as well as nonparametric mixture models for count data of Poiss ...[more]
Similar Datasets
Project description:Meta-analysis, the statistical procedure for combining results from multiple independent studies, has been widely used in medical research to evaluate intervention efficacy and drug safety. In many practical situations, treatment effects vary notably among the collected studies, and the variation, often modeled by the between-study variance parameter τ 2, can greatly affect the inference of the overall effect size. In the past, comparative studies have been conducted for both point and interval estimation of τ 2. However, most are incomplete, only including a limited subset of existing methods, and some are outdated. Further, none of the studies covers descriptive measures for assessing the level of heterogeneity, nor are they focused on rare binary events that require special attention. We summarize by far the most comprehensive set including 11 descriptive measures, 23 estimators, and 16 confidence intervals. In addition to providing synthesized information, we further categorize these methods according to their key features. We then evaluate their performance based on simulation studies that examine various realistic scenarios for rare binary events, with an illustration using a data example of a gestational diabetes meta-analysis. We conclude that there is no uniformly "best" method. However, methods with consistently better performance do exist in the context of rare binary events, and we provide practical guidelines based on numerical evidences.
| S-EPMC7901832 | biostudies-literature
Project description:Network meta-analysis (NMA) of rare events has attracted little attention in the literature. Until recently, networks of interventions with rare events were analyzed using the inverse-variance NMA approach. However, when events are rare the normal approximations made by this model can be poor and effect estimates are potentially biased. Other methods for the synthesis of such data are the recent extension of the Mantel-Haenszel approach to NMA or the use of the noncentral hypergeometric distribution. In this article, we suggest a new common-effect NMA approach that can be applied even in networks of interventions with extremely low or even zero number of events without requiring study exclusion or arbitrary imputations. Our method is based on the implementation of the penalized likelihood function proposed by Firth for bias reduction of the maximum likelihood estimate to the logistic expression of the NMA model. A limitation of our method is that heterogeneity cannot be taken into account as an additive parameter as in most meta-analytical models. However, we account for heterogeneity by incorporating a multiplicative overdispersion term using a two-stage approach. We show through simulation that our method performs consistently well across all tested scenarios and most often results in smaller bias than other available methods. We also illustrate the use of our method through two clinical examples. We conclude that our "penalized likelihood NMA" approach is promising for the analysis of binary outcomes with rare events especially for networks with very few studies per comparison and very low control group risks.
| S-EPMC9805041 | biostudies-literature
Project description:Random-effects (RE) meta-analysis is a crucial approach for combining results from multiple independent studies that exhibit heterogeneity. Recently, two frequentist goodness-of-fit (GOF) tests were proposed to assess the fit of RE model. However, they tend to perform poorly when assessing rare binary events. Under a general binomial-normal framework, we propose a novel GOF test for the meta-analysis of rare events. Our method is based on pivotal quantities that play an important role in Bayesian model assessment. It further adopts the Cauchy combination idea proposed in a 2019 JASA paper, to combine dependent p-values computed using posterior samples from Markov Chain Monte Carlo. The advantages of our method include clear conception and interpretation, incorporation of all data including double zeros without the need for artificial correction, well-controlled Type I error, and generally improved ability in detecting model misfits compared to previous GOF methods. We illustrate the proposed method via simulation and three real data applications.
| S-EPMC10584850 | biostudies-literature
Project description:When a meta-analysis consists of a few small trials that report zero events, accounting for heterogeneity in the (interval) estimation of the overall effect is challenging. Typically, we predefine meta-analytical methods to be employed. In practice, data poses restrictions that lead to deviations from the pre-planned analysis, such as the presence of zero events in at least one study arm. We aim to explore heterogeneity estimators behaviour in estimating the overall effect across different levels of sparsity of events. We performed a simulation study that consists of two evaluations. We considered an overall comparison of estimators unconditional on the number of observed zero cells and an additional one by conditioning on the number of observed zero cells. Estimators that performed modestly robust when (interval) estimating the overall treatment effect across a range of heterogeneity assumptions were the Sidik-Jonkman, Hartung-Makambi and improved Paul-Mandel. The relative performance of estimators did not materially differ between making a predefined or data-driven choice. Our investigations confirmed that heterogeneity in such settings cannot be estimated reliably. Estimators whose performance depends strongly on the presence of heterogeneity should be avoided. The choice of estimator does not need to depend on whether or not zero cells are observed.
| S-EPMC5898531 | biostudies-literature
Project description:ObjectiveTo estimate the proportion of systematic reviews that meet the optimal information size (OIS) and assess the impact heterogeneity and effect size have on the OIS estimate by type of outcome (eg, mortality, semiobjective or subjective).MethodsWe carried out searches of Medline and Cochrane to retrieve meta-analyses published in systematic reviews from 2010 to 2012. We estimated the OIS using Trial Sequential Analysis software (TSA V.0.9) and based on several heterogeneity and effect size scenarios, stratifying by type of outcome (mortality/semiobjective/subjective) and by Cochrane/non-Cochrane reviews.ResultsWe included 137 meta-analyses out of 218 (63%) potential systematic reviews (one meta-analysis from each systematic review). Of these reviews, 83 (61%) were Cochrane and 54 (39%) non-Cochrane. The Cochrane reviews included a mean of 6.5 (SD 6.1) studies and the non-Cochrane included a mean of 13.2 (SD 10.2) studies. The mean number of patients was 2619.1 (SD 6245.8 or median 586.0) for the Cochrane and 19 888.5 (SD 32 925.7 or median 6566.5) patients for the non-Cochrane reviews. The percentage of systematic reviews that achieved the OIS for all-cause mortality outcome were 0% Cochrane and 25% for non-Cochrane reviews; for semiobjective outcome 17% for Cochrane and 46% for non-Cochrane reviews and for subjective outcome 45% for Cochrane and 72% for non-Cochrane reviews.ConclusionsThe number of systematic reviews that meet an optimal information size is low and varies depending on the type of outcome and the type of publication. Less than half of primary outcomes synthesised in systematic reviews achieve the OIS, and therefore the conclusions are subject to substantial uncertainty.
| S-EPMC5695413 | biostudies-literature
Project description:Meta-analysis has been widely used to synthesize information from related studies to achieve reliable findings. However, in studies of rare events, the event counts are often low or even zero, and so standard meta-analysis methods such as fixed-effect models with continuity correction may cause substantial bias in estimation. Recently, Bhaumik et al. developed a simple average estimator for the overall treatment effect based on a random effects model. They proved that the simple average method with the continuity correction factor 0.5 (SA_0.5) is the least biased for large samples and showed via simulation that it has superior performance when compared with other commonly used estimators. However, the random effects models used in previous work are restrictive because they all assume that the variability in the treatment group is equal to or always greater than that in the control group. Under a general framework that explicitly allows treatment groups with unequal variability but assumes no direction, we prove that SA_0.5 is still the least biased for large samples. Meanwhile, to account for a trade-off between the bias and variance in estimation, we consider the mean squared error to assess estimation efficiency and show that SA_0.5 fails to minimize the mean squared error. Under a new random effects model that accommodates groups with unequal variability, we thoroughly compare the performance of various methods for both large and small samples via simulation and draw conclusions about when to use which method in terms of bias, mean squared error, type I error, and confidence interval coverage. A data example of rosiglitazone meta-analysis is used to provide further comparison.
| S-EPMC5599377 | biostudies-literature
Project description:Background/aimsRegulatory approval of a drug or device involves an assessment of not only the benefits but also the risks of adverse events associated with the therapeutic agent. Although randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness, the number of treated patients in a single RCT may not be enough to detect a rare but serious side effect of the treatment. Meta-analysis plays an important role in the evaluation of the safety of medical products and has advantage over analyzing a single RCT when estimating the rate of adverse events.MethodsIn this article, we compare 15 widely used meta-analysis models under both Bayesian and frequentist frameworks when outcomes are extremely infrequent or rare. We present extensive simulation study results and then apply these methods to a real meta-analysis that considers RCTs investigating the effect of rosiglitazone on the risks of myocardial infarction and of death from cardiovascular causes.ResultsOur simulation studies suggest that the beta hyperprior method modeling treatment group-specific parameters and accounting for heterogeneity performs the best. Most models ignoring between-study heterogeneity give poor coverage probability when such heterogeneity exists. In the data analysis, different methods provide a wide range of log odds ratio estimates between rosiglitazone and control treatments with a mixed conclusion on their statistical significance based on 95% confidence (or credible) intervals.ConclusionIn the rare event setting, treatment effect estimates obtained from traditional meta-analytic methods may be biased and provide poor coverage probability. This trend worsens when the data have large between-study heterogeneity. In general, we recommend methods that first estimate the summaries of treatment-specific risks across studies and then relative treatment effects based on the summaries when appropriate. Furthermore, we recommend fitting various methods, comparing the results and model performance, and investigating any significant discrepancies among them.
| S-EPMC8041270 | biostudies-literature
Project description:Random-effects meta-analyses are used to combine evidence of treatment effects from multiple studies. Since treatment effects may vary across trials due to differences in study characteristics, heterogeneity in treatment effects between studies must be accounted for to achieve valid inference. The standard model for random-effects meta-analysis assumes approximately normal effect estimates and a normal random-effects model. However, standard methods based on this model ignore the uncertainty in estimating the between-trial heterogeneity. In the special setting of only two studies and in the presence of heterogeneity, we investigate here alternatives such as the Hartung-Knapp-Sidik-Jonkman method (HKSJ), the modified Knapp-Hartung method (mKH, a variation of the HKSJ method) and Bayesian random-effects meta-analyses with priors covering plausible heterogeneity values; R code to reproduce the examples is presented in an appendix. The properties of these methods are assessed by applying them to five examples from various rare diseases and by a simulation study. Whereas the standard method based on normal quantiles has poor coverage, the HKSJ and mKH generally lead to very long, and therefore inconclusive, confidence intervals. The Bayesian intervals on the whole show satisfying properties and offer a reasonable compromise between these two extremes.
| S-EPMC5516158 | biostudies-literature
Project description:BackgroundWe explored whether clinically relevant baseline characteristics of patients with type 2 diabetes can modify the effect of glucagon-like peptide-1 receptor agonists (GLP-1 RA) or sodium-glucose cotransporter-2 inhibitors (SGLT-2i) on the risk of major adverse cardiovascular events (MACE).MethodsWe investigated Medline and EMBASE through June 2019. We included randomized clinical trials reporting the effect of GLP-1 RA or SGLT-2i on MACE in subgroups of patients with type 2 diabetes, identified through key baseline factors: established cardiovascular disease; heart failure; chronic kidney disease; uncontrolled diabetes; duration of diabetes; hypertension; obesity; age; gender and race. Hazard ratios (HRs) and 95% confidence intervals (CIs) from trials were meta-analyzed using random-effects models.ResultsTen trials enrolling 89,790 patients were included in the analyses. Subgroup meta-analyses showed a 14% risk reduction of MACE in patients with established cardiovascular disease [GLP1-RA: HR, 0.86 (95% CI, 0.80-0.93); SGLT-2i: 0.86 (0.80-0.93)], and no effect in at-risk patients without history of cardiovascular events [GLP1-RA: 0.94 (0.82-1.07); SGLT-2i: 1.00 (0.87-1.16)]. We observed a trend toward larger treatment benefits with SGLT-2i among patients with chronic kidney disease [0.82 (0.69-0.97)], and patients with uncontrolled diabetes for both GLP1-RA or SGLT-2i [GLP1-RA: 0.82 (0.71-0.95); SGLT-2i: 0.84 (0.75-0.95)]. Uncontrolled hypertension, obesity, gender, age and race did not appear to modify the effect of these drugs.ConclusionsIn this exploratory analysis, history of cardiovascular disease appeared to modify the treatment effect of SGLT2i or GLP1-RA on MACE. Chronic kidney disease and uncontrolled diabetes should be further investigated as potential effect modifiers.
| S-EPMC7525990 | biostudies-literature
Project description:Meta-analysis of clinical trials targeting rare events face particular challenges when the data lack adequate number of events and are susceptible to high levels of heterogeneity. The standard meta-analysis methods (DerSimonian Laird (DL) and Mantel-Haenszel (MH)) often lead to serious distortions because of such data sparsity. Applications of the methods suited to specific incidence and heterogeneity characteristics are lacking, thus we compared nine available methods in a simulation study. We generated 360 meta-analysis scenarios where each considered different incidences, sample sizes, between-study variance (heterogeneity) and treatment allocation. We include globally recommended methods such as inverse-variance fixed/random-effect (IV-FE/RE), classical-MH, MH-FE, MH-DL, Peto, Peto-DL and the two extensions for MH bootstrapped-DL (bDL) and Peto-bDL. Performance was assessed on mean bias, mean error, coverage and power. In the absence of heterogeneity, the coverage and power when combined revealed small differences in meta-analysis involving rare and very rare events. The Peto-bDL method performed best, but only in smaller sample sizes involving rare events. For medium-to-larger sample sizes, MH-bDL was preferred. For meta-analysis involving very rare events, Peto-bDL was the best performing method which was sustained across all sample sizes. However, in meta-analysis with 20% or more heterogeneity, the coverage and power were insufficient. Performance based on mean bias and mean error was almost identical across methods. To conclude, in meta-analysis of rare binary outcomes, our results suggest that Peto-bDL is better in both rare and very rare event settings in meta-analysis with limited sample sizes. However, when heterogeneity is large, the coverage and power to detect rare events are insufficient. Whilst this study shows that some of the less studied methods appear to have good properties under sparse data scenarios, further work is needed to assess them against the more complex distributional-based methods to understand their overall performances.
| S-EPMC8411477 | biostudies-literature