Dataset Information

Most published meta-regression analyses based on aggregate data suffer from methodological pitfalls: a meta-epidemiological study.

ABSTRACT:

Background

Due to clinical and methodological diversity, clinical studies included in meta-analyses often differ in ways that lead to differences in treatment effects across studies. Meta-regression analysis is generally recommended to explore associations between study-level characteristics and treatment effect, however, three key pitfalls of meta-regression may lead to invalid conclusions. Our aims were to determine the frequency of these three pitfalls of meta-regression analyses, examine characteristics associated with the occurrence of these pitfalls, and explore changes between 2002 and 2012.

Methods

A meta-epidemiological study of studies including aggregate data meta-regression analysis in the years 2002 and 2012. We assessed the prevalence of meta-regression analyses with at least 1 of 3 pitfalls: ecological fallacy, overfitting, and inappropriate methods to regress treatment effects against the risk of the analysed outcome. We used logistic regression to investigate study characteristics associated with pitfalls and examined differences between 2002 and 2012.

Results

Our search yielded 580 studies with meta-analyses, of which 81 included meta-regression analyses with aggregated data. 57 meta-regression analyses were found to contain at least one pitfall (70%): 53 were susceptible to ecological fallacy (65%), 14 had a risk of overfitting (17%), and 5 inappropriately regressed treatment effects against the risk of the analysed outcome (6%). We found no difference in the prevalence of meta-regression analyses with methodological pitfalls between 2002 and 2012, nor any study-level characteristic that was clearly associated with the occurrence of any of the pitfalls.

Conclusion

The majority of meta-regression analyses based on aggregate data contain methodological pitfalls that may result in misleading findings.

SUBMITTER: Geissbuhler M

PROVIDER: S-EPMC8207572 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Rationale, aims, and objectivesCOVID-19 has caused an ongoing public health crisis. Many systematic reviews and meta-analyses have been performed to synthesize evidence for better understanding this new disease. However, some concerns have been raised about rapid COVID-19 research. This meta-epidemiological study aims to methodologically assess the current systematic reviews and meta-analyses on COVID-19.MethodsWe searched in various databases for systematic reviews with meta-analyses published between 1 January 2020 and 31 October 2020. We extracted their basic characteristics, data analyses, evidence appraisal, and assessment of publication bias and heterogeneity.ResultsWe identified 295 systematic reviews on COVID-19. The median time from submission to acceptance was 33 days. Among these systematic reviews, 73.9% evaluated clinical manifestations or comorbidities of COVID-19. Stata was the most used software programme (43.39%). The odds ratio was the most used effect measure (34.24%). Moreover, 28.14% of the systematic reviews did not present evidence appraisal. Among those reporting the risk of bias results, 14.64% of studies had a high risk of bias. Egger's test was the most used method for assessing publication bias (38.31%), while 38.66% of the systematic reviews did not assess publication bias. The I2 statistic was widely used for assessing heterogeneity (92.20%); many meta-analyses had high values of I2 . Among the meta-analyses using the random-effects model, 75.82% did not report the methods for model implementation; among those meta-analyses reporting implementation methods, the DerSimonian-Laird method was the most used one.ConclusionsThe current systematic reviews and meta-analyses on COVID-19 might suffer from low transparency, high heterogeneity, and suboptimal statistical methods. It is recommended that future systematic reviews on COVID-19 strictly follow well-developed guidelines. Sensitivity analyses may be performed to examine how the synthesized evidence might depend on different methods for appraising evidence, assessing publication bias, and implementing meta-analysis models.

Project description:ObjectivesThe published literature provides useful exposure measurements that can aid retrospective exposure assessment efforts, but the analysis of this data is challenging as it is usually reported as means, ranges, and measures of variability. We used mixed-effects meta-analysis regression models, which are commonly used to summarize health risks from multiple studies, to predict temporal trends of blood and air lead concentrations in multiple US industries from the published data while accounting for within- and between-study variability in exposure.MethodsWe extracted the geometric mean (GM), geometric standard deviation (GSD), and number of measurements from journal articles reporting blood and personal air measurements from US worksites. When not reported, we derived the GM and GSD from other summary measures. Only industries with measurements in ≥2 time points and spanning ≥10 years were included in our analyses. Meta-regression models were developed separately for each industry and sample type. Each model used the log-transformed GM as the dependent variable and calendar year as the independent variable. It also incorporated a random intercept that weighted each study by a combination of the between- and within-study variances. The within-study variances were calculated as the squared log-transformed GSD divided by the number of measurements. Maximum likelihood estimation was used to obtain the regression parameters and between-study variances.ResultsThe blood measurement models predicted statistically significant declining trends of 2-11% per year in 8 of the 13 industries. The air measurement models predicted a statistically significant declining trend (3% per year) in only one of the seven industries; an increasing trend (7% per year) was also observed for one industry. Of the five industries that met our inclusion criteria for both air and blood, the exposure declines per year tended to be slightly greater based on blood measurements than on air measurements.ConclusionsMeta-analysis provides a useful tool for synthesizing occupational exposure data to examine exposure trends that can aid future retrospective exposure assessment. Data remained too sparse to account for other exposure predictors, such as job category or sampling strategy, but this limitation may be overcome by using additional data sources.