Dataset Information

Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol.

ABSTRACT: Background: Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. Objectives: To explore in SRs with meta-analyses of the association between food/diet and health-related outcomes: (i) whether systematic reviewers selectively included study effect estimates in meta-analyses when multiple effect estimates were available; (ii) what impact selective inclusion of study effect estimates may have on meta-analytic effects, and; (iii) the risk of bias due to missing results (publication bias and selective non-reporting bias) in meta-analyses. Methods: We will systematically search for SRs with meta-analysis of the association between food/diet and health-related outcomes in a generally healthy population, published between January 2018 and June 2019. We will randomly sort titles and abstracts and screen them until we identify 50 eligible SRs. The first reported meta-analysis of a binary or continuous outcome in each SR (the 'index meta-analysis') will be evaluated. We will extract from study reports all study effect estimates that were eligible for inclusion in the index meta-analyses (e.g. from multiple instruments and time points) and will quantify and test for evidence of selective inclusion of results. We will also assess the risk of bias due to missing results in the index meta-analyses using a new tool (ROB-ME). Ethics and dissemination: Ethics approval is not required because information will only be extracted from published studies. Dissemination of the results will be through peer-reviewed publications and presentations at conferences. We will make all data collected from this study publicly available via the Open Science Framework.

SUBMITTER: Page MJ

PROVIDER: S-EPMC7025772 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol.

Page Matthew J MJ Bero Lisa L Kroeger Cynthia M CM Dai Zhaoli Z McDonald Sally S Forbes Andrew A McKenzie Joanne E JE

F1000Research 20191016

<b>Background:</b> Dietary guidelines should be informed by systematic reviews (SRs) of the available scientific evidence. However, if the SRs that underpin dietary guidelines are flawed in their design, conduct or reporting, the recommendations contained therein may be misleading or harmful. To date there has been little empirical investigation of bias due to selective inclusion of results, and bias due to missing results, in SRs of food/diet-outcome relationships. <b>Objectives:</b> To explore ...[more]

PMID: 32117567

Similar Datasets

Project description:To explore whether systematic reviewers selectively include trial effect estimates in meta-analyses when multiple are available, and what impact this may have on meta-analytic effects.Cross-sectional study.We randomly selected systematic reviews of interventions from 2 clinical specialties published between January 2010 and 2012. The first presented meta-analysis of a continuous outcome in each review was selected (index meta-analysis), and all trial effect estimates that were eligible for inclusion in the meta-analysis (eg, from multiple scales or time points) were extracted from trial reports.We calculated a statistic (the Potential Bias Index (PBI)) to quantify and test for evidence of selective inclusion. The PBI ranges from 0 to 1; values above or below 0.5 are suggestive of selective inclusion of effect estimates more or less favourable to the intervention, respectively. The impact of any potential selective inclusion was investigated by comparing the index meta-analytic standardised mean difference (SMD) to the median of a randomly constructed distribution of meta-analytic SMDs (representing the meta-analytic SMD expected when there is no selective inclusion).31 reviews (250 trials) were included. The estimated PBI was 0.57 (95% CI 0.50 to 0.63), suggesting that trial effect estimates that were more favourable to the intervention were included in meta-analyses slightly more often than expected under a process consistent with random selection; however, the 95% CI included the null hypothesis of no selective inclusion. Any potential selective inclusion did not have an important impact on the meta-analytic effects.There was no clear evidence that selective inclusion of trial effect estimates occurred in this sample of meta-analyses. Further research on selective inclusion in other clinical specialties is needed. To enable readers to assess the risk of selective inclusion bias, we recommend that systematic reviewers report the methods used to select effect estimates to include in meta-analyses.

Project description:OBJECTIVES:Assess the frequency and reasons for disagreements in risk of bias assessments for randomised controlled trials (RCTs) included in more than one Cochrane review. DESIGN:Research on research study, using cross-sectional design. DATA SOURCES:2796 Cochrane reviews published between March 2011 and September 2014. DATA SELECTION:RCTs included in more than one review. DATA EXTRACTION:Risk of bias assessment and support for judgement for five key risk of bias items. DATA SYNTHESIS:For each item, we compared risk of bias assessment made in each review and calculated proportion of agreement. Two reviewers independently analysed 50% of all disagreements by comparing support for each judgement with information from study report to evaluate whether disagreements were related to a difference in information (eg, contact the study author) or a difference in interpretation (same support for judgement but different interpretation). They also identified main reasons for different interpretation. RESULTS:1604 RCTs were included in more than one review. Proportion of agreement ranged from 57% (770/1348 trials) for incomplete outcome data to 81% for random sequence generation (1193/1466). Most common source of disagreement was difference in interpretation of the same information, ranging from 65% (88/136) for random sequence generation to 90% (56/62) for blinding of participants and personnel. Access to different information explained 32/136 (24%) disagreements for random sequence generation and 38/205 (19%) for allocation concealment. Disagreements related to difference in interpretation were frequently related to incomplete or unclear reporting in the study report (83% of disagreements related to different interpretation for random sequence generation). CONCLUSIONS:Risk of bias judgements of RCTs included in more than one Cochrane review differed substantially. Most disagreements were related to a difference in interpretation of an incomplete or unclear description in the study report. A clearer guidance on common causes of incomplete information may improve agreement.

Project description:BACKGROUND:In spite of extensive research on the effect of mutation and selection on codon usage, a general model of codon usage bias due to mutational bias has been lacking. Because most amino acids allow synonymous GC content changing substitutions in the third codon position, the overall GC bias of a genome or genomic region is highly correlated with GC3, a measure of third position GC content. For individual amino acids as well, G/C ending codons usage generally increases with increasing GC bias and decreases with increasing AT bias. Arginine and leucine, amino acids that allow GC-changing synonymous substitutions in the first and third codon positions, have codons which may be expected to show different usage patterns. PRINCIPAL FINDINGS:In analyzing codon usage bias in hundreds of prokaryotic and plant genomes and in human genes, we find that two G-ending codons, AGG (arginine) and TTG (leucine), unlike all other G/C-ending codons, show overall usage that decreases with increasing GC bias, contrary to the usual expectation that G/C-ending codon usage should increase with increasing genomic GC bias. Moreover, the usage of some codons appears nonlinear, even nonmonotone, as a function of GC bias. To explain these observations, we propose a continuous-time Markov chain model of GC-biased synonymous substitution. This model correctly predicts the qualitative usage patterns of all codons, including nonlinear codon usage in isoleucine, arginine and leucine. The model accounts for 72%, 64% and 52% of the observed variability of codon usage in prokaryotes, plants and human respectively. When codons are grouped based on common GC content, 87%, 80% and 68% of the variation in usage is explained for prokaryotes, plants and human respectively. CONCLUSIONS:The model clarifies the sometimes-counterintuitive effects that GC mutational bias can have on codon usage, quantifies the influence of GC mutational bias and provides a natural null model relative to which other influences on codon bias may be measured.

Project description:BackgroundThere is now clear systematic review evidence that measurement can affect the people being measured; much of this evidence focusses on how asking people to complete a questionnaire can result in changes in behaviour. Changes in measured behaviour and other outcomes due to this reactivity may introduce bias in otherwise well-conducted randomised controlled trials (RCTs), yielding incorrect estimates of intervention effects. Despite this, measurement reactivity is not currently adequately considered in risk of bias frameworks. The present research aims to produce a set of guidance statements on how best to avoid or minimise bias due to measurement reactivity in studies of interventions to improve health, with a particular focus on bias in RCTs.MethodsThe MERIT study consists of a series of systematic and rapid reviews, a Delphi study and an expert workshop to develop guidance on how to minimise bias in trials due to measurement reactivity. An existing systematic review on question-behaviour effects on health-related behaviours will be updated and three new rapid reviews will be conducted to identify (1) existing guidance on measurement reactivity; (2) systematic reviews of studies that have quantified the effects of measurement on outcomes relating to behaviour and affective outcomes in health and non-health contexts and (3) trials that have investigated the effects of objective measurements of behaviour on concurrent or subsequent behaviour itself. A Delphi procedure will be used to combine the views of experts with a view to reaching agreement on the scope of the guidance statements. Finally, a workshop will be held in autumn 2018, with the aim of producing a set of guidance statements that will form the central part of new MRC guidance on how best to avoid bias due to measurement reactivity in studies of interventions to improve health.DiscussionOur ambition is to produce MRC guidance on measurement reactions in trials which will be used by future trial researchers, leading to the development of trials that are less likely to be at risk of bias.

Project description:Background:Sampling bias, like survey participants' nonresponse, needs to be adequately addressed in the analysis of sampling designs. Often survey weights will be calibrated on specific covariates related to the probability of selection and nonresponse to get representative population estimates. However, such calibrated survey (CS) weights are usually constructed for cross-sectional results, but not for longitudinal analyses. For example, when the outcome of interest is time to death, and sampling selection is related to time to death and censoring, sampling is informative. Then, unweighted or CS weighted inferential statistical analyses may be biased. In 2010, Switzerland changed from a decennial full enumeration census to a yearly registry-based (i.e., data from harmonised community registries) and a survey-based census system. In the present study, we investigated the potential bias due to informative sampling when time to death is the outcome of interest, using data from the new Swiss census system. Methods:We analysed more than 6.5 million individuals aged 15 years or older from registry-based census data from years 2010 to 2013, linked with mortality records up to end of 2014. Out of this population, a target sample of 3.5% was sampled from the Swiss Federal Statistical Office (SFSO) in a stratified yearly micro census. The SFSO calculated CS weights to enable representative population estimates from the micro census. We additionally constructed inverse probability (IP) weights, where we used survival information in addition to known sampling covariates. We compared CS and IP weighted mortality rates (MR) and life expectancy (LE) with estimates from the underlying population. Additionally, we performed a simulation study under different sampling and nonresponse scenarios. Results:We found that individuals who died in 2011, had a 0.67 (95% CI [0.64-0.70]) times lower odds of participating in the 2010 micro census, using a multivariable logistic regression model with covariates age, gender, nationality, civil status, region and survival information. IP weighted MR were comparable to estimates from the total population, whereas CS weighted MR underestimated the population MR in general. The IP weighted LE estimates at age 30 years for men were 50.9 years (95% CI [50.2-51.6] years), whereas the CS weighted overestimated LE by 2.5 years. Our results from the simulation study confirmed that IP weighted models are comparable to population estimates. Conclusion:Mortality analyses based on the new Swiss survey-based census system may be biased, because of informative sampling. We conclude that mortality analyses based on census-linked survey data have to be carefully conducted, and if possible, validated by registry information to allow for unbiased interpretation and generalisation.

Project description:IntroductionThe validity of systematic reviews and meta-analysis depends on methodological quality and unbiased dissemination of trials. Our objective is to evaluate the association of estimates of treatment effects with different bias-related study characteristics in meta-analyses of interventions used for treating pain in osteoarthritis (OA). From the findings, we hope to consolidate guidance on interpreting OA trials in systematic reviews based on empirical evidence from Cochrane reviews.Methods and analysisOnly systematic reviews that compare experimental interventions with sham, placebo or no intervention control will be considered eligible. Bias will be assessed with the risk of bias tool, used according to the Cochrane Collaboration's recommendations. Furthermore, center status, trial size and funding will be assessed. The primary outcome (pain) will be abstracted from the first appearing forest plot for overall pain in the Cochrane review. Treatment effect sizes will be expressed as standardised mean differences (SMDs), where the difference in mean values available from the forest plots is divided by the pooled SD. To empirically assess the risk of bias in treatment benefits, we will perform stratified analyses of the trials from the included meta-analyses and assess the interaction between trial characteristics and treatment effect. A relevant study-level covariate is defined as one that decreases the between-study variance (τ(2), estimated as Tau-squared) as a consequence of inclusion in the mixed effects statistical model.Ethics and disseminationMeta-analyses and randomised controlled trials provide the most reliable basis for treatment of patients with OA, but the actual impact of bias is unclear. This study will systematically examine the methodological quality in OA Cochrane reviews and explore the effect estimates behind possible bias. Since our study does not collect primary data, no formal ethical assessment and informed consent are required.Trial registration numberPROSPERO (CRD42013006924).

Dataset Information

Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol.

Publications

Investigation of Risk Of Bias due to Unreported and SelecTively included results in meta-analyses of nutrition research: the ROBUST study protocol.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets