Dataset Information

Methodological characteristics and treatment effect sizes in oral health randomised controlled trials: Is there a relationship? Protocol for a meta-epidemiological study.

ABSTRACT:

Introduction

It is fundamental that randomised controlled trials (RCTs) are properly conducted in order to reach well-supported conclusions. However, there is emerging evidence that RCTs are subject to biases which can overestimate or underestimate the true treatment effect, due to flaws in the study design characteristics of such trials. The extent to which this holds true in oral health RCTs, which have some unique design characteristics compared to RCTs in other health fields, is unclear. As such, we aim to examine the empirical evidence quantifying the extent of bias associated with methodological and non-methodological characteristics in oral health RCTs.

Methods and analysis

We plan to perform a meta-epidemiological study, where a sample size of 60 meta-analyses (MAs) including approximately 600 RCTs will be selected. The MAs will be randomly obtained from the Oral Health Database of Systematic Reviews using a random number table; and will be considered for inclusion if they include a minimum of five RCTs, and examine a therapeutic intervention related to one of the recognised dental specialties. RCTs identified in selected MAs will be subsequently included if their study design includes a comparison between an intervention group and a placebo group or another intervention group. Data will be extracted from selected trials included in MAs based on a number of methodological and non-methodological characteristics. Moreover, the risk of bias will be assessed using the Cochrane Risk of Bias tool. Effect size estimates and measures of variability for the main outcome will be extracted from each RCT included in selected MAs, and a two-level analysis will be conducted using a meta-meta-analytic approach with a random effects model to allow for intra-MA and inter-MA heterogeneity.

Ethics and dissemination

The intended audiences of the findings will include dental clinicians, oral health researchers, policymakers and graduate students. The aforementioned will be introduced to the findings through workshops, seminars, round table discussions and targeted individual meetings. Other opportunities for knowledge transfer will be pursued such as key dental conferences. Finally, the results will be published as a scientific report in a dental peer-reviewed journal.

SUBMITTER: Saltaji H

PROVIDER: S-EPMC3939646 | biostudies-literature | 2014 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Methodological characteristics and treatment effect sizes in oral health randomised controlled trials: Is there a relationship? Protocol for a meta-epidemiological study.

Saltaji Humam H Armijo-Olivo Susan S Cummings Greta G GG Amin Maryam M Flores-Mir Carlos C

BMJ open 20140225 2

<h4>Introduction</h4>It is fundamental that randomised controlled trials (RCTs) are properly conducted in order to reach well-supported conclusions. However, there is emerging evidence that RCTs are subject to biases which can overestimate or underestimate the true treatment effect, due to flaws in the study design characteristics of such trials. The extent to which this holds true in oral health RCTs, which have some unique design characteristics compared to RCTs in other health fields, is uncl ...[more]

PMID: 24568962

Similar Datasets

Project description:ObjectiveTo compare treatment effects from randomised trials conducted in more developed versus less developed countries.DesignMeta-epidemiological study.Data sourcesCochrane Database of Systematic Reviews (August 2012).Data extractionMeta-analyses with mortality outcomes including data from at least one randomised trial conducted in a less developed country and one in a more developed country. Relative risk estimates of more versus less developed countries were compared by calculating the relative relative risks for each topic and the summary relative relative risks across all topics. Similar analyses were performed for the primary binary outcome of each topic.Results139 meta-analyses with mortality outcomes were eligible. No nominally significant differences between more developed and less developed countries were found for 128 (92%) meta-analyses. However, differences were beyond chance in 11 (8%) cases, always showing more favourable treatment effects in trials from less developed countries. The summary relative relative risk was 1.12 (95% confidence interval 1.06 to 1.18; P<0.001; I(2)=0%), suggesting significantly more favourable mortality effects in trials from less developed countries. Results were similar for meta-analyses with nominally significant treatment effects for mortality (1.15), meta-analyses with recent trials (1.14), and when excluding trials from less developed countries that subsequently became more developed (1.12). For the primary binary outcomes (127 meta-analyses), 20 topics had differences in treatment effects beyond chance (more favourable in less developed countries in 15/20 cases).ConclusionsTrials from less developed countries in a few cases show significantly more favourable treatment effects than trials in more developed countries and, on average, treatment effects are more favourable in less developed countries. These discrepancies may reflect biases in reporting or study design as well as genuine differences in baseline risk or treatment implementation and should be considers when generalising evidence across different settings.

Project description:ObjectiveTo quantify and compare the treatment effect and risk of bias of trials reporting biomarkers or intermediate outcomes (surrogate outcomes) versus trials using final patient relevant primary outcomes.DesignMeta-epidemiological study.Data sourcesAll randomised clinical trials published in 2005 and 2006 in six high impact medical journals: Annals of Internal Medicine, BMJ, Journal of the American Medical Association, Lancet, New England Journal of Medicine, and PLoS Medicine.Study selectionTwo independent reviewers selected trials.Data extractionTrial characteristics, risk of bias, and outcomes were recorded according to a predefined form. Two reviewers independently checked data extraction. The ratio of odds ratios was used to quantify the degree of difference in treatment effects between the trials using surrogate outcomes and those using patient relevant outcomes, also adjusted for trial characteristics. A ratio of odds ratios >1.0 implies that trials with surrogate outcomes report larger intervention effects than trials with patient relevant outcomes.Results84 trials using surrogate outcomes and 101 using patient relevant outcomes were considered for analyses. Study characteristics of trials using surrogate outcomes and those using patient relevant outcomes were well balanced, except for median sample size (371 v 741) and single centre status (23% v 9%). Their risk of bias did not differ. Primary analysis showed trials reporting surrogate endpoints to have larger treatment effects (odds ratio 0.51, 95% confidence interval 0.42 to 0.60) than trials reporting patient relevant outcomes (0.76, 0.70 to 0.82), with an unadjusted ratio of odds ratios of 1.47 (1.07 to 2.01) and adjusted ratio of odds ratios of 1.46 (1.05 to 2.04). This result was consistent across sensitivity and secondary analyses.ConclusionsTrials reporting surrogate primary outcomes are more likely to report larger treatment effects than trials reporting final patient relevant primary outcomes. This finding was not explained by differences in the risk of bias or characteristics of the two groups of trials.

Project description:IntroductionAlthough interest in including non-randomised studies of interventions (NRSIs) in meta-analysis of randomised controlled trials (RCTs) is growing, estimates of effectiveness obtained from NRSIs are vulnerable to greater bias than RCTs. The objectives of this study are to: (1) explore how NRSIs can be integrated into a meta-analysis of RCTs; (2) assess concordance of the evidence from non-randomised and randomised trials and explore factors associated with agreement; and (3) investigate the impact on estimates of pooled bodies of evidence when NRSIs are included.Methods and analysisWe will conduct a systematic survey of 210 systematic reviews that include both RCTs and NRSIs, published from 2017 to 2022. We will randomly select reviews, stratified in a 1:1 ratio by Core vs non-Core clinical journals, as defined by the National Library of Medicine. Teams of paired reviewers will independently determine eligibility and abstract data using standardised, pilot-tested forms. The concordance of the evidence will be assessed by exploring agreement in the relative effect reported by NRSIs and RCT addressing the same clinical question, defined as similarity of the population, intervention/exposure, control and outcomes. We will conduct univariable and multivariable logistic regression analyses to examine the association of prespecified study characteristics with agreement in the estimates between NRSIs and RCTs. We will calculate the ratio of the relative effect estimate from NRSIs over that from RCTs, along with the corresponding 95% CI. We will use a bias-corrected meta-analysis model to investigate the influence on pooled estimates when NRSIs are included in the evidence synthesis.Ethics and disseminationEthics approval is not required. The findings of this study will be disseminated through peer-reviewed publications, conference presentations and condensed summaries for clinicians, health policymakers and guideline developers regarding the design, conduct, analysis, and interpretation of meta-analysis that integrate RCTs and NRSIs.

Project description:IntroductionThere has been increasing interest in pragmatic trials methodology. As a result, tools such as the Pragmatic-Explanatory Continuum Indicator Summary-2 (PRECIS-2) are being used prospectively to help researchers design randomised controlled trials (RCTs) within the pragmatic-explanatory continuum. There may be value in applying the PRECIS-2 tool retrospectively in a systematic review setting as it could provide important information about how to pool data based on the degree of pragmatism.ObjectivesTo investigate the role of pragmatism as a source of heterogeneity in systematic reviews by (1) identifying systematic reviews with meta-analyses of RCTs that have moderate to high heterogeneity, (2) applying PRECIS-2 to RCTs of systematic reviews, (3) evaluating the inter-rater reliability of PRECIS-2, (4) determining how much of this heterogeneity may be explained by pragmatism.MethodsA cross-sectional methodological review will be conducted on systematic reviews of RCTs published in the Cochrane Library from 1 January 2014 to 1 January 2017. Included systematic reviews will have a minimum of 10 RCTs in the meta-analysis of the primary outcome and moderate to substantial heterogeneity (I2≥50%). Of the eligible systematic reviews, a random selection of 10 will be included for quantitative evaluation. In each systematic review, RCTs will be scored using the PRECIS-2 tool, in duplicate. Agreement between raters will be measured using the intraclass correlation coefficient. Subgroup analyses and meta-regression will be used to evaluate how much variability in the primary outcome may be due to pragmatism.DisseminationThis review will be among the first to evaluate the PRECIS-2 tool in a systematic review setting. Results from this research will provide inter-rater reliability information about PRECIS-2 and may be used to provide methodological guidance when dealing with pragmatism in systematic reviews and subgroup considerations. On completion, this review will be submitted to a peer-reviewed journal for publication.

Project description:ObjectivesTo study the impact of blinding on estimated treatment effects, and their variation between trials; differentiating between blinding of patients, healthcare providers, and observers; detection bias and performance bias; and types of outcome (the MetaBLIND study).DesignMeta-epidemiological study.Data sourceCochrane Database of Systematic Reviews (2013-14).Eligibility criteria for selecting studiesMeta-analyses with both blinded and non-blinded trials on any topic.Review methodsBlinding status was retrieved from trial publications and authors, and results retrieved automatically from the Cochrane Database of Systematic Reviews. Bayesian hierarchical models estimated the average ratio of odds ratios (ROR), and estimated the increases in heterogeneity between trials, for non-blinded trials (or of unclear status) versus blinded trials. Secondary analyses adjusted for adequacy of concealment of allocation, attrition, and trial size, and explored the association between outcome subjectivity (high, moderate, low) and average bias. An ROR lower than 1 indicated exaggerated effect estimates in trials without blinding.ResultsThe study included 142 meta-analyses (1153 trials). The ROR for lack of blinding of patients was 0.91 (95% credible interval 0.61 to 1.34) in 18 meta-analyses with patient reported outcomes, and 0.98 (0.69 to 1.39) in 14 meta-analyses with outcomes reported by blinded observers. The ROR for lack of blinding of healthcare providers was 1.01 (0.84 to 1.19) in 29 meta-analyses with healthcare provider decision outcomes (eg, readmissions), and 0.97 (0.64 to 1.45) in 13 meta-analyses with outcomes reported by blinded patients or observers. The ROR for lack of blinding of observers was 1.01 (0.86 to 1.18) in 46 meta-analyses with subjective observer reported outcomes, with no clear impact of degree of subjectivity. Information was insufficient to determine whether lack of blinding was associated with increased heterogeneity between trials. The ROR for trials not reported as double blind versus those that were double blind was 1.02 (0.90 to 1.13) in 74 meta-analyses.ConclusionNo evidence was found for an average difference in estimated treatment effect between trials with and without blinded patients, healthcare providers, or outcome assessors. These results could reflect that blinding is less important than often believed or meta-epidemiological study limitations, such as residual confounding or imprecision. At this stage, replication of this study is suggested and blinding should remain a methodological safeguard in trials.

Project description:Backgroundn-of-1 trials are a type of crossover trial designed to optimise the evaluation of health technologies in individual patients. This trial design may be considered for the evaluation of health technologies in rare conditions where fewer patients are available to take part in research. This review describes the characteristics of randomised n-of-1 trials conducted over the span of 12 years, including how the n-of-1 design has been employed to study both rare and non-rare conditions.MethodsDatabases and clinical trials registries were searched for articles including "n-of-1" in the title between 2011 and 2023. The reference lists of reviews identified by the searches were searched for any additional eligible articles. Randomised n-of-1 trials were selected for inclusion and data were extracted on a range of design, population, and analysis characteristics. Descriptive statistics were produced for all variables.ResultsWe identified 74 studies meeting our eligibility criteria, 13 of which (17.6%) were conducted in rare conditions. They were conducted in a range of clinical areas with the most common being neurological conditions (n = 16, 21.6%). The median (Q1, Q3) number of participants randomised was 9 (4, 20) and 12 trials (16.2%) involved a single patient only. Forty-six (62.2%) trials evaluated pharmaceutical interventions and 49 (66.2%) trials were placebo controlled. Trials had a median (Q1, Q3) of six (4, 8) periods and 61 (82.4%) compared two health technologies. Fifty-seven (77.0%) trials incorporated blinding and 32 (43.2%) had a washout period. Forty-nine trials (66.2%) used patient-reported outcome measures (PROMs) to assess the primary outcome. Trials used a range of approaches to analysis and 48 (64.9%) combined data from multiple patients. The characteristics of the n-of-1 trials conducted in rare conditions were generally consistent with those in non-rare conditions.Conclusionsn-of-1 trials are still underused and the application of the n-of-1 design for the evaluation of health technologies for rare diseases has been particularly limited. We have summarised the characteristics of randomised n-of-1 trials in rare and non-rare conditions. We hope that it can inform researchers in the design of future n-of-1 studies. Further work is required to provide guidance on specific design considerations, implementation, and statistical analysis of these studies.Trial registrationNot applicable.

Dataset Information

Methodological characteristics and treatment effect sizes in oral health randomised controlled trials: Is there a relationship? Protocol for a meta-epidemiological study.

Introduction

Methods and analysis

Ethics and dissemination

Publications

Methodological characteristics and treatment effect sizes in oral health randomised controlled trials: Is there a relationship? Protocol for a meta-epidemiological study.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets