Is Measurement Noninvariance a Threat to Inferences Drawn from Randomized Control Trials? Evidence From Empirical and Simulation Studies.
Ontology highlight
ABSTRACT:
SUBMITTER: Soland J
PROVIDER: S-EPMC8361374 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectiveTo synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.DesignSystematic review of meta-epidemiological studies.MethodsWe retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.ResultsWe included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.ConclusionsCertain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.
| S-EPMC4939945 | biostudies-literature
Project description:Data from several thousand knockout mutations in yeast (Saccharomyces cerevisiae) were used to estimate the distribution of dominance coefficients. We propose a new unbiased likelihood approach to measuring dominance coefficients. On average, deleterious mutations are partially recessive, with a mean dominance coefficient ~0.2. Alleles with large homozygous effects are more likely to be more recessive than are alleles of weaker effect. Our approach allows us to quantify, for the first time, the substantial variance and skew in the distribution of dominance coefficients. This heterogeneity is so great that many population genetic processes analyses based on the mean dominance coefficient alone will be in substantial error. These results are applied to the debate about various mechanisms for the evolution of dominance, and we conclude that they are most consistent with models that depend on indirect selection on homeostatic gene expression or on the ability to perform well under periods of high demand for a protein.
| S-EPMC3030496 | biostudies-literature
Project description:BACKGROUND:Standard random-effects meta-analysis methods perform poorly when applied to few studies only. Such settings however are commonly encountered in practice. It is unclear, whether or to what extent small-sample-size behaviour can be improved by more sophisticated modeling. METHODS:We consider likelihood-based methods, the DerSimonian-Laird approach, Empirical Bayes, several adjustment methods and a fully Bayesian approach. Confidence intervals are based on a normal approximation, or on adjustments based on the Student-t-distribution. In addition, a linear mixed model and two generalized linear mixed models (GLMMs) assuming binomial or Poisson distributed numbers of events per study arm are considered for pairwise binary meta-analyses. We extract an empirical data set of 40 meta-analyses from recent reviews published by the German Institute for Quality and Efficiency in Health Care (IQWiG). Methods are then compared empirically as well as in a simulation study, based on few studies, imbalanced study sizes, and considering odds-ratio (OR) and risk ratio (RR) effect sizes. Coverage probabilities and interval widths for the combined effect estimate are evaluated to compare the different approaches. RESULTS:Empirically, a majority of the identified meta-analyses include only 2 studies. Variation of methods or effect measures affects the estimation results. In the simulation study, coverage probability is, in the presence of heterogeneity and few studies, mostly below the nominal level for all frequentist methods based on normal approximation, in particular when sizes in meta-analyses are not balanced, but improve when confidence intervals are adjusted. Bayesian methods result in better coverage than the frequentist methods with normal approximation in all scenarios, except for some cases of very large heterogeneity where the coverage is slightly lower. Credible intervals are empirically and in the simulation study wider than unadjusted confidence intervals, but considerably narrower than adjusted ones, with some exceptions when considering RRs and small numbers of patients per trial-arm. Confidence intervals based on the GLMMs are, in general, slightly narrower than those from other frequentist methods. Some methods turned out impractical due to frequent numerical problems. CONCLUSIONS:In the presence of between-study heterogeneity, especially with unbalanced study sizes, caution is needed in applying meta-analytical methods to few studies, as either coverage probabilities might be compromised, or intervals are inconclusively wide. Bayesian estimation with a sensibly chosen prior for between-trial heterogeneity may offer a promising compromise.
| S-EPMC6330405 | biostudies-literature
Project description:IntroductionFrailty is associated with reduced functional capacity, decreased resistance to stressors and is predictive of a range of adverse health outcomes, including dependency, hospitalisation and mortality. Early identification of frailty may prevent, reduce and postpone adverse health outcomes. However, there is a need for additional evidence to guide decision-making for the care of frail patients since frail persons are frequently excluded from studies, the differential impact of frailty is often not examined in clinical trials and few large-scale clinical trials examining frail cohorts have been conducted. Randomised control trials (RCTs) published to date have used a diverse range of definitions of frailty, as well as a variety of outcome measures. The objective of this systematic review is to comprehensively characterise the frail populations enrolled and the end points reported in frailty RCTs.Methods and analysisWe will identify all RCTs reporting on the outcome of interventions in adult (age ≥18 years) frail populations as defined by authors, in all settings of care. Databases will include MEDLINE, CINAHL, EMBASE, PsycInfo, Global Health, the Joanna Briggs database and Cochrane Library. Two reviewers will independently determine trial eligibility. For each included trial, we will conduct duplicate independent data extraction, inter-rater reliability, risk of bias assessment and evaluation of the quality of the evidence using the Grading of Recommendations, Assessment, Development and Evaluations approach.Ethics and disseminationThis systematic review will comprehensively identify RCTs including frail patients to identify how frailty is measured and which outcomes are reported. The results of this systematic review may inform clinicians caring for persons with frailty, facilitate conduct of future RCTs and inform future efforts to develop common data elements and core outcomes for frailty studies. Our findings will be disseminated through conference presentation and publication in peer-reviewed journals.Prospero registration numberCRD42017065233.
| S-EPMC5988125 | biostudies-literature
Project description:Previous work using color photographic scenes has shown that human observers are keenly sensitive to different types of threatening and negative stimuli and reliably classify them by the presence, and spatial and temporal directions of threat. To test whether such distinctions can be extracted from impoverished visual information, we used 500 line drawings made by hand-tracing the original set of photographic scenes. Sixty participants rated the scenes on spatial and temporal dimensions of threat. Based on these ratings, trend analysis revealed five scene categories that were comparable to those identified for the matching color photographic scenes. Another 61 participants were randomly assigned to rate the valence or arousal evoked by the line drawings. The line drawings perceived to be the most negative were also perceived to be the most arousing, replicating the finding for color photographic scenes. We demonstrate here that humans are very sensitive to the spatial and temporal directions of threat even when they must extract this information from simple line drawings, and rate the line drawings very similarly to matched color photographs. The set of 500 hand-traced line-drawing scenes has been made freely available to the research community: http://www.kveragalab.org/threat.html.
| S-EPMC5950935 | biostudies-literature
Project description:Recent studies have proposed using well-defined relationships between network productivity and accumulation-otherwise known as Network or Macroscopic Fundamental Diagrams (network MFDs)-to model the dynamics of large-scale urban traffic networks. Network MFDs have been used to develop a variety of network-wide traffic control policies to improve a network's operational efficiency. However, the relationship between a network's MFD and its safety performance has not been well explored. This study proposes the existence of a Macroscopic Safety Diagram (MSD) that relates safety performance (e.g., likelihood of a crash occurring or number of vehicle conflicts observed) with the current network state (i.e., average density) in an urban traffic network. We theoretically posit a relationship between a network's MSD and its MFD based on the average maneuver envelop of vehicles traveling within the network. Based on this model, we show that the density associated with maximum crash propensity is always expected to be larger than the density associated with maximum network productivity. This finding suggests that congested states are not only inefficient, but they might also be associated with more crashes, which can be both more unsafe and lead to decreased network reliability. These theoretical results are validated using surrogate safety assessment metrics in microsimulation and limited field empirical data from a small arterial network in Riyadh, Kingdom of Saudi Arabia. The existence of such MSDs can be used to develop more comprehensive network-wide control policies that can ensure both safe, efficient and reliable network operations.
| S-EPMC6080770 | biostudies-literature
Project description:BackgroundHuman mitochondrial DNA (mtDNA) has been extensively used in population and evolutionary genetics studies. Thus, a valid estimate of human mtDNA evolutionary rate is important in many research fields. The small number of estimations performed for the coding region of the molecule, showed important differences between phylogenetic and empirical approaches. We analyzed a portion of the coding region of mtDNA (tRNALeu, ND1 and tRNAIle genes), using individuals belonging to extended families from the Azores Islands (Portugal) with the main aim of providing empirical estimations of the mutation rate of the coding region of mtDNA under different assumptions, and hence to better understand the mtDNA evolutionary process.ResultsHeteroplasmy was detected in 6.5% (3/46) of the families analyzed. In all of the families the presence of mtDNA heteroplasmy resulted from three new point mutations, and no cases of insertions or deletions were identified. Major differences were found in the proportion and type of heteroplasmy found in the genes studied when compared to those obtained in a previous report for the D-loop. Our empirical estimation of mtDNA coding region mutation rate, calculated taking into account the sex of individuals carrying new mutations, the probability of intra-individual fixation of mutations present in heteroplasmy and, to the possible extent, the effect of selection, is similar to that obtained using phylogenetic approaches.ConclusionBased on our results, the discrepancy previously reported between the human mtDNA coding region mutation rates observed along evolutionary timescales and estimations obtained using family pedigrees can be resolved when correcting for the previously cited factors.
| S-EPMC2438339 | biostudies-literature
Project description:Randomized controlled trials (RCTs) and cohort studies are the most common study design types used to assess treatment effects of medical interventions. We aimed to hypothetically pool bodies of evidence (BoE) from RCTs with matched BoE from cohort studies included in the same systematic review. BoE derived from systematic reviews of RCTs and cohort studies published in the 13 medical journals with the highest impact factor were considered. We re-analyzed effect estimates of the included systematic reviews by pooling BoE from RCTs with BoE from cohort studies using random and common effects models. We evaluated statistical heterogeneity, 95% prediction intervals, weight of BoE from RCTs to the pooled estimate, and whether integration of BoE from cohort studies modified the conclusion from BoE of RCTs. Overall, 118 BoE-pairs based on 653 RCTs and 804 cohort studies were pooled. By pooling BoE from RCTs and cohort studies with a random effects model, for 61 (51.7%) out of 118 BoE-pairs, the 95% confidence interval (CI) excludes no effect. By pooling BoE from RCTs and cohort studies, the median I2 was 48%, and the median contributed percentage weight of RCTs to the pooled estimates was 40%. The direction of effect between BoE from RCTs and pooled effect estimates was mainly concordant (79.7%). The integration of BoE from cohort studies modified the conclusion (by examining the 95% CI) from BoE of RCTs in 32 (27%) of the 118 BoE-pairs, but the direction of effect was mainly concordant (88%). Our findings provide insights for the potential impact of pooling both BoE in systematic reviews. In medical research, it is often important to rely on both evidence of RCTs and cohort studies to get a whole picture of an investigated intervention-disease association. A decision for or against pooling different study designs should also always take into account, for example, PI/ECO similarity, risk of bias, coherence of effect estimates, and also the trustworthiness of the evidence. Overall, there is a need for more research on the influence of those issues on potential pooling.
| S-EPMC9590141 | biostudies-literature
Project description:This paper investigates how inconsistency (as measured by the I2 statistic) among studies in a meta-analysis may differ, according to the type of outcome data and effect measure. We used hierarchical models to analyse data from 3873 binary, 5132 continuous and 880 mixed outcome meta-analyses within the Cochrane Database of Systematic Reviews. Predictive distributions for inconsistency expected in future meta-analyses were obtained, which can inform priors for between-study variance. Inconsistency estimates were highest on average for binary outcome meta-analyses of risk differences and continuous outcome meta-analyses. For a planned binary outcome meta-analysis in a general research setting, the predictive distribution for inconsistency among log odds ratios had median 22% and 95% CI: 12% to 39%. For a continuous outcome meta-analysis, the predictive distribution for inconsistency among standardized mean differences had median 40% and 95% CI: 15% to 73%. Levels of inconsistency were similar for binary data measured by log odds ratios and log relative risks. Fitted distributions for inconsistency expected in continuous outcome meta-analyses using mean differences were almost identical to those using standardized mean differences. The empirical evidence on inconsistency gives guidance on which outcome measures are most likely to be consistent in particular circumstances and facilitates Bayesian meta-analysis with an informative prior for heterogeneity. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd. © 2015 The Authors. Research Synthesis Methods published by John Wiley & Sons, Ltd.
| S-EPMC5217093 | biostudies-literature
Project description:Urban transportation is an important determinant of health and environmental outcomes, and therefore essential to achieving the United Nation's Sustainable Development Goals. To better understand the health impacts of transportation initiatives, we conducted a systematic review of longitudinal health evaluations involving: a) bus rapid transit (BRT); b) bicycle lanes; c) Open Streets programs; and d) aerial trams/cable cars. We also synthesized systems-based simulation studies of the health-related consequences of walking, bicycling, aerial tram, bus and BRT use. Two reviewers screened 3302 unique titles and abstracts identified through a systematic search of MEDLINE (Ovid), Scopus, TRID and LILACS databases. We included 39 studies: 29 longitudinal evaluations and 10 simulation studies. Five studies focused on low- and middle-income contexts. Of the 29 evaluation studies, 19 focused on single component bicycle lane interventions; the rest evaluated multi-component interventions involving: bicycle lanes (n = 5), aerial trams (n = 1), and combined bicycle lane/BRT systems (n = 4). Bicycle lanes and BRT systems appeared effective at increasing bicycle and BRT mode share, active transport duration, and number of trips using these modes. Of the 10 simulation studies, there were 9 agent-based models and one system dynamics model. Five studies focused on bus/BRT expansions and incentives, three on interventions for active travel, and the rest investigated combinations of public transport and active travel policies. Synergistic effects were observed when multiple policies were implemented, with several studies showing that sizable interventions are required to significantly shift travel mode choices. Our review indicates that bicycle lanes and BRT systems represent promising initiatives for promoting population health. There is also evidence to suggest that synergistic effects might be achieved through the combined implementation of multiple transportation policies. However, more rigorous evaluation and simulation studies focusing on low- and middle-income countries, aerial trams and Open Streets programs, and a more diverse set of health and health equity outcomes is required.
| S-EPMC7343239 | biostudies-literature