Empirical evidence about inconsistency among studies in a pair-wise meta-analysis.
Ontology highlight
ABSTRACT:
SUBMITTER: Rhodes KM
PROVIDER: S-EPMC5217093 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Network meta-analysis (NMA) is a popular tool to synthesize direct and indirect evidence for simultaneously comparing multiple treatments, while evidence inconsistency greatly threatens its validity. One may use the inconsistency degrees of freedom (ICDF) to assess the potential that an NMA might suffer from inconsistency. Multi-arm studies provide intrinsically consistent evidence and complicate the ICDF's calculation; they commonly appear in NMAs. The existing ICDF measure may not feasibly handle multi-arm studies. Motivated from the effective numbers of parameters of Bayesian hierarchical models, we propose new ICDF measures in generic NMAs that may contain multi-arm studies. Under the fixed- or random-effects setting, the new ICDF measure is the difference between the effective numbers of parameters of the consistency and inconsistency NMA models. We used artificial NMAs created based on an illustrative example and 39 empirical NMAs to evaluate the performance of the existing and new measures. In NMAs with two-arm studies only, the proposed ICDF measure under the fixed-effects setting was nearly the same with the existing measure. Among the empirical NMAs, 27 (69%) contained at least one multi-arm study. The existing measure was not applicable to them, while the proposed measures led to interpretable ICDFs in all NMAs.
| S-EPMC8122016 | biostudies-literature
Project description:Background and study aimsIn recent years, cold snare polypectomy (CSP) has been increasingly used for small polyps (<10 mm) instead of hot snare polypectomy (HSP). However, evidence-based research regarding the effectiveness and safety of CSP and HSP are still lacking. Additionally, for 4-10 -mm non-pedunculated polyps, the polyp removal method is still controversial. Therefore, it is clinically significant to conduct pair-wise and network meta-analyses to assess such resection methods.MethodsWe searched PubMed, Embase, and the Cochrane library for randomized controlled trials (RCTs). Only studies that involved the resection of polyps <10 mm were included. Outcomes included the complete resection rate, polyp retrieval rate, procedure-related complications, and procedure times.ResultsOverall, 23 RCTs (5,352 patients) were identified. In meta-analysis compared CSP versus HSP for polyps <10 mm, CSP showed lower complete resection rate than HSP although with no statistically significant difference [odds ratio (OR): 0.77, 95% confidence interval (CI): 0.56-1.06]. CSP showed a lower risk of major post-polypectomy complications compared to HSP (OR: 0.28, 95% CI: 0.11-0.73). In the network meta-analysis for 4-10 mm non-pedunculated polyps, HSP, and endoscopic mucosal resection (EMR) showed a higher complete resection rate than CSP (OR: 2.7, 95% CI: 1.3-9.2 vs. OR: 2.6, 95% CI: 1.0-10) but a significantly longer time than CSP (WMD: 16.55 s, 95% CI [7.48 s, 25.25 s], p < 0.001), (WMD: 48.00 s, 95% CI [16.54 s, 79.46 s], p = 0.003). Underwater CSP ranked third for complete resection with no complications.ConclusionFor <10 mm polyps, CSP is safer than HSP, especially for patients taking antithrombotic drugs. For 4-10 mm non-pedunculated polyps, HSP, and EMR have higher complete resection rates than CSP.Systematic review registrationhttps://www.crd.york.ac.uk/PROSPERO/, identifier CRD42022315575.
| S-EPMC10117900 | biostudies-literature
Project description:BACKGROUND:Standard random-effects meta-analysis methods perform poorly when applied to few studies only. Such settings however are commonly encountered in practice. It is unclear, whether or to what extent small-sample-size behaviour can be improved by more sophisticated modeling. METHODS:We consider likelihood-based methods, the DerSimonian-Laird approach, Empirical Bayes, several adjustment methods and a fully Bayesian approach. Confidence intervals are based on a normal approximation, or on adjustments based on the Student-t-distribution. In addition, a linear mixed model and two generalized linear mixed models (GLMMs) assuming binomial or Poisson distributed numbers of events per study arm are considered for pairwise binary meta-analyses. We extract an empirical data set of 40 meta-analyses from recent reviews published by the German Institute for Quality and Efficiency in Health Care (IQWiG). Methods are then compared empirically as well as in a simulation study, based on few studies, imbalanced study sizes, and considering odds-ratio (OR) and risk ratio (RR) effect sizes. Coverage probabilities and interval widths for the combined effect estimate are evaluated to compare the different approaches. RESULTS:Empirically, a majority of the identified meta-analyses include only 2 studies. Variation of methods or effect measures affects the estimation results. In the simulation study, coverage probability is, in the presence of heterogeneity and few studies, mostly below the nominal level for all frequentist methods based on normal approximation, in particular when sizes in meta-analyses are not balanced, but improve when confidence intervals are adjusted. Bayesian methods result in better coverage than the frequentist methods with normal approximation in all scenarios, except for some cases of very large heterogeneity where the coverage is slightly lower. Credible intervals are empirically and in the simulation study wider than unadjusted confidence intervals, but considerably narrower than adjusted ones, with some exceptions when considering RRs and small numbers of patients per trial-arm. Confidence intervals based on the GLMMs are, in general, slightly narrower than those from other frequentist methods. Some methods turned out impractical due to frequent numerical problems. CONCLUSIONS:In the presence of between-study heterogeneity, especially with unbalanced study sizes, caution is needed in applying meta-analytical methods to few studies, as either coverage probabilities might be compromised, or intervals are inconclusively wide. Bayesian estimation with a sensibly chosen prior for between-trial heterogeneity may offer a promising compromise.
| S-EPMC6330405 | biostudies-literature
Project description:ObjectivesSeveral methods are commonly used for meta-analyses of diagnostic studies, such as the bivariate linear mixed model (LMM). It estimates the overall sensitivity, specificity, their correlation, diagnostic OR (DOR) and the area under the curve (AUC) of the summary receiver operating characteristic (ROC) estimates. Nevertheless, the bivariate LMM makes potentially unrealistic assumptions (ie, normality of within-study estimates), which could be avoided by the bivariate generalised linear mixed model (GLMM). This article aims at investigating the real-world performance of the bivariate LMM and GLMM using meta-analyses of diagnostic studies from the Cochrane Library.MethodsWe compared the bivariate LMM and GLMM using the relative differences in the overall sensitivity and specificity, their 95% CI widths, between-study variances, and the correlation between the (logit) sensitivity and specificity. We also explored their relationships with the number of studies, number of subjects, overall sensitivity and overall specificity.ResultsAmong the extracted 1379 meta-analyses, point estimates of overall sensitivities and specificities by the bivariate LMM and GLMM were generally similar, but their CI widths could be noticeably different. The bivariate GLMM generally produced narrower CIs than the bivariate LMM when meta-analyses contained 2-5 studies. For meta-analyses with <100 subjects or the overall sensitivities or specificities close to 0% or 100%, the bivariate LMM could produce substantially different AUCs, DORs and DOR CI widths from the bivariate GLMM.ConclusionsThe variation of estimates calls into question the appropriateness of the normality assumption within individual studies required by the bivariate LMM. In cases of notable differences presented in these methods' results, the bivariate GLMM may be preferred.
| S-EPMC9086644 | biostudies-literature
Project description:BackgroundCircular RNAs (circRNAs), proven as single-stranded closed RNA molecules, have been implicated in the onset and development of multiple cancers. This study aimed to summarize existing evidences regarding the clinicopathologic, diagnostic, and prognostic significances of circRNAs in gastric cancer (GC).MethodsEligible studies were identified using online databases. The quality of the included studies was judged, and patients' clinical characteristics, diagnostic data, and overall survival (OS) were extracted from the electronic medical record. Fisher's method was adopted to determine P values for clinicopathologic features. The diagnostic and prognostic data from all included studies were merged.ResultsThirty eligible studies were comprised of 2687 GC patients were enrolled in the meta-analyses. Altered expressions of circRNAs in GC tissues were significantly associated with worse clinicopathologic features. Abnormally expressed circRNAs yielded a pooled sensitivity of 0.76 (95% CI: 0.69-0.81) and a specificity of 0.77 (95% CI: 0.70-0.83) in distinguishing GC from noncancerous controls, which corresponded to an area under the curve (AUC) of 0.83. The survival analysis showed that the oncogenic circRNA signature could be an independent risk factor of OS (HR = 2.11, 95% CI: 1.60-2.78, P = .000). Patients with down-regulated circRNAs (tumor suppressor genes) presented a significantly shorter OS time than those with high-level circRNAs (HR = 0.33, 95% CI: 0.27-0.42, P = .000). Stratified analyses based on sample type, control source, circRNA expression status, and cutoff setting also produced robust results.ConclusionsCircRNAs may play an important role as potential diagnostic and prognostic biomarkers of GC.
| S-EPMC7439415 | biostudies-literature
Project description:ObjectiveTo synthesise evidence on the average bias and heterogeneity associated with reported methodological features of randomized trials.DesignSystematic review of meta-epidemiological studies.MethodsWe retrieved eligible studies included in a recent AHRQ-EPC review on this topic (latest search September 2012), and searched Ovid MEDLINE and Ovid EMBASE for studies indexed from Jan 2012-May 2015. Data were extracted by one author and verified by another. We combined estimates of average bias (e.g. ratio of odds ratios (ROR) or difference in standardised mean differences (dSMD)) in meta-analyses using the random-effects model. Analyses were stratified by type of outcome ("mortality" versus "other objective" versus "subjective"). Direction of effect was standardised so that ROR < 1 and dSMD < 0 denotes a larger intervention effect estimate in trials with an inadequate or unclear (versus adequate) characteristic.ResultsWe included 24 studies. The available evidence suggests that intervention effect estimates may be exaggerated in trials with inadequate/unclear (versus adequate) sequence generation (ROR 0.93, 95% CI 0.86 to 0.99; 7 studies) and allocation concealment (ROR 0.90, 95% CI 0.84 to 0.97; 7 studies). For these characteristics, the average bias appeared to be larger in trials of subjective outcomes compared with other objective outcomes. Also, intervention effects for subjective outcomes appear to be exaggerated in trials with lack of/unclear blinding of participants (versus blinding) (dSMD -0.37, 95% CI -0.77 to 0.04; 2 studies), lack of/unclear blinding of outcome assessors (ROR 0.64, 95% CI 0.43 to 0.96; 1 study) and lack of/unclear double blinding (ROR 0.77, 95% CI 0.61 to 0.93; 1 study). The influence of other characteristics (e.g. unblinded trial personnel, attrition) is unclear.ConclusionsCertain characteristics of randomized trials may exaggerate intervention effect estimates. The average bias appears to be greatest in trials of subjective outcomes. More research on several characteristics, particularly attrition and selective reporting, is needed.
| S-EPMC4939945 | biostudies-literature
Project description:Informative gene selection can have important implications for the improvement of cancer diagnosis and the identification of new drug targets. Individual-gene-ranking methods ignore interactions between genes. Furthermore, popular pair-wise gene evaluation methods, e.g. TSP and TSG, are helpless for discovering pair-wise interactions. Several efforts to discover pair-wise synergy have been made based on the information approach, such as EMBP and FeatKNN. However, the methods which are employed to estimate mutual information, e.g. binarization, histogram-based and KNN estimators, depend on known data or domain characteristics. Recently, Reshef et al. proposed a novel maximal information coefficient (MIC) measure to capture a wide range of associations between two variables that has the property of generality. An extension from MIC(X; Y) to MIC(X1; X2; Y) is therefore desired. We developed an approximation algorithm for estimating MIC(X1; X2; Y) where Y is a discrete variable. MIC(X1; X2; Y) is employed to detect pair-wise synergy in simulation and cancer microarray data. The results indicate that MIC(X1; X2; Y) also has the property of generality. It can discover synergic genes that are undetectable by reference feature selection methods such as MIC(X; Y) and TSG. Synergic genes can distinguish different phenotypes. Finally, the biological relevance of these synergic genes is validated with GO annotation and OUgene database.
| S-EPMC4965793 | biostudies-literature
Project description:Two large-scale pharmacogenomic studies were published recently in this journal. Genomic data are well correlated between studies; however, the measured drug response data are highly discordant. Although the source of inconsistencies remains uncertain, it has potential implications for using these outcome measures to assess gene-drug associations or select potential anticancer drugs on the basis of their reported results.
| S-EPMC4237165 | biostudies-literature
Project description:In 2013, we published a comparative analysis of mutation and gene expression profiles and drug sensitivity measurements for 15 drugs characterized in the 471 cancer cell lines screened in the Genomics of Drug Sensitivity in Cancer (GDSC) and Cancer Cell Line Encyclopedia (CCLE). While we found good concordance in gene expression profiles, there was substantial inconsistency in the drug responses reported by the GDSC and CCLE projects. We received extensive feedback on the comparisons that we performed. This feedback, along with the release of new data, prompted us to revisit our initial analysis. We present a new analysis using these expanded data, where we address the most significant suggestions for improvements on our published analysis - that targeted therapies and broad cytotoxic drugs should have been treated differently in assessing consistency, that consistency of both molecular profiles and drug sensitivity measurements should be compared across cell lines, and that the software analysis tools provided should have been easier to run, particularly as the GDSC and CCLE released additional data. Our re-analysis supports our previous finding that gene expression data are significantly more consistent than drug sensitivity measurements. Using new statistics to assess data consistency allowed identification of two broad effect drugs and three targeted drugs with moderate to good consistency in drug sensitivity data between GDSC and CCLE. For three other targeted drugs, there were not enough sensitive cell lines to assess the consistency of the pharmacological profiles. We found evidence of inconsistencies in pharmacological phenotypes for the remaining eight drugs. Overall, our findings suggest that the drug sensitivity data in GDSC and CCLE continue to present challenges for robust biomarker discovery. This re-analysis provides additional support for the argument that experimental standardization and validation of pharmacogenomic response will be necessary to advance the broad use of large pharmacogenomic screens.
| S-EPMC5580432 | biostudies-literature
Project description:Results from genome-wide association studies of complex traits account for only a modest proportion of the trait variance predicted to be due to genetics. We hypothesize that joint analysis of polymorphisms may account for more variance. We evaluated this hypothesis on a case-control smoking phenotype by examining pairs of nicotinic receptor single-nucleotide polymorphisms (SNPs) using the Restricted Partition Method (RPM) on data from the Collaborative Genetic Study of Nicotine Dependence (COGEND). We found evidence of joint effects that increase explained variance. Four signals identified in COGEND were testable in independent American Cancer Society (ACS) data, and three of the four signals replicated. Our results highlight two important lessons: joint effects that increase the explained variance are not limited to loci displaying substantial main effects, and joint effects need not display a significant interaction term in a logistic regression model. These results suggest that the joint analyses of variants may indeed account for part of the genetic variance left unexplained by single SNP analyses. Methodologies that limit analyses of joint effects to variants that demonstrate association in single SNP analyses, or require a significant interaction term, will likely miss important joint effects.
| S-EPMC3030551 | biostudies-literature