Random-effects meta-analysis for systematic reviews of phase I clinical trials: Rare events and missing data.
Ontology highlight
ABSTRACT:
SUBMITTER: Kim MO
PROVIDER: S-EPMC5149121 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:ObjectiveTo assess the risk of bias associated with missing outcome data in systematic reviews.DesignImputation study.SettingSystematic reviews.Population100 systematic reviews that included a group level meta-analysis with a statistically significant effect on a patient important dichotomous efficacy outcome.Main outcome measuresMedian percentage change in the relative effect estimate when applying each of the following assumption (four commonly discussed but implausible assumptions (best case scenario, none had the event, all had the event, and worst case scenario) and four plausible assumptions for missing data based on the informative missingness odds ratio (IMOR) approach (IMOR 1.5 (least stringent), IMOR 2, IMOR 3, IMOR 5 (most stringent)); percentage of meta-analyses that crossed the threshold of the null effect for each method; and percentage of meta-analyses that qualitatively changed direction of effect for each method. Sensitivity analyses based on the eight different methods of handling missing data were conducted.Results100 systematic reviews with 653 randomised controlled trials were included. When applying the implausible but commonly discussed assumptions, the median change in the relative effect estimate varied from 0% to 30.4%. The percentage of meta-analyses crossing the threshold of the null effect varied from 1% (best case scenario) to 60% (worst case scenario), and 26% changed direction with the worst case scenario. When applying the plausible assumptions, the median percentage change in relative effect estimate varied from 1.4% to 7.0%. The percentage of meta-analyses crossing the threshold of the null effect varied from 6% (IMOR 1.5) to 22% (IMOR 5) of meta-analyses, and 2% changed direction with the most stringent (IMOR 5).ConclusionEven when applying plausible assumptions to the outcomes of participants with definite missing data, the average change in pooled relative effect estimate is substantive, and almost a quarter (22%) of meta-analyses crossed the threshold of the null effect. Systematic review authors should present the potential impact of missing outcome data on their effect estimates and use this to inform their overall GRADE (grading of recommendations assessment, development, and evaluation) ratings of risk of bias and their interpretation of the results.
| S-EPMC7448113 | biostudies-literature
Project description:Motivated by the self-thinning meta-data, a random-effects meta-analysis model with unknown precision parameters is proposed with a truncated Poisson regression model for missing sample sizes. The random effects are assumed to follow a heavy-tailed distribution to accommodate outlying aggregate values in the response variable. The logarithm of the pseudo-marginal likelihood (LPML) is used for model comparison. In addition, in order to determine which self-thinning law is more supported by the meta-data, a measure called "Plausibility Index (PI)" is developed. A simulation study is conducted to examine empirical performance of the proposed methodology. Finally, the proposed model and the PI measure are applied to analyze a self-thinning meta-data set in details.
| S-EPMC8315582 | biostudies-literature
Project description:Multi-domain data are widely leveraged in vision applications taking advantage of complementary information from different modalities, e.g., brain tumor segmentation from multi-parametric magnetic resonance imaging (MRI). However, due to possible data corruption and different imaging protocols, the availability of images for each domain could vary amongst multiple data sources in practice, which makes it challenging to build a universal model with a varied set of input data. To tackle this problem, we propose a general approach to complete the random missing domain(s) data in real applications. Specifically, we develop a novel multi-domain image completion method that utilizes a generative adversarial network (GAN) with a representational disentanglement scheme to extract shared content encoding and separate style encoding across multiple domains. We further illustrate that the learned representation in multi-domain image completion could be leveraged for high-level tasks, e.g., segmentation, by introducing a unified framework consisting of image completion and segmentation with a shared content encoder. The experiments demonstrate consistent performance improvement on three datasets for brain tumor segmentation, prostate segmentation, and facial expression image completion respectively.
| S-EPMC8136445 | biostudies-literature
Project description:BackgroundThe methodological quality of animal studies is an important factor hampering the translation of results from animal studies to a clinical setting. Systematic reviews of animal studies may provide a suitable method to assess and thereby improve their methodological quality.ObjectivesThe aims of this study were: 1) to evaluate the risk of bias assessment in animal-based systematic reviews, and 2) to study the internal validity of the primary animal studies included in these systematic reviews.Data sourcesWe systematically searched Pubmed and Embase for SRs of preclinical animal studies published between 2005 and 2012.ResultsA total of 91 systematic reviews met our inclusion criteria. The risk of bias was assessed in 48 (52.7%) of these 91 systematic reviews. Thirty-three (36.3%) SRs provided sufficient information to evaluate the internal validity of the included studies. Of the evaluated primary studies, 24.6% was randomized, 14.6% reported blinding of the investigator/caretaker, 23.9% blinded the outcome assessment, and 23.1% reported drop-outs.ConclusionsTo improve the translation of animal data to clinical practice, systematic reviews of animal studies are worthwhile, but the internal validity of primary animal studies needs to be improved. Furthermore, risk of bias should be assessed by systematic reviews of animal studies to provide insight into the reliability of the available evidence.
| S-EPMC3966727 | biostudies-literature
Project description:Background/aimsRegulatory approval of a drug or device involves an assessment of not only the benefits but also the risks of adverse events associated with the therapeutic agent. Although randomized controlled trials (RCTs) are the gold standard for evaluating effectiveness, the number of treated patients in a single RCT may not be enough to detect a rare but serious side effect of the treatment. Meta-analysis plays an important role in the evaluation of the safety of medical products and has advantage over analyzing a single RCT when estimating the rate of adverse events.MethodsIn this article, we compare 15 widely used meta-analysis models under both Bayesian and frequentist frameworks when outcomes are extremely infrequent or rare. We present extensive simulation study results and then apply these methods to a real meta-analysis that considers RCTs investigating the effect of rosiglitazone on the risks of myocardial infarction and of death from cardiovascular causes.ResultsOur simulation studies suggest that the beta hyperprior method modeling treatment group-specific parameters and accounting for heterogeneity performs the best. Most models ignoring between-study heterogeneity give poor coverage probability when such heterogeneity exists. In the data analysis, different methods provide a wide range of log odds ratio estimates between rosiglitazone and control treatments with a mixed conclusion on their statistical significance based on 95% confidence (or credible) intervals.ConclusionIn the rare event setting, treatment effect estimates obtained from traditional meta-analytic methods may be biased and provide poor coverage probability. This trend worsens when the data have large between-study heterogeneity. In general, we recommend methods that first estimate the summaries of treatment-specific risks across studies and then relative treatment effects based on the summaries when appropriate. Furthermore, we recommend fitting various methods, comparing the results and model performance, and investigating any significant discrepancies among them.
| S-EPMC8041270 | biostudies-literature
Project description:BACKGROUND:Well designed and conducted systematic reviews are essential to clinical practice. Surgical intervention is more complex than medical intervention when considering special items related to procedures. There has been no cross-sectional study of the reporting quality of systematic reviews of surgical randomized trials focused on special items relating to surgical interventions. METHODS:A cross-sectional survey of systematic reviews of surgical randomized trials published in 2007 and 2017 was undertaken via a PubMed search. Quality of reporting was assessed by the PRISMA checklist, with intervention details containing 27 items. Univariable and multivariable linear regression was used to explore factors in the checklist as indicators of reporting quality. RESULTS:A total of 204 systematic reviews were identified. The median score for the PRISMA checklist was 22 (i.q.r. 20-24), and systematic reviews published in 2017 had a significantly higher median score than those from 2007 (22 (i.q.r. 21-24) versus 20 (17-22); P < 0·001). Among the 27 items, 15 were reported adequately and three were reported poorly (in less than 50 per cent of reports). The proportion of other items reported ranged from 54·4 to 77·9 per cent. In multivariable analysis, systematic reviews published in 2017 (coefficient 0·59, 95 per cent c.i. 0·50 to 0·69) and Cochrane reviews (coefficient 0·67, 0·55 to 0·81) were associated with better reporting. CONCLUSION:The quality of reporting of systematic reviews of surgical randomized trials has improved in the past 10 years. Some information relating to specific surgical interventions is, however, still reported poorly.
| S-EPMC7260405 | biostudies-literature
Project description:IntroductionFragility index (FI) refers to the smallest value that change statistical significance of study results. Meta-analyses of Cochrane systematic reviews are considered as the best evidence for stake holders because they enable effect size estimation that cannot be derived by individual studies, particularly in the field of complementary and integrative medicine (CIM). Thus, this study aimed to evaluate robustness of meta-analysis in Cochrane systematic reviews of acupuncture, one of the most used CIM treatment, using FI of meta-analysis.MethodsMeta-analyses of acupuncture Cochrane systematic reviews with binary benefit outcome measures were searched in PubMed, Embase, and CENTRAL and subject to analysis. Randomized clinical trials (RCTs) or quasi RCTs adopted penetrating needles as treatment and compared with controls such as sham acupuncture, usual care, and active control were included. FI of meta-analyses was calculated in web (https://clinicalepidemio.fr/fragility_ma/); and fragility quotient (FQ) was calculated by dividing FI by total sample size of meta-analysis.ResultsOf 248 retrieved studies, 12 Cochrane systematic reviews with 48 meta-analyses were analyzed. The median FI for statistically significant and non-significant meta-analyses was 7 [interquartile range: 3 - 12] and 6 [interquartile range: 3 - 10]. FQ was similar irrespective of statistical significance. Among controls, meta-analyses adopting sham acupuncture displayed the widest range of FI and FQ.ConclusionRobustness of meta-analyses in Cochrane systematic reviews for acupuncture was similar irrespective of statistical significance. Impact of control intervention cannot be determined in this study. As FI enables intuitive interpretation, applications for CIM field can be useful.
| S-EPMC9634365 | biostudies-literature
Project description:There are two possible ways to conceptualize the term "insomnia": insomnia disorder and insomnia symptoms, which are often poorly reported. The purpose of this study was to examine the proportion of randomized controlled trials (RCTs) and systematic reviews (SRs) that mention insomnia in their abstracts and cannot distinguish between insomnia disorder and insomnia symptoms from the abstract. We included RCT and SR articles that included the word "insomnia" in the methods or results sections of their structured abstracts, published after 2010. We searched PubMed using English language restrictions on 10 March 2022. From 1580 PubMed articles, we obtained 100 random samples each for eligible RCTs and SRs. The unclear insomnia concept accounted for 88% of the RCT abstracts and 94% of the SR abstracts. Among the RCT and SR abstracts with unclearness, the concept of insomnia was unclear in 27% of RCTs and 57% of SRs after investigating the full text. The concept of insomnia has been unclear in many RCTs and SRs abstracts. The authors of RCTs and SRs are recommended to state "insomnia disorder" or "insomnia symptoms" in the methods and results sections of their abstracts.
| S-EPMC9566752 | biostudies-literature
Project description:The development of coherent missing data models to account for nonmonotone missing at random (MAR) data by inverse probability weighting (IPW) remains to date largely unresolved. As a consequence, IPW has essentially been restricted for use only in monotone missing data settings. We propose a class of models for nonmonotone missing data mechanisms that spans the MAR model, while allowing the underlying full data law to remain unrestricted. For parametric specifications within the proposed class, we introduce an unconstrained maximum likelihood estimator for estimating the missing data probabilities which can be easily implemented using existing software. To circumvent potential convergence issues with this procedure, we also introduce a Bayesian constrained approach to estimate the missing data process which is guaranteed to yield inferences that respect all model restrictions. The efficiency of the standard IPW estimator is improved by incorporating information from incomplete cases through an augmented estimating equation which is optimal within a large class of estimating equations. We investigate the finite-sample properties of the proposed estimators in a simulation study and illustrate the new methodology in an application evaluating key correlates of preterm delivery for infants born to HIV infected mothers in Botswana, Africa.
| S-EPMC6051732 | biostudies-literature
Project description:Instrumental variables are routinely used to recover a consistent estimator of an exposure causal effect in the presence of unmeasured confounding. Instrumental variable approaches to account for nonignorable missing data also exist but are less familiar to epidemiologists. Like instrumental variables for exposure causal effects, instrumental variables for missing data rely on exclusion restriction and instrumental variable relevance assumptions. Yet these two conditions alone are insufficient for point identification. For estimation, researchers have invoked a third assumption, typically involving fairly restrictive parametric constraints. Inferences can be sensitive to these parametric assumptions, which are typically not empirically testable. The purpose of our article is to discuss another approach for leveraging a valid instrumental variable. Although the approach is insufficient for nonparametric identification, it can nonetheless provide informative inferences about the presence, direction, and magnitude of selection bias, without invoking a third untestable parametric assumption. An important contribution of this article is an Excel spreadsheet tool that can be used to obtain empirical evidence of selection bias and calculate bounds and corresponding Bayesian 95% credible intervals for a nonidentifiable population proportion. For illustrative purposes, we used the spreadsheet tool to analyze HIV prevalence data collected by the 2007 Zambia Demographic and Health Survey (DHS).
| S-EPMC5882580 | biostudies-literature