Dataset Information

Comparative effectiveness and safety of pharmaceuticals assessed in observational studies compared with randomized controlled trials.

ABSTRACT:

Background

There have been ongoing efforts to understand when and how data from observational studies can be applied to clinical and regulatory decision making. The objective of this review was to assess the comparability of relative treatment effects of pharmaceuticals from observational studies and randomized controlled trials (RCTs).

Methods

We searched PubMed and Embase for systematic literature reviews published between January 1, 1990, and January 31, 2020, that reported relative treatment effects of pharmaceuticals from both observational studies and RCTs. We extracted pooled relative effect estimates from observational studies and RCTs for each outcome, intervention-comparator, or indication assessed in the reviews. We calculated the ratio of the relative effect estimate from observational studies over that from RCTs, along with the corresponding 95% confidence interval (CI) for each pair of pooled RCT and observational study estimates, and we evaluated the consistency in relative treatment effects.

Results

Thirty systematic reviews across 7 therapeutic areas were identified from the literature. We analyzed 74 pairs of pooled relative effect estimates from RCTs and observational studies from 29 reviews. There was no statistically significant difference (based on the 95% CI) in relative effect estimates between RCTs and observational studies in 79.7% of pairs. There was an extreme difference (ratio < 0.7 or > 1.43) in 43.2% of pairs, and, in 17.6% of pairs, there was a significant difference and the estimates pointed in opposite directions.

Conclusions

Overall, our review shows that while there is no significant difference in the relative risk ratios between the majority of RCTs and observational studies compared, there is significant variation in about 20% of comparisons. The source of this variation should be the subject of further inquiry to elucidate how much of the variation is due to differences in patient populations versus biased estimates arising from issues with study design or analytical/statistical methods.

SUBMITTER: Hong YD

PROVIDER: S-EPMC8647453 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundThere is considerable debate as to the relative merits of using randomised controlled trial (RCT) data as opposed to observational data in systematic reviews of adverse effects. This meta-analysis of meta-analyses aimed to assess the level of agreement or disagreement in the estimates of harm derived from meta-analysis of RCTs as compared to meta-analysis of observational studies.Methods and findingsSearches were carried out in ten databases in addition to reference checking, contacting experts, citation searches, and hand-searching key journals, conference proceedings, and Web sites. Studies were included where a pooled relative measure of an adverse effect (odds ratio or risk ratio) from RCTs could be directly compared, using the ratio of odds ratios, with the pooled estimate for the same adverse effect arising from observational studies. Nineteen studies, yielding 58 meta-analyses, were identified for inclusion. The pooled ratio of odds ratios of RCTs compared to observational studies was estimated to be 1.03 (95% confidence interval 0.93-1.15). There was less discrepancy with larger studies. The symmetric funnel plot suggests that there is no consistent difference between risk estimates from meta-analysis of RCT data and those from meta-analysis of observational studies. In almost all instances, the estimates of harm from meta-analyses of the different study designs had 95% confidence intervals that overlapped (54/58, 93%). In terms of statistical significance, in nearly two-thirds (37/58, 64%), the results agreed (both studies showing a significant increase or significant decrease or both showing no significant difference). In only one meta-analysis about one adverse effect was there opposing statistical significance.ConclusionsEmpirical evidence from this overview indicates that there is no difference on average in the risk estimate of adverse effects of an intervention derived from meta-analyses of RCTs and meta-analyses of observational studies. This suggests that systematic reviews of adverse effects should not be restricted to specific study types. Please see later in the article for the Editors' Summary.

Project description:Background: The use of direct oral anticoagulants (DOACs) is recommended as the preferred treatment drug in patients with nonvalvular atrial fibrillation (AF). However, the effectiveness and safety of DOACs compared with vitamin K antagonists (VKAs) in patients with cancer and AF are still controversial. Therefore, we performed a meta-analysis regarding the effectiveness and safety of DOACs vs. VKAs in AF patients with cancer. Methods: A search of the Pubmed and EMBASE databases until August 2021 was performed. Adjusted risk ratios (RRs) and 95% confidence intervals (CIs) were pooled using a random-effects model with an inverse variance method. Results: Thirteen studies were deemed to meet the criteria. For the effectiveness outcomes, the use of DOACs compared with VKAs use was significantly associated with decreased risks of stroke or systemic embolism (RR = 0.66, 95% CI: 0.54-0.80) and venous thromboembolism (RR = 0.40, 95% CI: 0.26-0.61), but not ischemic stroke (RR = 0.79, 95% CI: 0.56-1.11), myocardial infarction (RR = 0.78, 95% CI: 0.56-1.11), cardiovascular death (RR = 0.76, 95% CI: 0.53-1.09), and all-cause death (RR = 0.82, 95% CI: 0.43-1.56). For the safety outcomes, compared with VKAs use, the use of DOACs was associated with reduced risks of intracranial bleeding (RR = 0.60, 95% CI: 0.50-0.71) and gastrointestinal bleeding (RR = 0.87, 95% CI: 0.80-0.95). There were no significant differences in major bleeding (RR = 0.87, 95% CI: 0.74-1.04), major or nonmajor clinically relevant bleeding (RR = 0.87, 95% CI: 0.74-1.01), and any bleeding (RR = 0.88, 95% CI: 0.76-1.03). Conclusion: Compared with VKAs, DOACs appeared to have significant reductions in stroke or systemic embolism, venous thromboembolism, intracranial bleeding, and gastrointestinal bleeding, but comparable risks of ischemic stroke, myocardial infarction, cardiovascular death, all-cause death, major bleeding, major or nonmajor clinically relevant bleeding, and any bleeding in patients with AF and cancer.

Project description:Worldwide, pertussis remains a major health problem among children. During the recent outbreaks of pertussis, maternal antenatal immunisation was introduced in several industrial countries. This systematic review aimed to synthesize evidence for the efficacy and safety of the pertussis vaccination that was given to pregnant women to protect infants from pertussis infection.We searched literature in the Cochrane Central Register of Controlled Trials, Medline, Embase, and OpenGrey between inception of the various databases and 16 May 2016. The search terms included 'pertussis', 'whooping cough', 'pertussis vaccine,' 'tetanus, diphtheria and pertussis vaccines' and 'pregnancy' and 'perinatal'.We included 15 articles in this review, which represented 12 study populations, involving a total of 203,835 mother-infant pairs from the US, the UK, Belgium, Israel, and Vietnam. Of the included studies, there were two randomised controlled trials (RCTs) and the rest were observational studies. Existing evidence suggests that vaccinations administered during 19-37 weeks of gestation are associated with significantly increased antibody levels in the blood of both mothers and their newborns at birth compared to placebo or no vaccination. However, there is a lack of robust evidence to suggest whether these increased antibodies can also reduce the incidence of pertussis (one RCT, n?=?48, no incidence in either group) and pertussis-related severe complications (one observational study) or mortality (no study) in infants. Meanwhile, there is no evidence of increased risk of serious complications such as stillbirth (e.g. one RCT, n?=?103, RR?=?0, meaning no case in the vaccine group), or preterm birth (two RCTs, n?=?151, RR?=?0.86, 95%CI: 0.14-5.21) related to administration of the vaccine during pregnancy.Given that pertussis infection is increasing in many countries and that newborn babies are at greatest risk of developing severe complications from pertussis, maternal vaccination in the later stages of pregnancy should continue to be supported while further research should fill knowledge gaps and strengthen evidence of its efficacy and safety.

Project description:BackgroundChinese medicine belly button application (CMBBA) has been used to treat childhood diarrhea (CD) in several randomized controlled trials (RCTs), but its effectiveness and combination strategy still need to be clarified. Therefore, we aimed to evaluate the effectiveness, safety, and the optimal combination strategy of CMBBA in treating CD.MethodsUp until January 2023, we searched for studies that met our inclusion criteria in six databases, including PubMed, the Cochrane Library, Chinese SinoMed, CNKI, VIP, and Wanfang. Heterogeneity was quantified using I2 statistics. A methodological evaluation was performed using the Cochrane Risk Bias Tool 2.0. The Confidence in Network Meta-Analysis online software was employed to evaluate evidence grading. A minimally contextualized framework was used to provide a comprehensive conclusion for the network meta-analysis. This study protocol was registered with PROSPERO.ResultsWe analyzed data from 33 RCTs that included 4,490 children with diarrhea. In terms of clinical effectiveness, CMBBA plus montmorillonite powder plus anti-infectives may be the most effective treatment option for children with diarrhea and concurrent infection according to a minimally contextualized framework. Either exclusive use of CMBBA or CMBBA in combination with modern medicine was beneficial in reducing the time to diarrhea disappearance (MD = -1.33 days, 95% CI: -1.59 to -1.08, Z = -10.103, p < 0.001) compared to modern medicine exclusively, and the difference was statistically significant. The combined usage of CMBBA could shorten the recovery time of dehydration by an average of 0.74 days (MD = -0.74 days, 95% CI: -1.10 to -0.37, Z = -3.931.103, p < 0.001). While some studies have reported mild allergic reactions and mild abdominal pain after CMBBA use, these symptoms can be cured in a relatively short period of time.ConclusionsThe combination of CMBBA, montmorillonite powder, and anti-infectives may provide superior clinical effectiveness for children with diarrhea and concurrent infection. To treat CD, CMBBA can be used effectively and safely. However, the findings must be interpreted with cautiously due to the limited number of clinical trials and the low quality of the studies. In addition, the choice of treatment plan should also be based on the specific conditions of each patient.Systematic review registrationhttps://www.crd.york.ac.uk/prospero/, identifier: CRD42022380694.