Dataset Information

Evaluating the methodology of studies conducted during the global COVID-19 pandemic: A systematic review of randomized controlled trials.

ABSTRACT:

Background

The therapeutic evidence collected from well-designed studies is needed to help manage the global pandemic of the coronavirus disease 2019 (COVID-19). Evaluating the quality of therapeutic data collected during this most recent pandemic is important for improving future clinical research under similar circumstances.

Objective

To assess the methodological quality and variability in implementation of randomized controlled trials (RCTs) for treating COVID-19, and to analyze the support that should be provided to improve data collected during an urgent pandemic situation.

Search strategy

PubMed, Excerpta Medica Database, China National Knowledge Infrastructure, Wanfang, and Chongqing VIP, and the preprint repositories including Social Science Research Network and MedRxiv were systematically searched, up to September 30, 2020, using the keywords "coronavirus disease 2019 (COVID-19)," "2019 novel coronavirus (2019-nCoV)," "severe acute respiratory syndrome-related coronavirus-2 (SARS-CoV-2)," "novel coronavirus pneumonia (NCP)," "randomized controlled trial (RCT)" and "random."

Inclusion criteria

RCTs studying the treatment of COVID-19 were eligible for inclusion.

Data extraction and analysis

Screening of published RCTs for inclusion and data extraction were each conducted by two researchers. Analysis of general information on COVID-19 RCTs was done using descriptive statistics. Methodological quality was assessed using the risk-of-bias tools in the Cochrane Handbook for Systematic Reviews of Interventions (Version 5.1.0). Variability in implementation was assessed by comparing consistency between RCT reports and registration information.

Results

A total of 5886 COVID-19 RCTs were identified. Eighty-one RCTs were finally included, of which, 45 had registration information. Methodological quality of the RTCs was not optimal due to deficiencies in five main domains: allocation concealment, blinding of participants and personnel, blinding of outcome assessment, incomplete outcome data, and selective reporting. Comparisons of consistency between published protocols and registration information showed that the 45 RCTs with registration information had common deviations in seven items: inclusion and exclusion criteria, sample size, outcomes, research sites of recruitment, interventions, and blinding.

Conclusion

The methodological quality of COVID-19 RCTs conducted in early to mid 2020 was consistently low and variability in implementation was common. More support for implementing high-quality methodology is needed to obtain the quality of therapeutic evidence needed to provide positive guidance for clinical care. We make an urgent appeal for accelerating the construction of a collaborative sharing platform and preparing multidisciplinary talent and professional teams to conduct excellent clinical research when faced with epidemic diseases of the future. Further, variability in RCT implementation should be clearly reported and interpreted to improve the utility of data resulting from those trials.

SUBMITTER: Zhao MZ

PROVIDER: S-EPMC7970417 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundRandomized controlled trials (RCTs) that are inappropriately designed or executed may provide biased findings and mislead clinical practice. In view of recent interest in the treatment and prevention of thrombotic complications in cancer patients we evaluated the characteristics, risk of bias and their time trends in RCTs of anticoagulation in patients with cancer.MethodsWe conducted a comprehensive search, including a search of four electronic databases (MEDLINE, EMBASE, ISI the Web of Science, and CENTRAL) up to February 2010. We included RCTs in which the intervention and/or comparison consisted of: vitamin K antagonists, unfractionated heparin (UFH), low molecular weight heparin (LMWH), direct thrombin inhibitors or fondaparinux. We performed descriptive analyses and assessed the association between the variables of interest and the year of publication.ResultsWe included 67 RCTs with 24,071 participants. In twenty one trials (31%) DVT diagnosis was triggered by clinical suspicion; the remaining trials either screened for DVT or were unclear about their approach. 41 (61%), 22 (33%), and 11 (16%) trials respectively reported on major bleeding, minor bleeding, and thrombocytopenia. The percentages of trials satisfying risk of bias criteria were: adequate sequence generation (85%), adequate allocation concealment (61%), participants' blinding (39%), data collectors' blinding (44%), providers' blinding (41%), outcome assessors' blinding (75%), data analysts' blinding (15%), intention to treat analysis (57%), no selective outcome reporting (12%), no stopping early for benefit (97%). The mean follow-up rate was 96%. Adequate allocation concealment and the reporting of intention to treat analysis were the only two quality criteria that improved over time.ConclusionsMany RCTs of anticoagulation in patients with cancer appear to use insufficiently rigorous outcome assessment methods and to have deficiencies in key methodological features. It is not clear whether this reflects a problem in the design, conduct or the reporting of these trials, or both. Future trials should avoid the shortcomings described in this article.

Project description:BackgroundRoad traffic injuries are among the top ten causes of death globally, with the highest burden in low and middle-income countries, where over a third of deaths occur among pedestrians and cyclists. Several interventions to mitigate the burden among pedestrians have been widely implemented, however, the effectiveness has not been systematically examined.ObjectivesTo assess the effectiveness of interventions to reduce road traffic crashes, injuries, hospitalizations and deaths among pedestrians.MethodsWe considered studies that evaluated interventions to reduce road traffic crashes, injuries, hospitalizations and/or deaths among pedestrians. We considered randomized controlled trials, interrupted time-series studies, and controlled before-after studies. We searched MEDLINE, EMBASE, Web of Science, WHO Global Health Index, Health Evidence, Transport Research International Documentation and ClinicalTrials.gov through 31 August 2020, and the reference lists of all included studies. Two reviewers independently screened titles and abstracts and full texts, extracted data and assessed the risk of bias. We summarized findings narratively with text and tables.ResultsA total of 69123 unique records were identified through the searches, with 26 of these meeting our eligibility criteria. All except two of these were conducted in high-income countries and most were from urban settings. The majority of studies observed either a clear effect favoring the intervention or an unclear effect potentially favoring the intervention and these included: changes to the road environment (19/27); changes to legislation and enforcement (12/12); and road user behavior/education combined with either changes to the road environment (3/3) or with legislation and enforcement (1/1). A small number of studies observed either a null effect or an effect favoring the control.ConclusionsAlthough the highest burden of road traffic injuries exists in LMICs, very few studies have examined the effectiveness of available interventions in these settings. Studies indicate that road environment, legislation and enforcement interventions alone produce positive effects on pedestrian safety. In combination with or with road user behavior/education interventions they are particularly effective in improving pedestrian safety.

Project description:BackgroundThe purported ergogenic and health effects of probiotics have been a topic of great intrigue among researchers, practitioners, and the lay public alike. There has also been an increased research focus within the realm of sports science and exercise medicine on the athletic gut microbiota. However, compared to other ergogenic aids and dietary supplements, probiotics present unique study challenges. The objectives of this systematic scoping review were to identify and characterize study methodologies of randomized controlled trials investigating supplementation with probiotics in athletes and physically active individuals.MethodsFour databases (MEDLINE, CINAHL, Cochrane CENTRAL, and Cochrane Database of Systematic Reviews) were searched for randomized controlled studies involving healthy athletes or physically active individuals. An intervention with probiotics and inclusion of a control and/or placebo group were essential. Only peer-reviewed articles in English were considered, and there were no date restrictions. Results were extracted and presented in tabular form to detail study protocols, characteristics, and outcomes. Bias in randomized controlled trials was determined with the RoB 2.0 tool.ResultsA total of 45 studies were included in the review, with 35 using a parallel group design and 10 using a cross-over design. Approximately half the studies used a single probiotic and the other half a multi-strain preparation. The probiotic dose ranged from 2 × 108 to 1 × 1011 colony forming units daily, and the length of intervention was between 7 and 150 days. Fewer than half the studies directly assessed gastrointestinal symptoms, gut permeability, or the gut microbiota. The sex ratio of participants was heavily weighted toward males, and only 3 studies exclusively investigated females. Low-level adverse events were reported in only 2 studies, although the methodology of reporting varied widely. The risk of bias was generally low, although details on randomization were lacking in some studies.ConclusionThere is a substantial body of research on the effects of probiotic supplementation in healthy athletes and physically active individuals. Considerable heterogeneity in probiotic selection and dosage as well as outcome measures has made clinical and mechanistic interpretation challenging for both health care practitioners and researchers. Attention to issues of randomization of participants, treatments and interventions, selection of outcomes, demographics, and reporting of adverse events will facilitate more trustworthy interpretation of probiotic study results and inform evidence-based guidelines.

Project description:BackgroundGamification refers to the use of game elements in nongame contexts. The use of gamification to change behaviors and promote physical activity (PA) is a promising avenue for tackling the global physical inactivity pandemic and the current prevalence of chronic diseases. However, there is no evidence of the effectiveness of gamified interventions with the existence of mixed results in the literature.ObjectiveThe aim of this systematic review and meta-analysis is to evaluate the effectiveness of gamified interventions and their health care potential by testing the generalizability and sustainability of their influence on PA and sedentary behavior.MethodsA total of 5 electronic databases (PubMed, Embase, Scopus, Web of Science, and the Cochrane Central Register of Controlled Trials) were searched for randomized controlled trials published in English from 2010 to 2020. Eligibility criteria were based on the components of the participants, interventions, comparators, and outcomes framework. Studies were included when they used gamified interventions in daily life with an active or inactive control group and when they assessed a PA or sedentary behavior outcome. We conducted meta-analyses using a random-effects model approach. Sensitivity analyses, influence analyses, and publication bias analyses were performed to examine the robustness of our results.ResultsThe main meta-analysis performed on 16 studies and 2407 participants revealed a small to medium summary effect of gamified interventions on PA behavior (Hedges g=0.42, 95% CI 0.14-0.69). No statistical difference among different subgroups (adults vs adolescents and healthy participants vs adults with chronic diseases) and no interaction effects with moderators such as age, gender, or BMI were found, suggesting good generalizability of gamified interventions to different user populations. The effect was statistically significant when gamified interventions were compared with inactive control groups, such as waiting lists (Hedges g=0.58, 95% CI 0.08-1.07), and active control groups that included a nongamified PA intervention (Hedges g=0.23, 95% CI 0.05-0.41). This suggests that gamified interventions are not only efficient in changing behavior but also more effective compared with other behavioral interventions. The long-term effect (measured with follow-up averaging 14 weeks after the end of the intervention) was weaker, with a very small to small effect (Hedges g=0.15, 95% CI 0.07-0.23).ConclusionsThis meta-analysis confirms that gamified interventions are promising for promoting PA in various populations. Additional analyses revealed that this effect persists after the follow-up period, suggesting that it is not just a novelty effect caused by the playful nature of gamification, and that gamified products appear effective compared with equivalent nongamified PA interventions. Future rigorous trials are required to confirm these findings.

Project description:BackgroundThis systematic review evaluated the effectiveness of process interventions (education for clinicians, parent presence, education of parents [before and on day of vaccination], and education of patients on day of vaccination) on reducing vaccination pain, fear, and distress and increasing the use of interventions during vaccination.Design/methodsDatabases were searched using a broad search strategy to identify relevant randomized and quasi-randomized controlled trials. Critical outcomes were pain, fear, distress (when applicable), and use of pain management interventions. Data were extracted according to procedure phase (preprocedure, acute, recovery, combinations of these) and pooled using established methods. Analyses were conducted using standardized mean differences (SMD) and risk ratios (RR).ResultsThirteen studies were included. Results were generally mixed. On the basis of low to very low-quality evidence, the following specific critical outcomes showed significant effects suggesting: (1) clinicians should be educated about vaccine injection pain management (use of interventions: SMD 0.66; 95% confidence interval [CI]: 0.47, 0.85); (2) parents should be present (distress preprocedure: SMD -0.85; 95% CI: -1.35, -0.35); (3) parents should be educated before the vaccination day (use of intervention preprocedure: SMD 0.83; 95% CI: 0.25, 1.41 and RR, 2.08; 95% CI: 1.51, 2.86; distress acute: SMD, -0.35; 95% CI: -0.57, -0.13); (4) parents should be educated on the vaccination day (use of interventions: SMD 1.02; 95% CI: 0.22, 1.83 and RR, 2.42; 95% CI: 1.47, 3.99; distress preprocedure+acute+recovery: SMD -0.48; 95% CI: -0.82, -0.15); and (5) individuals 3 years of age and above should be educated on the day of vaccination (fear preprocedure: SMD -0.67; 95% CI: -1.28, -0.07).ConclusionsEducating individuals involved in the vaccination procedure (clinicians, parents of children being vaccinated; individuals above 3 y of age) is beneficial to increase use of pain management strategies, reduce distress surrounding with vaccination, and to reduce fear. When possible, parent presence is also recommended for children undergoing vaccination.

Project description:BackgroundProton beam therapy (PBT) delivers high-energy radiation to target tumours while sparing surrounding normal tissues. The dosimetric advantages of PBT over traditional photon radiotherapy may be clear but the translation of this benefit into clinically meaningful reductions in toxicities and improved quality-of-life (QoL) needs to be determined. Randomised controlled trials (RCTs) are considered the gold standard for generating the highest-level evidence in medicine. The objectives of this systematic review were to provide an overview of published clinical studies evaluating the benefits of PBT, and to examine the methodology used in clinical trials with respect to study design and outcomes.MethodsPubMed, EMBASE and Cochrane databases were systematically searched for published clinical studies where PBT was a cancer treatment intervention. All randomised and non-randomised studies, prospective or retrospective, were eligible for inclusion.ResultsIn total, 219 studies were included. Prospective studies comprised 89/219 (41%), and of these, the number of randomised phase II and III trials were 5/89 (6%) and 3/89 (3%) respectively. Of all the phase II and III trials, 18/24 (75%) were conducted at a single PBT centre. Over one-third of authors recommended an increase in length of follow up. Research design and/or findings were poorly reported in 74/89 (83%) of prospective studies. Patient reported outcomes were assessed in only 19/89 (21%) of prospective studies.ConclusionsProspective randomised evidence for PBT is limited. The set-up of national PBT services in several countries provides an opportunity to guide the optimal design of prospective studies, including RCTs, to evaluate the benefits of PBT across various disease sites. Collaboration between PBT centres, both nationally and internationally, would increase potential for the generation of practice changing evidence. There is a need to facilitate and guide the collection and analysis of meaningful outcome data, including late toxicities and patient reported QoL.