Dataset Information

Dynamic methods for ongoing assessment of site-level risk in risk-based monitoring of clinical trials: A scoping review.

ABSTRACT:

Background/aims

It is increasingly recognised that reliance on frequent site visits for monitoring clinical trials is inefficient. Regulators and trialists have recently encouraged more risk-based monitoring. Risk assessment should take place before a trial begins to define the overarching monitoring strategy. It can also be done on an ongoing basis, to target sites for monitoring activity. Various methods have been proposed for such prioritisation, often using terms like 'central statistical monitoring', 'triggered monitoring' or, as in the International Conference on Harmonization Good Clinical Practice guidance, 'targeted on-site monitoring'. We conducted a scoping review to identify such methods, to establish if any were supported by adequate evidence to allow wider implementation, and to guide future developments in this field of research.

Methods

We used seven publication databases, two sets of methodological conference abstracts and an Internet search engine to identify methods for using centrally held trial data to assess site conduct during a trial. We included only reports in English, and excluded reports published before 1996 or not directly relevant to our research question. We used reference and citation searches to find additional relevant reports. We extracted data using a predefined template. We contacted authors to request additional information about included reports.

Results

We included 30 reports in our final dataset, of which 21 were peer-reviewed publications. In all, 20 reports described central statistical monitoring methods (of which 7 focussed on detection of fraud or misconduct) and 9 described triggered monitoring methods; 21 reports included some assessment of their methods' effectiveness, typically exploring the methods' characteristics using real trial data without known integrity issues. Of the 21 with some effectiveness assessment, most contained limited information about whether or not concerns identified through central monitoring constituted meaningful problems. Several reports demonstrated good classification ability based on more than one classification statistic, but never without caveats of unclear reporting or other classification statistics being low or unavailable. Some reports commented on cost savings from reduced on-site monitoring, but none gave detailed costings for the development and maintenance of central monitoring methods themselves.

Conclusion

Our review identified various proposed methods, some of which could be combined within the same trial. The apparent emphasis on fraud detection may not be proportionate in all trial settings. Despite some promising evidence and some self-justifying benefits for data cleaning activity, many proposed methods have limitations that may currently prevent their routine use for targeting trial monitoring activity. The implementation costs, or uncertainty about these, may also be a barrier. We make recommendations for how the evidence-base supporting these methods could be improved.

SUBMITTER: Cragg WJ

PROVIDER: S-EPMC8010889 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:Muscle loss alone, or in the context of sarcopenia or cachexia, is a prevalent condition and a predictor of negative outcomes in aging and disease. As adequate nutrition is essential for muscle maintenance, a growing number of studies has been conducted to explore the role of specific nutrients on muscle mass or function. Nonetheless, more research is needed to guide evidence-based recommendations. This scoping review aimed to compile and document ongoing clinical trials investigating nutrition interventions as a strategy to prevent or treat low muscle mass or function (strength and physical performance), sarcopenia, or cachexia. ClinicalTrials.gov and the WHO International Clinical Trials Registry Platform were searched up to 21 April 2021 for planned and ongoing trials. Randomized controlled trials with ≥20 participants per arm were included based on intent to explore the effects of nutrition interventions on muscle-related outcomes (i.e. muscle mass or strength, physical performance, or muscle synthesis rate) in both clinical and non-clinical conditions (i.e. aging). Two reviewers independently screened records for eligibility, and a descriptive synthesis of trials characteristics was conducted. A total of 113 trials were included in the review. Most trials (69.0%) enroll adults with clinical conditions, such as cancer (19.5%), obesity and metabolic diseases (16.8%), and musculoskeletal diseases (10.7%). The effects of nutrition interventions on age-related muscle loss are explored in 31% of trials. Although nutrition interventions of varied types were identified, food supplements alone (48.7%) or combined with dietary advice (11.5%) are most frequently reported. Protein (17.7%), amino acids (10.6%), and β-hydroxy-β-methylbutyrate (HMB, 6.2%) are the top three food supplements' nutrients under investigation. Primary outcome of most trials (54.9%) consists of measures of muscle mass alone or in combination with muscle strength and/or performance (as either primary or secondary outcomes). Muscle strength and physical performance are primary outcomes of 38% and 31.9% of the trials, respectively. These measurements were obtained using a variety of techniques. Only a few trials evaluate muscle synthesis rate either as a primary or secondary outcome (5.3%). Several nutrition studies focusing on muscle, sarcopenia, and cachexia are underway and can inform future research in this area. Although many trials have similar type of interventions, methodological heterogeneity may challenge study comparisons, and future meta-analyses aiming to provide evidence-based recommendations. Upcoming research in this area may benefit from guidelines for the assessment of therapeutic effects of nutrition interventions.

Project description:BackgroundSocial media has changed the communication landscape, exposing individuals to an ever-growing amount of information while also allowing them to create and share content. Although vaccine skepticism is not new, social media has amplified public concerns and facilitated their spread globally. Multiple studies have been conducted to monitor vaccination discussions on social media. However, there is currently insufficient evidence on the best methods to perform social media monitoring.ObjectiveThe aim of this study was to identify the methods most commonly used for monitoring vaccination-related topics on different social media platforms, along with their effectiveness and limitations.MethodsA systematic scoping review was conducted by applying a comprehensive search strategy to multiple databases in December 2018. The articles' titles, abstracts, and full texts were screened by two reviewers using inclusion and exclusion criteria. After data extraction, a descriptive analysis was performed to summarize the methods used to monitor and analyze social media, including data extraction tools; ethical considerations; search strategies; periods monitored; geolocalization of content; and sentiments, content, and reach analyses.ResultsThis review identified 86 articles on social media monitoring of vaccination, most of which were published after 2015. Although 35 out of the 86 studies used manual browser search tools to collect data from social media, this was time-consuming and only allowed for the analysis of small samples compared to social media application program interfaces or automated monitoring tools. Although simple search strategies were considered less precise, only 10 out of the 86 studies used comprehensive lists of keywords (eg, with hashtags or words related to specific events or concerns). Partly due to privacy settings, geolocalization of data was extremely difficult to obtain, limiting the possibility of performing country-specific analyses. Finally, 20 out of the 86 studies performed trend or content analyses, whereas most of the studies (70%, 60/86) analyzed sentiments toward vaccination. Automated sentiment analyses, performed using leverage, supervised machine learning, or automated software, were fast and provided strong and accurate results. Most studies focused on negative (n=33) and positive (n=31) sentiments toward vaccination, and may have failed to capture the nuances and complexity of emotions around vaccination. Finally, 49 out of the 86 studies determined the reach of social media posts by looking at numbers of followers and engagement (eg, retweets, shares, likes).ConclusionsSocial media monitoring still constitutes a new means to research and understand public sentiments around vaccination. A wide range of methods are currently used by researchers. Future research should focus on evaluating these methods to offer more evidence and support the development of social media monitoring as a valuable research design.

Project description:BackgroundIn November 2016, the ICH published a requirement for sponsors to develop a systematic, prioritised, risk-based approach to monitoring clinical trials. This approach is more commonly known as risk-based monitoring (RBM). However, recent evidence suggests that a 'gold standard', validated approach to RBM does not exist and it is unclear how sponsors will introduce RBM into their organisations. A first step needed to inform the implementation of RBM is to explore academic trialists' readiness and ability to perform RBM. The aim of this paper is to identify the attitudes and perceived barriers and facilitators to the implementation of RBM in academic-led clinical trials in Ireland.MethodsThis is a mixed-methods, explanatory sequential design, with quantitative survey followed by semistructured interviews. Academic clinical researchers (N = 132) working in Ireland were surveyed to examine their use and perceptions of RBM. A purposive sample of survey participants (n = 22) were then interviewed to gain greater insight into the quantitative findings. The survey and interview data were merged to generate a list of perceived barriers and facilitators to RBM implementation, with suggestions for, and solutions to, these issues.ResultsSurvey response rate was 49% (132/273). Thirteen percent (n = 18) of responders were not familiar with the term risk-based monitoring and less than a quarter of respondents (21%, n = 28) had performed RBM in a clinical trial. Barriers to RBM implementation included lack of RBM knowledge/training, increased costs caused by greater IT demands, increased workload for trial staff and lack of evidence to support RBM as an effective monitoring approach. Facilitators included participants' legal obligation to perform RBM under the new ICH-GCP guidelines, availability of RBM guidance and perception of cost savings by performing RBM in future trials.ConclusionThe results of this study demonstrate a need for training and regulatory-endorsed guidelines to support the implementation of RBM in academic-led clinical trials. The study provides valuable insights to inform interventions and strategies by policy-makers and clinical trial regulators to improve RBM uptake.

Project description:Background/aimsThere has been growing interest in better understanding the potential of observational research methods in medical product evaluation and regulatory decision-making. Previously, we used linked claims and electronic health record data to emulate two ongoing randomized controlled trials, characterizing the populations and results of each randomized controlled trial prior to publication of its results. Here, our objective was to compare the populations and results from the emulated trials with those of the now-published randomized controlled trials.MethodsThis study compared participants' demographic and clinical characteristics and study results between the emulated trials, which used structured data from OptumLabs Data Warehouse, and the published PRONOUNCE and GRADE trials. First, we examined the feasibility of implementing the baseline participant characteristics included in the published PRONOUNCE and GRADE trials' using real-world data and classified each variable as ascertainable, partially ascertainable, or not ascertainable. Second, we compared the emulated trials and published randomized controlled trials for baseline patient characteristics (concordance determined using standardized mean differences <0.20) and results of the primary and secondary endpoints (concordance determined by direction of effect estimates and statistical significance).ResultsThe PRONOUNCE trial enrolled 544 participants, and the emulated trial included 2226 propensity score-matched participants. In the PRONOUNCE trial publication, one of the 32 baseline participant characteristics was listed as an exclusion criterion on ClinicalTrials.gov but was ultimately not used. Among the remaining 31 characteristics, 9 (29.0%) were ascertainable, 11 (35.5%) were partially ascertainable, and 10 (32.2%) were not ascertainable using structured data from OptumLabs. For one additional variable, the PRONOUNCE trial did not provide sufficient detail to allow its ascertainment. Of the nine variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 6 (66.7%). The primary endpoint of time from randomization to the first major adverse cardiovascular event and secondary endpoints of nonfatal myocardial infarction and stroke were concordant between the emulated trial and published randomized controlled trial. The GRADE trial enrolled 5047 participants, and the emulated trial included 7540 participants. In the GRADE trial publication, 8 of 34 (23.5%) baseline participant characteristics were ascertainable, 14 (41.2%) were partially ascertainable, and 11 (32.4%) were not ascertainable using structured data from OptumLabs. For one variable, the GRADE trial did not provide sufficient detail to allow for ascertainment. Of the eight variables that were ascertainable, values in the emulated trial and published randomized controlled trial were discordant for 4 (50.0%). The primary endpoint of time to hemoglobin A1c ≥7.0% was mostly concordant between the emulated trial and the published randomized controlled trial.ConclusionDespite challenges, observational methods and real-world data can be leveraged in certain important situations for a more timely evaluation of drug effectiveness and safety in more diverse and representative patient populations.

Project description:In a cluster-randomized trial (CRT), the number of participants enrolled often varies across clusters. This variation should be considered during both trial design and data analysis to ensure statistical performance goals are achieved. Most methodological literature on the CRT design has assumed equal cluster sizes. This scoping review focuses on methodology for unequal cluster size CRTs. EMBASE, Medline, Google Scholar, MathSciNet and Web of Science databases were searched to identify English-language articles reporting on methodology for unequal cluster size CRTs published until March 2021. We extracted data on the focus of the paper (power calculation, Type I error etc.), the type of CRT, the type and the range of parameter values investigated (number of clusters, mean cluster size, cluster size coefficient of variation, intra-cluster correlation coefficient, etc.), and the main conclusions. Seventy-nine of 5032 identified papers met the inclusion criteria. Papers primarily focused on the parallel-arm CRT (p-CRT, n = 60, 76%) and the stepped-wedge CRT (n = 14, 18%). Roughly 75% of the papers addressed trial design issues (sample size/power calculation) while 25% focused on analysis considerations (Type I error, bias, etc.). The ranges of parameter values explored varied substantially across different studies. Methods for accounting for unequal cluster sizes in the p-CRT have been investigated extensively for Gaussian and binary outcomes. Synthesizing the findings of these works is difficult as the magnitude of impact of the unequal cluster sizes varies substantially across the combinations and ranges of input parameters. Limited investigations have been done for other combinations of a CRT design by outcome type, particularly methodology involving binary outcomes-the most commonly used type of primary outcome in trials. The paucity of methodological papers outside of the p-CRT with Gaussian or binary outcomes highlights the need for further methodological development to fill the gaps.