Dataset Information

Strategies to improve retention in randomised trials: a Cochrane systematic review and meta-analysis.

ABSTRACT:

Objective

To quantify the effect of strategies to improve retention in randomised trials.

Design

Systematic review and meta-analysis.

Data sources

Sources searched: MEDLINE, EMBASE, PsycINFO, DARE, CENTRAL, CINAHL, C2-SPECTR, ERIC, PreMEDLINE, Cochrane Methodology Register, Current Controlled Trials metaRegister, WHO trials platform, Society for Clinical Trials (SCT) conference proceedings and a survey of all UK clinical trial research units.

Review methods

Included trials were randomised evaluations of strategies to improve retention embedded within host randomised trials. The primary outcome was retention of trial participants. Data from trials were pooled using the fixed-effect model. Subgroup analyses were used to explore the heterogeneity and to determine whether there were any differences in effect by the type of strategy.

Results

38 retention trials were identified. Six broad types of strategies were evaluated. Strategies that increased postal questionnaire responses were: adding, that is, giving a monetary incentive (RR 1.18; 95% CI 1.09 to 1.28) and higher valued incentives (RR 1.12; 95% CI 1.04 to 1.22). Offering a monetary incentive, that is, an incentive given on receipt of a completed questionnaire, also increased electronic questionnaire response (RR 1.25; 95% CI 1.14 to 1.38). The evidence for shorter questionnaires (RR 1.04; 95% CI 1.00 to 1.08) and questionnaires relevant to the disease/condition (RR 1.07; 95% CI 1.01 to 1.14) is less clear. On the basis of the results of single trials, the following strategies appeared effective at increasing questionnaire response: recorded delivery of questionnaires (RR 2.08; 95% CI 1.11 to 3.87); a 'package' of postal communication strategies (RR 1.43; 95% CI 1.22 to 1.67) and an open trial design (RR 1.37; 95% CI 1.16 to 1.63). There is no good evidence that the following strategies impact on trial response/retention: adding a non-monetary incentive (RR=1.00; 95% CI 0.98 to 1.02); offering a non-monetary incentive (RR=0.99; 95% CI 0.95 to 1.03); 'enhanced' letters (RR=1.01; 95% CI 0.97 to 1.05); monetary incentives compared with offering prize draw entry (RR=1.04; 95% CI 0.91 to 1.19); priority postal delivery (RR=1.02; 95% CI 0.95 to 1.09); behavioural motivational strategies (RR=1.08; 95% CI 0.93 to 1.24); additional reminders to participants (RR=1.03; 95% CI 0.99 to 1.06) and questionnaire question order (RR=1.00, 0.97 to 1.02). Also based on single trials, these strategies do not appear effective: a telephone survey compared with a monetary incentive plus questionnaire (RR=1.08; 95% CI 0.94 to 1.24); offering a charity donation (RR=1.02, 95% CI 0.78 to 1.32); sending sites reminders (RR=0.96; 95% CI 0.83 to 1.11); sending questionnaires early (RR=1.10; 95% CI 0.96 to 1.26); longer and clearer questionnaires (RR=1.01, 0.95 to 1.07) and participant case management by trial assistants (RR=1.00; 95% CI 0.97 to 1.04).

Conclusions

Most of the trials evaluated questionnaire response rather than ways to improve participants return to site for follow-up. Monetary incentives and offers of monetary incentives increase postal and electronic questionnaire response. Some strategies need further evaluation. Application of these results would depend on trial context and follow-up procedures.

SUBMITTER: Brueton VC

PROVIDER: S-EPMC3918995 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:This review is an abridged version of a Cochrane Review previously published in the Cochrane Database of Systematic Reviews 2010, Issue 4, Art. No.: MR000013 DOI: 10.1002/14651858.MR000013.pub5 (see www.thecochranelibrary.com for information). Cochrane Reviews are regularly updated as new evidence emerges and in response to feedback, and Cochrane Database of Systematic Reviews should be consulted for the most recent version of the review. To identify interventions designed to improve recruitment to randomised controlled trials, and to quantify their effect on trial participation. Systematic review. The Cochrane Methodology Review Group Specialised Register in the Cochrane Library, MEDLINE, EMBASE, ERIC, Science Citation Index, Social Sciences Citation Index, C2-SPECTR, the National Research Register and PubMed. Most searches were undertaken up to 2010; no language restrictions were applied. Randomised and quasi-randomised controlled trials, including those recruiting to hypothetical studies. Studies on retention strategies, examining ways to increase questionnaire response or evaluating the use of incentives for clinicians were excluded. The study population included any potential trial participant (eg, patient, clinician and member of the public), or individual or group of individuals responsible for trial recruitment (eg, clinicians, researchers and recruitment sites). Two authors independently screened identified studies for eligibility. 45 trials with over 43 000 participants were included. Some interventions were effective in increasing recruitment: telephone reminders to non-respondents (risk ratio (RR) 1.66, 95% CI 1.03 to 2.46; two studies, 1058 participants), use of opt-out rather than opt-in procedures for contacting potential participants (RR 1.39, 95% CI 1.06 to 1.84; one study, 152 participants) and open designs where participants know which treatment they are receiving in the trial (RR 1.22, 95% CI 1.09 to 1.36; two studies, 4833 participants). However, the effect of many other strategies is less clear, including the use of video to provide trial information and interventions aimed at recruiters. There are promising strategies for increasing recruitment to trials, but some methods, such as open-trial designs and opt-out strategies, must be considered carefully as their use may also present methodological or ethical challenges. Questions remain as to the applicability of results originating from hypothetical trials, including those relating to the use of monetary incentives, and there is a clear knowledge gap with regard to effective strategies aimed at recruiters.

Project description:Background Retention of participants is essential to ensure the statistical power and internal validity of clinical trials. Poor participant retention reduces power and can bias the estimates of intervention effect. There is sparse evidence from randomised comparisons of effective strategies to retain participants in randomised trials. Currently, non-randomised evaluations of trial retention interventions embedded in host clinical trials are rejected from the Cochrane review of strategies to improve retention because it only included randomised evaluations. However, the systematic assessment of non-randomised evaluations may inform trialists’ decision-making about retention methods that have been evaluated in a trial context.Therefore, we performed a systematic review to synthesise evidence from non-randomised evaluations of retention strategies in order to supplement existing randomised trial evidence. Methods We searched MEDLINE, EMBASE, and Cochrane CENTRAL from 2007 to October 2017. Two reviewers independently screened abstracts and full-text articles for non-randomised studies that compared two or more strategies to increase participant retention in randomised trials. The retention trials had to be nested in real ‘host’ trials ( including feasibility studies) but not hypothetical trials. Two investigators independently rated the risk of bias of included studies using the ROBINS-I tool and determined the certainty of evidence using GRADE (Grading of Recommendations Assessment, Development and Evaluation) framework. Results Fourteen non-randomised studies of retention were included in this review. Most retention strategies (in 10 studies) aimed to increase questionnaire response rate. Favourable strategies for increasing questionnaire response rate were telephone follow-up compared to postal questionnaire completion, online questionnaire follow-up compared to postal questionnaire, shortened version of questionnaires versus longer questionnaires, electronically transferred monetary incentives compared to cash incentives, cash compared with no incentive and reminders to non-responders (telephone or text messaging). However, each retention strategy was evaluated in a single observational study. This, together with risk of bias concerns, meant that the overall GRADE certainty was low or very low for all included studies. Conclusions This systematic review provides low or very low certainty evidence on the effectiveness of retention strategies evaluated in non-randomised studies. Some strategies need further evaluation to provide confidence around the size and direction of the underlying effect.

Project description:Randomised control trials are regarded as the gold standard for evaluating the effectiveness and efficacy of healthcare interventions with thousands of trials published every year. Despite significant investment in infrastructure, a staggering number of clinical trials continue to face challenges with retention. Dropouts could lead to negative consequences-from lengthy delays to missing data that can undermine the results and integrity of the trial. Summarising evidence from non-randomised evaluations of retention strategies could provide complementary information to randomised evaluations that could guide trialists to the most effective ways of increasing retention of participants in clinical trials.The following electronic databases will be searched for relevant studies: EMBASE, MEDLINE, the Cochrane Controlled Trials Register, and Cochrane Methodology Register and the search will be limited to English-published studies during the last 10 years to increase relevance to current trials. Non-randomised studies (observational studies) including a comparison of two or more strategies to increase participant retention in randomised trials or comparing one or more strategies with no strategy will be included. The primary outcome will be the proportion of participants remained at the primary analysis as determined in each retention study.This review aims to gather and evaluate evidence on the effect of retention strategies examined in non-randomised studies. It is imperative to collect evidence from obseravational studies to infer whether or not these studies could be considered a practical way to complement or even replace a broadly favourable randomised design. If we find that non-randomised studies to be included in this review are of high quality with adequate control of biases, we will recommend to trialists and others not to rely exclusively on randomised studies and to give meticulous attention to the plentiful evidence that can be obtained from non-randomised studies. Should the results of this review suggest that evaluating retention strategies in observational studies provides insufficient evidence to trialists planning their retention strategies, we will be able to say that there is little point in conducting non-randomised studies and that they would do better to invest their time and resources in a randomised evaluation if possible. Where a non-randomised study design is chosen, the review authors will offer recommendations to trialists and others regarding how to ensure that these studies are conducted in a way that can minimise the risk of bias and increase confidence in the findings.PROSPERO 2017: CRD42017072775 .

Project description:Search strategies for systematic reviews aim to identify all evidence relevant to the research question posed. Reports of methodological research can be difficult to find leading to biased results in systematic reviews of research methodology. Evidence suggests that contact with investigators can help to identify unpublished research. To identify additional eligible randomised controlled trials (RCTs) for a Cochrane systematic review of strategies to improve retention in RCTs, we conducted a survey of UK clinical trials units (CTUs) and made contact with RCT methodologists.Key contacts for all UK CTUs were sent a personalised email with a short questionnaire and summary protocol of the Cochrane methodology review. The questionnaire asked whether a RCT evaluating strategies to improve retention embedded in a RCT had ever been conducted by the CTU. Questions about the stage of completion and publication of such RCTs were included. The summary protocol outlined the aims, eligibility criteria, examples of types of retention strategies, and the primary outcome for the systematic review. Personal communication with RCT methodologists and presentations of preliminary results of the review at conferences were also used to identify additional eligible RCTs. We checked the results of our standard searches to see if eligible studies identified through these additional methods were also found using our standard searches.We identified 14 of the 38 RCTs included in the Cochrane methodology review by contacting trials units and methodologists. Eleven of the 14 RCTs identified by these methods were either published in grey literature, in press or unpublished. Three remaining RCTs were fully published at the time. Six of the RCTs identified were not found through any other searches. The RCTs identified represented data for 6 of 14 RCTs of incentive strategies (52% of randomised participants included in the review), and 6 of 14 RCTs of communication strategies (52% of randomised participants included in the Cochrane review). Data were unavailable for two of the RCTs identified.Methodological evaluations embedded in RCTs may be unpublished, published in the grey literature or where published, poorly indexed in bibliographic databases. To identify such studies and minimise selection bias in systematic reviews of methodological evaluations, reviewers should consider contacting CTUs and trial methodologists.

Project description:BACKGROUND:Nudge interventions are those that seek to modify the social and physical environment to enhance capacity for subconscious behaviours that align with the intrinsic values of an individual, without actively restricting options. This study sought to describe the application and effects of nudge strategies on clinician implementation of health-related guidelines, policies and practices within studies included in relevant Cochrane systematic reviews. METHODS:As there is varied terminology used to describe nudge, this study examined studies within relevant systematic reviews. A two-stage screening process was undertaken where, firstly, all systematic reviews published in the Cochrane Library between 2016 and 2018 were screened to identify reviews that included quantitative studies to improve implementation of guidelines among healthcare providers. Secondly, individual studies within relevant systematic reviews were included if they were (i) randomised controlled trials (RCTs), (ii) included a nudge strategy in at least one intervention arm, and (iii) explicitly aimed to improve clinician implementation behaviour. We categorised nudge strategies into priming, salience and affect, default, incentives, commitment and ego, and norms and messenger based on the Mindspace framework. SYNTHESIS:The number and percentage of trials using each nudge strategy was calculated. Due to substantial heterogeneity, we did not undertake a meta-analysis. Instead, we calculated within-study point estimates and 95% confidence intervals, and used a vote-counting approach to explore effects. RESULTS:Seven reviews including 42 trials reporting on 57 outcomes were included. The most common nudge strategy was priming (69%), then norms and messenger (40%). Of the 57 outcomes, 86% had an effect on clinician behaviour in the hypothesised direction, and 53% of those were statistically significant. For continuous outcomes, the median effect size was 0.39 (0.22, 0.45), while for dichotomous outcomes the median Odds Ratio was 1.62 (1.13, 2.76). CONCLUSIONS:This review of 42 RCTs included in Cochrane systematic reviews found that the impact of nudge strategies on clinician behaviour was at least comparable to other interventions targeting implementation of evidence-based guidelines. While uncertainty remains, the review provides justification for ongoing investigation of the evaluation and application of nudge interventions to support provider behaviour change. TRIAL REGISTRATION:This review was not prospectively registered.

Project description:BackgroundRecruitment and retention challenges are very common in mental health randomised trials. Investigators utilise different methods to improve recruitment or retention. However, evidence of the effectiveness and efficiency of these strategies in mental health has not been synthesised. This systematic review is to investigate and assess the effectiveness and cost-effectiveness of different strategies to improve recruitment and retention in mental health randomised trials.Methods and materialsMEDLINE, EMBASE, the Cochrane Methodology Register and PsycINFO were searched from beginning of record up to July 2016. Randomised trials involving participants with mental health problems which compared different strategies for recruitment or retention were selected. Two authors independently screened identified studies for eligibility.ResultsA total of 5,157 citations were identified. Thirteen articles were included, 11 on recruitment and 2 on retention. Three randomised controlled trials compared different recruitment strategies, none of which found statistically significant differences between the interventional recruitment strategies and the routine recruitment methods. Retrospective comparisons of recruitment methods showed that non-web-based advertisement and recruitment by clinical research staff each have advantages in efficiency. Web-based adverts had the lowest cost per person recruited (£13.41 per person recruited). Specialised care referral cost £183.24 per person, non-web-based adverts cost £372.03 per patient and recruitment via primary care cost £407.65 for each patient. Financial incentives, abridged questionnaires and pre-notification had a positive effect on retention rates.ConclusionThe recruitment studies included showed differences in strategies, clinical settings, mental health conditions and study design. It is difficult to assess the overall effectiveness of any particular recruitment strategy as some strategies that worked well for a particular population may not work as well for others. Paying attention to the accessibility of information and consent materials may help improve recruitment. More research in this area is needed given its important implications.

Project description:AimTo quantify recruitment, retention and differential retention rates and associated trial, participant and intervention characteristics in randomised controlled trials (RCTs) evaluating the effect of exercise therapy in people with multimorbidity.Data sourcesMEDLINE, EMBASE, CINAHL and CENTRAL from 1990 to April 20, 2020.Study selectionRCTs including people with multimorbidity comparing exercise therapy with a non-exposed comparator group reporting at least one of the following outcomes: physical function, health-related quality of life, depression symptoms, or anxiety symptoms.Data extraction and synthesisRecruitment rates (proportion of people randomised/proportion of people eligible), retention rates (proportion of people providing the outcomes of interest/proportion randomised) and differential retention rates (difference in proportion of people providing the outcomes in the intervention group and comparator group) were calculated. Meta-analysis using a random-effects model was used to estimate pooled proportions. Methodological quality was assessed using Cochrane ´Risk of Bias tool 2.0´ for individual studies, and the GRADE approach was used to assess the overall quality of the evidence.ResultsTwenty-three RCTs with 3363 people were included. The pooled prevalence for recruitment rate was 75% (95%CI 66 to 84%). The pooled prevalence for retention rate was 90% (95%CI 86 to 94%) at the end of the intervention (12 weeks; interquartile range (IQR) (12 to 12)). Meta-regression analyses showed that increasing age and including a higher proportion of people with hypertension was associated with lower retention rates. Retention rates did not differ between the intervention and comparator groups. The overall quality of the evidence was deemed very low.ConclusionThree in four eligible people with multimorbidity were randomised to RCTs using exercise therapy, of which nine out of 10 provided end of treatment outcomes with no difference seen between the intervention and comparison groups. However, the results must be interpreted with caution due to large differences between the included studies.Trial registrationClinicalTrials.gov CRD42020161329 . Registered on 28 April 2020.

Project description:OBJECTIVE:Randomised controlled trials (RCT) are the gold standard to provide unbiased data. However, when patients have a treatment preference, randomisation may influence participation and outcomes (eg, external and internal validity). The aim of this study was to assess the influence of patients' preference in RCTs by analysing partially randomised patient preference trials (RPPT); an RCT and preference cohort combined. DESIGN:Systematic review and meta-analyses. DATA SOURCES:MEDLINE, Embase, PsycINFO and the Cochrane Library. ELIGIBILITY CRITERIA FOR SELECTING STUDIES:RPPTs published between January 2005 and October 2018 reporting on allocation of patients to randomised and preference cohorts were included. DATA EXTRACTION AND SYNTHESIS:Two independent reviewers extracted data. The main outcomes were the difference in external validity (participation and baseline characteristics) and internal validity (lost to follow-up, crossover and the primary outcome) between the randomised and the preference cohort within each RPPT, compared in a meta-regression using a Wald test. Risk of bias was not assessed, as no quality assessment for RPPTs has yet been developed. RESULTS:In total, 117 of 3734 identified articles met screening criteria and 44 were eligible (24 873 patients). The participation rate in RPPTs was >95% in 14 trials (range: 48%-100%) and the randomisation refusal rate was >50% in 26 trials (range: 19%-99%). Higher education, female, older age, race and prior experience with one treatment arm were characteristics of patients declining randomisation. The lost to follow-up and cross-over rate were significantly higher in the randomised cohort compared with the preference cohort. Following the meta-analysis, the reported primary outcomes were comparable between both cohorts of the RPPTs, mean difference 0.093 (95% CI -0.178 to 0.364, p=0.502). CONCLUSIONS:Patients' preference led to a substantial proportion of a specific patient group refusing randomisation, while it did not influence the primary outcome within an RPPT. Therefore, RPPTs could increase external validity without compromising the internal validity compared with RCTs. PROSPERO REGISTRATION NUMBER:CRD42019094438.