Dataset Information

Progression from external pilot to definitive randomised controlled trial: a methodological review of progression criteria reporting.

ABSTRACT:

Objectives

Prespecified progression criteria can inform the decision to progress from an external randomised pilot trial to a definitive randomised controlled trial. We assessed the characteristics of progression criteria reported in external randomised pilot trial protocols and results publications, including whether progression criteria were specified a priori and mentioned in prepublication peer reviewer reports.

Study design

Methodological review.

Methods

We searched four journals through PubMed: British Medical Journal Open, Pilot and Feasibility Studies, Trials and Public Library of Science One. Eligible publications reported external randomised pilot trial protocols or results, were published between January 2018 and December 2019 and reported progression criteria. We double data extracted 25% of the included publications. Here we report the progression criteria characteristics.

Results

We included 160 publications (123 protocols and 37 completed trials). Recruitment and retention were the most frequent indicators contributing to progression criteria. Progression criteria were mostly reported as distinct thresholds (eg, achieving a specific target; 133/160, 83%). Less than a third of the planned and completed pilot trials that included qualitative research reported how these findings would contribute towards progression criteria (34/108, 31%). The publications seldom stated who established the progression criteria (12/160, 7.5%) or provided rationale or justification for progression criteria (44/160, 28%). Most completed pilot trials reported the intention to proceed to a definitive trial (30/37, 81%), but less than half strictly met all of their progression criteria (17/37, 46%). Prepublication peer reviewer reports were available for 153/160 publications (96%). Peer reviewer reports for 86/153 (56%) publications mentioned progression criteria, with peer reviewers of 35 publications commenting that progression criteria appeared not to be specified.

Conclusions

Many external randomised pilot trial publications did not adequately report or propose prespecified progression criteria to inform whether to proceed to a future definitive randomised controlled trial.

SUBMITTER: Mellor K

PROVIDER: S-EPMC8240572 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundExternal randomised pilot trials aim to determine whether a future definitive randomised controlled trial (RCT) should be conducted, and if so, how. However, not every pilot trial that suggests that a definitive trial will be feasible will progress to a definitive study. In this study, we surveyed corresponding authors of external randomised pilot trial publications to assess pilot trial outcomes in terms of feasibility and progression.MethodsWeb-based surveys were sent to corresponding authors of external randomised pilot trial publications, open for four weeks between January and February 2022. Four surveys were produced depending on whether the corresponding author had published a trial protocol or results publication, and whether progression criteria were reported. Surveys asked whether a future RCT was considered feasible, whether progression criteria were met (if applicable), what other factors informed the assessment of pilot trial feasibility, and whether the pilot trial has progressed to further research. Data was analysed using descriptive statistics and conventional content analysis.Results98 of 276 corresponding authors completed the survey (average response rate of 36% across all surveys). Of these, 89 respondents indicated that their trial had completed. Ninety per cent of respondents who were corresponding authors of completed pilot trials stated that their pilot trial was either feasible (42/89, 47%) or feasible with changes to the trial design (38/89, 43%), yet only 66% (59/89) reported the intention to conduct a future definitive trial. Availability of funding for a future definitive trial and changing priorities of the Chief Investigator were the most common barriers to progression identified. Qualitative research findings was the most frequent factor considered both by corresponding authors who reported and who did not report progression criteria when determining trial feasibility.ConclusionsJust under one quarter (21/89, 24%) of respondents who considered their external randomised pilot trial to be feasible, or feasible with changes, did not intend to conduct a definitive trial highlighting research inefficiency and waste.Trial registrationOpen Science Framework osf.io/d28hr [20 December 2021].

Project description:Feasibility studyObjectives:Refine a bespoke enhanced support intervention (ESI) (including self-help materials, intervention manual and training) for implementation by community pharmacy (CP) staff to people with sub-threshold depression and long-term conditions (LTCs) based upon evidence-supported interventions in primary careDevelop and refine study procedures (recruitment strategies and set up, screening, participant recruitment, assessment, suitability of outcome measures and data collection procedures) for testing in the pilot study phaseDesign: A case series/qualitative studySetting: UK community pharmacyPopulation: Adults with long-term health conditions who screen-positive for depression but who do not reach the threshold for DSM IV Moderate Depressive disorderIntervention: Enhanced support intervention (ESI) delivered by an appropriately trained community pharmacy team member involving four to six sessions over four months. ESI is a modified form of an intervention within the collaborative care framework for sub-threshold depression validated in previous studies in UK primary care which appears suitable for implementation in community settings.Sample size: 20-30 participantsOutcomes: Study implementation (recruitment and attrition rates), quality of data collection at baseline and 4 months and ESI adherence (number of contacts, DNA and drop out) as per objectives 1a/bQualitative evaluation: Semi-structured interviews with up to 10 participants and ESI facilitators and focus group(s) (range of pharmacy staff n =?8-10) will be conducted to explore the acceptability of the intervention and feasibility of the study, training and study procedures.External pilot studyObjectives:Quantify the flow of participants (eligibility, recruitment and follow-up rate)Evaluate proposed recruitment, assessment and outcome measure collection methodsExamine the delivery of the enhanced support intervention in a community pharmacy setting (intervention uptake, retention and dose) to inform process evaluationProcess evaluation, using semi-structured interviews with participants across a range of socio-economic settings, and pharmacy staff to explore the acceptability of the ESI within community pharmacy, elements of the intervention that were considered useful (or not) and appropriateness of study proceduresDesign: Pilot randomised controlled trial, including a prospective economic and qualitative evaluationSetting: As abovePopulation: As aboveIntervention: As above with adaptations post feasibility studyComparator: Usual careSample size: 100 participantsOutcomes: Data will be used to estimate recruitment, intervention delivery and study completion rates as per objectives 2a-d. Definitive estimates of the effectiveness of ESI will not be made.Primary outcome: Depression severity (Patient Health Questionnaire 9) at four months.Secondary outcomes: Patient acceptance, uptake and attrition. ICD10 depression status, anxiety (GAD 7), health-related quality of life (SF-12v2) and health-state utility (EQ5D 3L) will be measured at four months.Economic evaluation: The incremental cost per QALY will be calculated from both the NHS and societal perspective.Process evaluation: Using mixed methods, potential mediators/moderators of the intervention, the acceptability (to participants and pharmacy staff), barriers and facilitators to the use of ESI in community pharmacy, and impact on usual practice will be examined. Semi-structured interviews with approximately 30 study participants, 20 pharmacy staff and eight GPs near participating pharmacies will be conducted.Trial registrationISRCTN: ISRCTN11290592Protocol version number: Version 4.1 (dated 16th January 2018)Study Sponsor Tees Esk and Wear Valleys NHS Foundation Trust.

Project description:ObjectivesTo determine the reporting quality of published randomised controlled trial (RCT) protocols before and after the Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) statement (2013), and any association with author, trial or journal factors.DesignMethodological study.Data sourcesMEDLINE, Embase and CENTRAL were electronically searched using optimised search strategies.Eligibility criteriaProtocols written for an RCT of living humans, published in full text in a peer-reviewed journal and published in the English language.Main outcomePrimary outcome was the overall proportion of checklist items which were adequately reported in RCT protocols published before and after the SPIRIT statement.Results300 RCT protocols were retrieved; 150 from the period immediately before the SPIRIT statement (9 July 2012 to 28 December 2012) and 150 from a recent period after the SPIRIT statement (25 January 2019 to 20 March 2019). 47.9% (95% CI, 46.5% to 49.3%) of checklist items were adequately reported in RCT protocols before the SPIRIT statement and 56.7% (95% CI, 54.9% to 58.5%) after the SPIRIT statement. This represents an 8.8% (95% CI, 6.6% to 11.1%; p<0.0001) mean improvement in the overall proportion of checklist items adequately reported since the SPIRIT statement. While 40% of individual checklist items had a significant improvement in adequate reporting after the SPIRIT statement, 11.3% had a significant deterioration and there were no RCT protocols in which all individual checklist items were complete. The factors associated with higher reporting quality of RCT protocols in multiple regression analysis were author expertise or experience in epidemiology or statistics, multicentre trials, longer protocol word length and publicly reported journal policy of compliance with the SPIRIT statement.ConclusionThe overall reporting quality of RCT protocols has significantly improved since the SPIRIT statement, although a substantial proportion of individual checklist items remain poorly reported. Continued and concerted efforts are required by journals, editors, reviewers and investigators to improve the completeness and transparency of RCT protocols.

Project description:BackgroundStandard treatment of Duchenne muscular dystrophy (DMD) includes regular physiotherapy. There are no data to show whether adding aquatic therapy (AT) to land-based exercises helps maintain motor function. We assessed the feasibility of recruiting and collecting data from boys with DMD in a parallel-group pilot randomised trial (primary objective), also assessing how intervention and trial procedures work.MethodsAmbulant boys with DMD aged 7-16 years established on steroids, with North Star Ambulatory Assessment (NSAA) score ≥8, who were able to complete a 10-m walk test without aids or assistance, were randomly allocated (1:1) to 6 months of either optimised land-based exercises 4 to 6 days/week, defined by local community physiotherapists, or the same 4 days/week plus AT 2 days/week. Those unable to commit to a programme, with >20% variation between NSAA scores 4 weeks apart, or contraindications to AT were excluded. The main outcome measures included feasibility of recruiting 40 participants in 6 months from six UK centres, clinical outcomes including NSAA, independent assessment of treatment optimisation, participant/therapist views on acceptability of intervention and research protocols, value of information (VoI) analysis and cost-impact analysis.ResultsOver 6 months, 348 boys were screened: most lived too far from centres or were enrolled in other trials; 12 (30% of the targets) were randomised to AT (n = 8) or control (n = 4). The mean change in NSAA at 6 months was -5.5 (SD 7.8) in the control arm and -2.8 (SD 4.1) in the AT arm. Harms included fatigue in two boys, pain in one. Physiotherapists and parents valued AT but believed it should be delivered in community settings. Randomisation was unattractive to families, who had already decided that AT was useful and who often preferred to enrol in drug studies. The AT prescription was considered to be optimised for three boys, with other boys given programmes that were too extensive and insufficiently focused. Recruitment was insufficient for VoI analysis.ConclusionsNeither a UK-based RCT of AT nor a twice weekly AT therapy delivered at tertiary centres is feasible. Our study will help in the optimisation of AT service provision and the design of future research.Trial registrationISRCTN41002956.

Project description:Cocaine use disorder (CUD) is a debilitating condition with no NICE-recommended medication or specific psychosocial interventions. In the United Kingdom (UK), general counselling (treatment-as-usual; TAU) is widely delivered, but has limited effectiveness. We tested the feasibility, safety and preliminary efficacy of a novel, adjunctive psychosocial intervention for CUD, called 'memory-focused cognitive therapy' (MFCT).We did a two-arm, external pilot randomised controlled trial at a specialist community National Health Service addictions clinic in London, UK. 30 adults (?18years), voluntarily seeking treatment for CUD (enrolled ?14days; all with moderate-to-severe DSM5 CUD), were individually randomised (1:1) to a control group (ongoing TAU; 3×90min CUD cognitive conceptualisation assessments; 2×30min cocaine-related cue-induction procedures; and 3×30min research follow-ups); or to an intervention group (ongoing TAU; 3×90min cognitive conceptualisation assessments; 2×30min cocaine-related cue-induction procedures; 5×120min, one-to-one, MFCT sessions [in 1week]; and 3×60min research follow-ups and MFCT-relapse prevention). The primary outcome was the total percentage score on the frequency version of the Craving Experiences Questionnaire (CEQ-F) at 1-month follow-up after the intensive intervention week (clinical endpoint; recall period past 2weeks; higher score indicating greater craving). Secondary outcomes at the 1-month follow-up were percentage days abstinent (PDA) from cocaine, and longest period (days) of continuous abstinence (LPA) in the prior 28days. Outcomes were analysed as an unadjusted group mean difference (with Hedge's g effect size [ES]) and a 95% Confidence Interval [CI] for the primary outcome and a 90% CI for the secondary outcomes. Exploratory, multivariable linear (primary outcome) and Poisson regression models (secondary outcomes), with sex, age, months of regular cocaine use, baseline outcome score, and group estimated the effectiveness of the intervention. The trial is registered with the ISCRTN (ISRCTN16462783).Between July 15, 2015, and November 27, 2016, 58 patients were assessed for eligibility and 30 participants were randomised (14 to the control group and 16 to the intervention). With outcome data collected for all participants at the endpoint, the intervention group mean CEQ-F score (14·77; SD 21·47) was lower than the control group mean (51·75; SD 22·72); ES -1·62; 95% CI -2·45 to -0·80. MFCT was associated with more cocaine abstinence in the intervention group (PDA 85·94; SD 18·96) than the control group (PDA 54·59; SD 30·29); ES 1·19; 90% CI 0·54 to 1·84. There was also greater maximum abstinence in the intervention group (LPA 15·69; SD 10·10) than the control group (6·00; SD 7·36); ES 1·06; 90% CI 0·41 to 1·70. Exploratory, confounder-adjusted regression models for this preliminary effect supported the treatment association for reduced craving experiences (CEQ-F Coef. -28·25; 95% CI -45·15 to -11·35); more abstinence (PDA Incidence Rate Ratio [IRR] 1·56; 95% CI 1·31 to 1·88); and greater maximum abstinence (LPA IRR 2·56; 95% CI 1·96 to 3·35), although relative weak unmeasured confounding could overturn these model-adjusted exposure-outcome associations. There were four serious adverse events (among three participants). None were judged related to study procedures or interventions.In this first external pilot randomised controlled trial of MFCT for CUD, we have shown that the intervention and control procedures and acceptable feasible and safe, and report preliminary evidence that MFCT is associated with reduced craving and increased abstinence. These findings support progression to a substantive trial.UK National Institute for Health Research, Biomedical Research Centre.

Project description:BACKGROUND: Acupuncture is increasingly being used for many conditions including chronic neck pain. However the evidence remains inconclusive, indicating the need for further well-designed research. The aim of this study was to conduct a pilot randomised controlled parallel arm trial, to establish key features required for the design and implementation of a large-scale trial on acupuncture for chronic neck pain. METHODS: Patients whose GPs had diagnosed neck pain were recruited from one general practice, and randomised to receive usual GP care only, or acupuncture (up to 10 treatments over 3 months) as an adjunctive treatment to usual GP care. The primary outcome measure was the Northwick Park Neck Pain Questionnaire (NPQ) at 3 months. The primary analysis was to determine the sample size for the full scale study. RESULTS: Of the 227 patients with neck pain identified from the GP database, 28 (12.3%) consenting patients were eligible to participate in the pilot and 24 (10.5%) were recruited to the trial. Ten patients were randomised to acupuncture, receiving an average of eight treatments from one of four acupuncturists, and 14 were randomised to usual GP care alone. The sample size for the full scale trial was calculated from a clinically meaningful difference of 5% on the NPQ and, from this pilot, an adjusted standard deviation of 15.3%. Assuming 90% power at the 5% significance level, a sample size of 229 would be required in each arm in a large-scale trial when allowing for a loss to follow-up rate of 14%. In order to achieve this sample, one would need to identify patients from databases of GP practices with a total population of 230,000 patients, or approximately 15 GP practices roughly equal in size to the one involved in this study (i.e. 15,694 patients). CONCLUSION: This pilot study has allowed a number of recommendations to be made to facilitate the design of a large-scale trial, which in turn will help to clarify the existing evidence base on acupuncture for neck pain.

Project description:IntroductionFamily-based physical activity (PA) interventions present a promising avenue to promote children's activity; however, high-quality experimental research is lacking. This paper describes the protocol for the FRESH (Families Reporting Every Step to Health) pilot trial, a child-led family-based PA intervention delivered online.Methods and analysisFRESH is a three-armed, parallel-group, randomised controlled pilot trial using a 1:1:1 allocation ratio with follow-up assessments at 8 and 52 weeks postbaseline. Families will be eligible if a minimum of one child in school Years 3-6 (aged 7-11 years) and at least one adult responsible for that child are willing to participate. Family members can take part in the intervention irrespective of their participation in the accompanying evaluation and vice versa.Following baseline assessment, families will be randomly allocated to one of three arms: (1) FRESH; (2) pedometer-only or (3) no-intervention control. All family members in the pedometer-only and FRESH arms receive pedometers and generic PA promotion information. FRESH families additionally receive access to the intervention website; allowing participants to select step challenges to 'travel' to target cities around the world, log steps and track progress as they virtually globetrot. Control families will receive no treatment. All family members will be eligible to participate in the evaluation with two follow-ups (8 and 52 weeks). Physical (eg, fitness and blood pressure), psychosocial (eg, social support) and behavioural (eg, objectively measured family PA) measures will be collected at each time point. At 8-week follow-up, a mixed methods process evaluation will be conducted (questionnaires and family focus groups) assessing acceptability of the intervention and evaluation. FRESH families' website engagement will also be explored.Ethics and disseminationThis study received ethical approval from the Ethics Committee for the School of the Humanities and Social Sciences at the University of Cambridge. Findings will be disseminated via peer-reviewed publications, conferences and to participating families.Trial registration numberISRCTN12789422.

Project description:BackgroundBefore considering whether to use a multivariable (diagnostic or prognostic) prediction model, it is essential that its performance be evaluated in data that were not used to develop the model (referred to as external validation). We critically appraised the methodological conduct and reporting of external validation studies of multivariable prediction models.MethodsWe conducted a systematic review of articles describing some form of external validation of one or more multivariable prediction models indexed in PubMed core clinical journals published in 2010. Study data were extracted in duplicate on design, sample size, handling of missing data, reference to the original study developing the prediction models and predictive performance measures.Results11,826 articles were identified and 78 were included for full review, which described the evaluation of 120 prediction models. in participant data that were not used to develop the model. Thirty-three articles described both the development of a prediction model and an evaluation of its performance on a separate dataset, and 45 articles described only the evaluation of an existing published prediction model on another dataset. Fifty-seven percent of the prediction models were presented and evaluated as simplified scoring systems. Sixteen percent of articles failed to report the number of outcome events in the validation datasets. Fifty-four percent of studies made no explicit mention of missing data. Sixty-seven percent did not report evaluating model calibration whilst most studies evaluated model discrimination. It was often unclear whether the reported performance measures were for the full regression model or for the simplified models.ConclusionsThe vast majority of studies describing some form of external validation of a multivariable prediction model were poorly reported with key details frequently not presented. The validation studies were characterised by poor design, inappropriate handling and acknowledgement of missing data and one of the most key performance measures of prediction models i.e. calibration often omitted from the publication. It may therefore not be surprising that an overwhelming majority of developed prediction models are not used in practice, when there is a dearth of well-conducted and clearly reported (external validation) studies describing their performance on independent participant data.