Dataset Information

Multi-Round compared to Real-Time Delphi for consensus in core outcome set (COS) development: a randomised trial.

ABSTRACT:

Background

The Delphi method is used in a wide variety of settings as a method of building consensus on important issues. Traditionally, the Delphi method uses multiple rounds of a survey to allow for feedback of other participants' survey responses in between rounds. By informing participants about how others answer a question or prioritise specific topics, it allows for diverse opinions to inform the consensus process. For this reason, the Delphi method is popular as a consensus building approach in developing core outcome sets (COS), i.e. the minimum agreed set of standardised outcomes that should be measured and reported in studies on a specific health condition. In a COS setting, participants prioritise the importance of outcomes for inclusion in a COS. This usually involves participating in multiple rounds of a survey that can span several weeks or months. Challenges with participant retention have been highlighted in previous COS. We will compare a three-round with a Real-Time Delphi approach on prioritised outcomes. This trial is embedded within the COHESION study which is developing a COS for interventions treating neonatal encephalopathy.

Methods

One hundred and eighty stakeholders (parents/caregivers of infants diagnosed and treated with neonatal encephalopathy, healthcare providers and researchers) will be randomised using stratified randomisation to take part in either the Multi-Round or Real-Time Delphi. Stakeholders will rate the importance of the same set of outcomes in both arms. We will compare the prioritised outcomes at the end of both surveys as well as other parameters such as feedback, initial condition and iteration effects.

Discussion

This trial will provide evidence to inform decisions on the use of Multi-Round compared to Real-Time Delphi survey methods.

Trial registration

NCT04471103 . Registered on 14 July 2020.

SUBMITTER: Quirke FA

PROVIDER: S-EPMC7885346 | biostudies-literature | 2021 Feb

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Multi-Round compared to Real-Time Delphi for consensus in core outcome set (COS) development: a randomised trial.

Quirke Fiona A FA Healy Patricia P Bhraonáin Elaine Ní EN Daly Mandy M Biesty Linda L Hurley Tim T Walker Karen K Meher Shireen S Haas David M DM Bloomfield Frank H FH Kirkham Jamie J JJ Molloy Eleanor J EJ Devane Declan D

Trials 20210215 1

<h4>Background</h4>The Delphi method is used in a wide variety of settings as a method of building consensus on important issues. Traditionally, the Delphi method uses multiple rounds of a survey to allow for feedback of other participants' survey responses in between rounds. By informing participants about how others answer a question or prioritise specific topics, it allows for diverse opinions to inform the consensus process. For this reason, the Delphi method is popular as a consensus buildi ...[more]

PMID: 33588938

Similar Datasets

Project description:BackgroundDelphi surveys are commonly used to prioritise critical outcomes in core outcome set (COS) development. This trial aims to compare a three-round (Multi-Round) Delphi (MRD) with a Real-Time Delphi (RTD) in the prioritisation of outcomes for inclusion in a COS for neonatal encephalopathy treatments and explore whether 'feedback', 'iteration', and 'initial condition' effects may occur in the two survey methods.MethodsWe recruited 269 participants (parents/caregivers, healthcare providers and researchers/academics) of which 222 were randomised to either the MRD or the RTD. We investigated the outcomes prioritised in each survey and the 'feedback', 'iteration', and 'initial condition' effects to identify differences between the two survey methods.ResultsIn the RTD, n = 92 participants (83%) fully completed the survey. In the MRD, n = 60 participants (54%) completed all three rounds. Of the 92 outcomes presented, 26 (28%) were prioritised differently between the RTD and MRD. Significantly fewer participants amended their scores when shown stakeholder responses in the RTD compared to the MRD ('feedback effect'). The 'iteration effect' analysis found most experts appeared satisfied with their initial ratings in the RTD and did not amend their scores following stakeholder response feedback. Where they did amend their scores, ratings were amended substantially, suggesting greater convergence. Variance in scores reduced with subsequent rounds of the MRD ('iteration effect'). Whilst most participants did not change their initial scores in the RTD, of those that did, later recruits tended to align their final score more closely to the group mean final score than earlier recruits (an 'initial condition' effect).ConclusionThe feedback effect differed between the two Delphi methods but the magnitude of this difference was small and likely due to the large number of observations rather than because of a meaningfully large difference. It did not appear to be advantageous to require participants to engage in three rounds of a survey due to the low change in scores. Larger drop-out through successive rounds in the MRD, together with a lesser convergence of scores and longer time to completion, indicate considerable benefits of the RTD approach.Trial registrationNCT04471103. Registered on 14 July 2020.

Project description:BackgroundAs the development of core outcome sets (COS) increases, guidance for developing and reporting high-quality COS continues to evolve; however, a number of methodological uncertainties still remain. The objectives of this study were: (1) to explore the impact of including patient interviews in developing a COS, (2) to examine the impact of using a 5-point versus a 9-point rating scale during Delphi consensus methods on outcome selection and (3) to inform and contribute to COS development methodology by advancing the evidence base on COS development techniques.MethodsSemi-structured patient interviews and a nested randomised controlled parallel group trial as part of the Pelvic Girdle Pain Core Outcome Set project (PGP-COS). Patient interviews, as an adjunct to a systematic review of outcomes reported in previous studies, were undertaken to identify preliminary outcomes for including in a Delphi consensus survey. In the Delphi survey, participants were randomised (1:1) to a 5-point or 9-point rating scale for rating the importance of the list of preliminary outcomes.ResultsFour of the eight patient interview derived outcomes were included in the preliminary COS, however, none of these outcomes were included in the final PGP-COS. The 5-point rating scale resulted in twice as many outcomes reaching consensus after the 3-round Delphi survey compared to the 9-point scale. Consensus on all five outcomes included in the final PGP-COS was achieved by participants allocated the 5-point rating scale, whereas consensus on four of these was achieved by those using the 9-point scale.ConclusionsUsing patient interviews to identify preliminary outcomes as an adjunct to conducting a systematic review of outcomes measured in the literature did not appear to influence outcome selection in developing the COS in this study. The use of different rating scales in a Delphi survey, however, did appear to impact on outcome selection. The 5-point scale demonstrated greater congruency than the 9-point scale with the outcomes included in the final PGP-COS. Future research to substantiate our findings and to explore the impact of other rating scales on outcome selection during COS development, however, is warranted.

Project description:Introduction: Outcome measures are key to tailor rehabilitation goals to the stroke patient's individual needs and to monitor poststroke recovery. The large number of available outcome measures leads to high variability in clinical use. Currently, an internationally agreed core set of motor outcome measures for clinical application is lacking. Therefore, the goal was to develop such a set to serve as a quality standard in clinical motor rehabilitation poststroke. Methods: Outcome measures for the upper and lower extremities, and activities of daily living (ADL)/stroke-specific outcomes were identified and presented to stroke rehabilitation experts in an electronic Delphi study. In round 1, clinical feasibility and relevance of the outcome measures were rated on a 7-point Likert scale. In round 2, those rated at least as "relevant" and "feasible" were ranked within the body functions, activities, and participation domains of the International Classification of Functioning, Disability, and Health (ICF). Furthermore, measurement time points poststroke were indicated. In round 3, answers were reviewed in reference to overall results to reach final consensus. Results: In total, 119 outcome measures were presented to 33 experts from 18 countries. The recommended core set includes the Fugl-Meyer Motor Assessment and Action Research Arm Test for the upper extremity section; the Fugl-Meyer Motor Assessment, 10-m Walk Test, Timed-Up-and-Go, and Berg Balance Scale for the lower extremity section; and the National Institutes of Health Stroke Scale, and Barthel Index or Functional Independence Measure for the ADL/stroke-specific section. The Stroke Impact Scale was recommended spanning all ICF domains. Recommended measurement time points are days 2 ± 1 and 7; weeks 2, 4, and 12; 6 months poststroke and every following 6th month. Discussion and Conclusion: Agreement was found upon a set of nine outcome measures for application in clinical motor rehabilitation poststroke, with seven measurement time points following the stages of poststroke recovery. This core set was specifically developed for clinical practice and distinguishes itself from initiatives for stroke rehabilitation research. The next challenge is to implement this clinical core set across the full stroke care continuum with the aim to improve the transparency, comparability, and quality of stroke rehabilitation at a regional, national, and international level.

Project description:The use of core outcome sets (COS) ensures that researchers measure and report those outcomes that are most likely to be relevant to users of their research. Several hundred COS projects have been systematically identified to date, but there has been no formal quality assessment of these studies. The Core Outcome Set-STAndards for Development (COS-STAD) project aimed to identify minimum standards for the design of a COS study agreed upon by an international group, while other specific guidance exists for the final reporting of COS development studies (Core Outcome Set-STAndards for Reporting [COS-STAR]).An international group of experienced COS developers, methodologists, journal editors, potential users of COS (clinical trialists, systematic reviewers, and clinical guideline developers), and patient representatives produced the COS-STAD recommendations to help improve the quality of COS development and support the assessment of whether a COS had been developed using a reasonable approach. An open survey of experts generated an initial list of items, which was refined by a 2-round Delphi survey involving nearly 250 participants representing key stakeholder groups. Participants assigned importance ratings for each item using a 1-9 scale. Consensus that an item should be included in the set of minimum standards was defined as at least 70% of the voting participants from each stakeholder group providing a score between 7 and 9. The Delphi survey was followed by a consensus discussion with the study management group representing multiple stakeholder groups. COS-STAD contains 11 minimum standards that are the minimum design recommendations for all COS development projects. The recommendations focus on 3 key domains: the scope, the stakeholders, and the consensus process.The COS-STAD project has established 11 minimum standards to be followed by COS developers when planning their projects and by users when deciding whether a COS has been developed using reasonable methods.

Dataset Information

Multi-Round compared to Real-Time Delphi for consensus in core outcome set (COS) development: a randomised trial.

Background

Methods

Discussion

Trial registration

Publications

Multi-Round compared to Real-Time Delphi for consensus in core outcome set (COS) development: a randomised trial.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets