Dataset Information

"Vicarious thinking" was a key driver of score change in Delphi surveys for COS development and is facilitated by feedback of results.

ABSTRACT:

Objective

The objectives of this nested study were to (1) assess whether changes in scores between rounds altered the final degree of consensus achieved in three Delphi surveys conducted as part of COS development projects (anal, gastric, and prostate cancer), and (2) explore participants' reasons for changing scores between rounds.

Study design and setting

All Delphi surveys were conducted online using DelphiManager software and included healthcare professionals and participating patients. Participants were invited to give a free-text reason whenever they changed their score across an important threshold on a 1-9 Likert scale (1-3 not important, 4-5 important, 7-9 critically important). Reasons for score change were coded by four researchers independently using an inductive-iterative approach.

Results

In all three Delphi surveys, the number of outcomes reaching criteria for consensus was greater in R2 than R1. Twelve themes and 23 subthemes emerged from 2298 discrete reasons given for score change. The most common reasons for the change were "time to reflect" (482 responses, 23%) and vicarious thinking (424, 21%), with 68% (291) of vicarious thinking attributed to seeing other participant's scores.

Conclusion

Our findings support conducting a Delphi survey over the use of a single questionnaire where building consensus is the objective. Time to reflect and vicarious thinking, facilitated by seeing other participant's scores, were important drivers of score change. How results are presented to participants between rounds and the duration of and time between rounds in a Delphi survey may, therefore, influence the results and should be clearly reported.

SUBMITTER: Fish R

PROVIDER: S-EPMC7716748 | biostudies-literature | 2020 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

"Vicarious thinking" was a key driver of score change in Delphi surveys for COS development and is facilitated by feedback of results.

Fish Rebecca R MacLennan Steven S Alkhaffaf Bilal B Williamson Paula R PR

Journal of clinical epidemiology 20201001

<h4>Objective</h4>The objectives of this nested study were to (1) assess whether changes in scores between rounds altered the final degree of consensus achieved in three Delphi surveys conducted as part of COS development projects (anal, gastric, and prostate cancer), and (2) explore participants' reasons for changing scores between rounds.<h4>Study design and setting</h4>All Delphi surveys were conducted online using DelphiManager software and included healthcare professionals and participating ...[more]

PMID: 33011214

Similar Datasets

Project description:It is well known that some survey respondents reduce the effort they invest in answering questions by taking mental shortcuts - survey satisficing. This is a concern because such shortcuts can reduce the quality of responses and, potentially, the accuracy of survey estimates. This article explores "speeding," an extreme type of satisficing, which we define as answering so quickly that respondents could not have given much, if any, thought to their answers. To reduce speeding among online respondents we implemented an interactive prompting technique. When respondents answered faster than a minimal response time threshold, they received a message encouraging them to answer carefully and take their time. Across six web survey experiments, this prompting technique reduced speeding on subsequent questions compared to a no prompt control. Prompting slowed response times whether the speeding that triggered the prompt occurred early or late in the questionnaire, in the first or later waves of a longitudinal survey, among respondents recruited from non-probability or probability panels, or whether the prompt was delivered on only the first or on all speeding episodes. In addition to reducing speeding, the prompts increased response accuracy on simple arithmetic questions for a key subgroup. Prompting also reduced later straightlining in one experiment, suggesting the benefits may generalize to other types of mental shortcuts. Although the prompting could have annoyed respondents, it was not accompanied by a noticeable increase in breakoffs. As an alternative technique, respondents in one experiment were asked to explicitly commit to responding carefully. This global approach complemented the more local, interactive prompting technique on several measures. Taken together, these results suggest that interactive interventions of this sort may be useful for increasing respondents' conscientiousness in online questionnaires, even though these questionnaires are self-administered.

Dataset Information

"Vicarious thinking" was a key driver of score change in Delphi surveys for COS development and is facilitated by feedback of results.

Objective

Study design and setting

Results

Conclusion

Publications

"Vicarious thinking" was a key driver of score change in Delphi surveys for COS development and is facilitated by feedback of results.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets