Discrete choice experiments or best-worst scaling? A qualitative study to determine the suitability of preference elicitation tasks in research with children and young people.
Ontology highlight
ABSTRACT: Background
Ordinal tasks are increasingly used to explore preferences for health states. This study aimed to determine the suitability of two ordinal preference elicitation tasks (discrete choice experiments (DCE) and best-worst scaling (BWS)) for use with children and young people to generate health state utility values. The study explored children's understanding, the relationship between their age and level of understanding, and how many tasks they felt they could complete.Methods
Children aged 11-16 years were recruited from a secondary school in South Yorkshire, UK. Participants were asked to 'think aloud' as they completed a computer-based survey that contained both DCE and BWS tasks relating to dental caries (tooth decay) health states. Health states involved descriptions of the impact of tooth decay on children's daily lives. One-to-one semi-structured interviews were then held with participants, with use of a topic guide. Qualitative data were transcribed verbatim and analysed thematically.Results
A total of 33 children (12 male, 21 female) participated, comprising 5-6 children from each school year group. Children expressed a preference for BWS and demonstrated a better understanding of these tasks than DCE. There was no clear relationship between children's level of understanding and age. Children felt they could manage between 8 and 10 BWS tasks comfortably.Conclusion
This study suggests that BWS tasks are the most appropriate type of preference elicitation task to value health states for children and young people aged 11-16 years to complete.
SUBMITTER: Rogers HJ
PROVIDER: S-EPMC7947050 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:This study undertook a head-to-head comparison of best-worst, best-best and ranking discrete choice experiments (DCEs) to help decide which method to use if moving beyond traditional single-best DCEs. Respondents were randomized to one of three preference elicitation methods. Rank-ordered (exploded) mixed logit models and respondent-reported data were used to compare methods and first and second choices. First choices differed from second choices and preferences differed between elicitation methods, even beyond scale and scale dynamics. First choices of best-worst had good choice consistency, scale dynamics and statistical efficiency, but this method's second choices performed worst. Ranking performed best on respondent-reported difficulty and preference; best-best's second choices on statistical efficiency. All three preference elicitation methods improve efficiency of data collection relative to using first choices only. However, differences in preferences between first and second choices challenge moving beyond single-best DCE. If nevertheless doing so, best-best and ranking are preferred over best-worst DCE.
| S-EPMC9826006 | biostudies-literature
Project description:Given the increasing number of genetic tests available, decisions have to be made on how to allocate limited health-care resources to them. Different criteria have been proposed to guide priority setting. However, their relative importance is unclear. Discrete-choice experiments (DCEs) and best-worst scaling experiments (BWSs) are methods used to identify and weight various criteria that influence orders of priority. This study tests whether these preference eliciting techniques can be used for prioritising genetic tests and compares the empirical findings resulting from these two approaches. Pilot DCE and BWS questionnaires were developed for the same criteria: prevalence, severity, clinical utility, alternatives to genetic testing available, infrastructure for testing and care established, and urgency of care. Interview-style experiments were carried out among different genetics professionals (mainly clinical geneticists, researchers and biologists). A total of 31 respondents completed the DCE and 26 completed the BWS experiment. Weights for the levels of the six attributes were estimated by conditional logit models. Although the results derived from the DCE and BWS experiments differed in detail, we found similar valuation patterns in the DCE and BWS experiments. The respondents attached greatest value to tests with high clinical utility (defined by the availability of treatments that reduce mortality and morbidity) and to testing for highly prevalent conditions. The findings from this study exemplify how decision makers can use quantitative preference eliciting methods to measure aggregated preferences in order to prioritise alternative clinical interventions. Further research is necessary to confirm the survey results.
| S-EPMC3798841 | biostudies-other
Project description:A common goal in psychological research is the measurement of subjective impressions, such as first impressions of faces. These impressions are commonly measured using Likert ratings. Although these ratings are simple to administer, they are associated with response issues that can limit reliability. Here we examine best-worst scaling (BWS), a forced-choice method, as a potential alternative to Likert ratings for measuring participants' facial first impressions. We find that at the group level, BWS scores correlated almost perfectly with Likert scores, indicating that the two methods measure the same impressions. However, at the individual participant level BWS outperforms Likert ratings, both in terms of ability to predict preferences in a third task, and in terms of test-retest reliability. These benefits highlight the power of BWS, particularly for use in individual differences research.
| S-EPMC6757072 | biostudies-literature
Project description:IntroductionBest-worst scaling (BWS) is becoming increasingly popular to elicit preferences in health care. However, little is known about current practice and trends in the use of BWS in health care. This study aimed to identify, review and critically appraise BWS in health care, and to identify trends over time in key aspects of BWS.MethodsA systematic review was conducted, using Medline (via Pubmed) and EMBASE to identify all English-language BWS studies published up until April 2016. Using a predefined extraction form, two reviewers independently selected articles and critically appraised the study quality, using the Purpose, Respondents, Explanation, Findings, Significance (PREFS) checklist. Trends over time periods (?2010, 2011, 2012, 2013, 2014 and 2015) were assessed further.ResultsA total of 62 BWS studies were identified, of which 26 were BWS object case studies, 29 were BWS profile case studies and seven were BWS multi-profile case studies. About two thirds of the studies were performed in the last 2 years. Decreasing sample sizes and decreasing numbers of factors in BWS object case studies, as well as use of less complicated analytical methods, were observed in recent studies. The quality of the BWS studies was generally acceptable according to the PREFS checklist, except that most studies did not indicate whether the responders were similar to the non-responders.ConclusionUse of BWS object case and BWS profile case has drastically increased in health care, especially in the last 2 years. In contrast with previous discrete-choice experiment reviews, there is increasing use of less sophisticated analytical methods.
| S-EPMC5110583 | biostudies-literature
Project description:Epilepsy is a common, serious condition. Fortunately, seizure risk decreases with increasing seizure-free time on antiseizure medications (ASMs). Eventually, patients may consider whether to stop ASMs, which requires weighing treatment benefit versus burden. We developed a questionnaire to quantify patient preferences relevant to ASM decision-making. Respondents rated how concerning they would finding relevant items (e.g., seizure risks, side effects, cost) on a Visual Analogue Scale (VAS, 0-100) and then repeatedly chose the most and least concerning item from subsets (best-worst scaling, BWS). We pretested with neurologists, then recruited adults with epilepsy who were seizure-free at least one year. Primary outcomes were recruitment rate, and qualitative and Likert-based feedback. Secondary outcomes included VAS ratings and best-minus-worst scores. Thirty-one of 60 (52%) contacted patients completed the study. Most patients felt VAS questions were clear (28; 90%), easy to use (27; 87%), and assessed preferences well (25; 83%). Corresponding results for BWS questions were 27 (87%), 29 (97%), and 23 (77%). Physicians suggested adding a 'warmup' question showing a completed example and simplifying terminology. Patients suggested ways to clarify instructions. Cost, inconvenience of taking medication, and laboratory monitoring were the least concerning items. Cognitive side effects and a 50% seizure risk in the next year were the most concerning items. Twelve (39%) of patients made at least one 'inconsistent choice' for example ranking a higher seizure risk as lower concern compared with a lower seizure risk, though 'inconsistent choices' represented only 3% of all question blocks. Our recruitment rate was favorable, most patients agreed the survey was clear, and we describe areas for improvement. 'Inconsistent' responses may lead us to collapse seizure probability items into a single 'seizure' category. Evidence regarding how patients weigh benefits and harms may inform care and guideline development.
| S-EPMC9983827 | biostudies-literature
Project description:As exome sequencing expands as a diagnostic tool, patients and providers have voiced concerns about communicating the breadth and scope of potential results when obtaining informed consent. This study aimed to understand how genetic counselors prioritize essential components of the informed consent process and whether counselor factors influence these decisions. Development of a best-worst scaling experiment was informed by a systematic literature review and two focus groups. In all, 11 choice sets were created using a balanced incomplete block design, where participants selected the most and least important object in each set. Mean best-worst (BW) scores were calculated to summarize the relative importance of each object, and mediation analyses assessed whether responses were associated with genetic counselor factors and attitudes. In all, 342 members of the National Society of Genetic Counselors completed the online survey. Ranking of BW scores suggests that participants prioritize collaborative decision-making, assessing understanding and managing expectations, with the least emphasis placed on discussing technological complexities. Stratified analyses found that counselors more experienced with obtaining informed consent for exome sequencing and those reporting higher perceptions of patients' ability to manage information rated discussing variants of uncertain significance as significantly more important (p < .05). Our results suggest that genetic counselors report intentions to prioritize individual patient needs when obtaining informed consent for exome sequencing. Professional characteristics and attitudes may influence preemptive discussion of uncertain results.
| S-EPMC7422696 | biostudies-literature
Project description:Health-related quality of life (HRQOL) instruments are sometimes used without explicit understanding of which HRQOL domains are important to a given population. In this study, we sought to elicit an importance hierarchy among 27 HRQOL domains (derived from the general population) via a best-worst scaling survey of the population in Singapore, and to determine whether these domains were consistently valued across gender, age, ethnicity, and presence of chronic illnesses. We conducted a community-based study that sampled participants with quotas for gender, ethnicity, age, presence of chronic illness, and interview language. For the best-worst scaling exercise, we constructed comparison sets according to a balanced incomplete block design resulting in 13 sets of questions, each with nine choice tasks. Each task involved three HRQOL domains from which participants identified the most and least important domain. We performed a standard analysis of best-worst object scaling design (Case 1) using simple summary statistics; 603 residents participated in the survey. The three most important domains of health were: "the ability to take care of self without help from others" (best-worst score (BWS): 636), "healing and resistance to illness" (BWS: 461), and "having good relationships with family, friends, and others" (BWS: 373). The 10 top-ranked domains included physical, mental, and social health. The three least important domains were: "having a satisfying sex life" (BWS: -803), "having normal physical appearance" (BWS: -461), and "interacting with others (talking, shared activities, etc.)" (BWS: -444). Generally, top-ranked domains were consistently valued across gender, age, ethnicity, and presence of chronic illness. We conclude that the 10 top-ranked domains reflect physical, mental, and social dimensions of well-being suggesting that the sampled population's views on health are consistent with the World Health Organization's definition of health, "a state of complete physical, mental and social well-being and not merely the absence of disease or infirmity".
| S-EPMC5805165 | biostudies-literature
Project description:OBJECTIVE:To quantitatively evaluate parental preferences for the various treatments for vesicoureteral reflux using crowd-sourced best-worst scaling, a novel technique in urologic preference estimation. METHODS:Preference data were collected from a community sample of parents via 2 best-worst scaling survey instruments published to Amazon's Mechanical Turk online community. Attributes and attribute levels were selected following extensive review of the reflux literature. Respondents completed an object case best-worst scaling exercise to prioritize general aspects of reflux treatments and multiprofile case best-worst scaling to elicit their preferences for the specific differences in reflux treatments. Data were analyzed using multinomial logistic regression. Results from the object-case provided probability scaled values (PSV) that reflected the order of importance of attributes. RESULTS:We analyzed data for 248 and 228 respondents for object and multiprofile case BWS, respectively. When prioritizing general aspects of reflux treatment, effectiveness (PSV?=?20.37), risk of future urinary tract infection (PSV?=?14.85), and complication rate (PSV?=?14.55) were most important to parents. Societal cost (PSV?=?1.41), length of hospitalization (PSV?=?1.09), and cosmesis (PSV?=?0.91) were least important. Parents perceived no difference in preference for the cosmetic outcome of open vs minimally invasive surgery (P?=?.791). Bundling attribute preference weights, parents in our study would choose open surgery 74.9% of the time. CONCLUSION:High treatment effectiveness was the most important and preferred attribute to parents. Alternatively, cost and cosmesis were among the least important. Our findings serve to inform shared parent-physician decision-making for vesicoureteral reflux.
| S-EPMC6525647 | biostudies-literature
Project description:Background and objectiveBest-worst scaling is a theory-driven method that can be used to prioritize objects in health. We sought to characterize all studies of best-worst scaling to prioritize objects in health, to assess trends of using best-worst scaling in prioritization over time, and to assess the relationship between a legacy measure of quality (PREFS) and a novel assessment of subjective quality and policy relevance.MethodsA systematic review identified studies published through to the end of 2021 that applied best-worst scaling to study priorities in health (PROSPERO CRD42020209745), updating a prior review published in 2016. The PubMed, EBSCOhost, Embase, Scopus, APA PsychInfo, Web of Science, and Google Scholar databases were used and were supplemented by a hand search. Data describing the application, development, design, administration/analysis, quality, and policy relevance were summarized and we tested for trends by comparing articles before and after 1 January, 2017. Multivariate statistics were then used to assess the relationships between PREFS, subjective quality, policy relevance, and other possible indicators.ResultsFrom a total of 2826 unique papers identified, 165 best-worst scaling studies were included in this review. Applications of best-worst scaling to study priorities in health have continued to grow (p < 0.01) and are now used in all regions of the world, most often to study the priorities of patients/consumers (67%). Several key trends can be observed over time: increased use of pretesting (p < 0.05); increased use of online administration (p < 0.01), and decreased use of paper self-administered surveys (p = 0.02); increased use of heterogeneity analysis (p = 0.02); an increase in having a clearly stated purpose (p < 0.01); and a decrease in comparing respondents to non-respondents (p = 0.01). The average sample size has more than doubled, from 228 to 472 respondents, but formal sample size justifications remain low (5.3%) and unchanged over time (p = 0.68). While the average PREFS score remained unchanged at 3.1/5, both subjective quality and policy relevance trended up, but changes were not statistically significant (p = 0.06 and p = 0.13). Most of the variation in subjective quality was driven by PREFS (R2 = 0.42), but it was also positively assosciated with policy relevance, heterogeneity analysis, and using a balanced incomplete block design, and was negatively associated with not using developmental methods and an increasing sample size.ConclusionsUsing best-worst scaling to prioritize objects is now commonly used around the world to assess the priorities of patients and other stakeholders in health. Best practices are clearly emerging for best-worst scaling. Although legacy measures (PREFS) to measure study quality are reasonable, there may need to be new tools to assess both study quality and policy relevance.
| S-EPMC9363399 | biostudies-literature
Project description:ContextAlthough patients with acute myeloid leukemia (AML) experience significant toxicities and poor outcomes, few studies have quantified patients' experience.MethodsA community-centered approach was used to develop an AML-specific best-worst scaling (BWS) instrument involving 13 items in four domains (psychological, physical, decision-making, and treatment delivery) to quantify patient worry. A survey of patients and caregivers was conducted using the instrument. Data were analyzed using conditional logistic regression.ResultsThe survey was completed by 832 patients and 237 caregivers. Patients were predominantly white (88%), married/partnered (72%), and in remission (95%). The median age was 55 years (range: 19-87). Median time since diagnosis was 8 years (range: 1-40). Patients worried most about "the possibility of dying from AML" (BWS score = 15.5, confidence interval [CI] [14.2-16.7]) and "long-term side effects of treatments" (14.0, CI [12.9-15.2]). Patients found these items more than twice as worrisome as all items within the domains of care delivery and decision-making. Patients were least worried about "communicating openly with doctors" (2.50, CI [1.97-3.04]) and "having access to the best medical care" (3.90, CI [3.28-4.61]). Caregiver reports were highly correlated to patients' (Spearman's ρ = 0.89) though noted significantly more worry about the possibility of dying and spending time in the hospital.ConclusionThis large convenience sample demonstrates that AML patients have two principal worries: dying from their disease and suffering long-term side effects from treatment. To better foster patient-centered care, therapeutic decision-making and drug development should reflect the importance of both potential outcomes. Further work should explore interventions to address these worries.
| S-EPMC10246445 | biostudies-literature