Dataset Information

Investigating the impact of design characteristics on statistical efficiency within discrete choice experiments: A systematic survey.

ABSTRACT:

Objectives

This study reviews simulation studies of discrete choice experiments to determine (i) how survey design features affect statistical efficiency, (ii) and to appraise their reporting quality.

Outcomes

Statistical efficiency was measured using relative design (D-) efficiency, D-optimality, or D-error.

Methods

For this systematic survey, we searched Journal Storage (JSTOR), Since Direct, PubMed, and OVID which included a search within EMBASE. Searches were conducted up to year 2016 for simulation studies investigating the impact of DCE design features on statistical efficiency. Studies were screened and data were extracted independently and in duplicate. Results for each included study were summarized by design characteristic. Previously developed criteria for reporting quality of simulation studies were also adapted and applied to each included study.

Results

Of 371 potentially relevant studies, 9 were found to be eligible, with several varying in study objectives. Statistical efficiency improved when increasing the number of choice tasks or alternatives; decreasing the number of attributes, attribute levels; using an unrestricted continuous "manipulator" attribute; using model-based approaches with covariates incorporating response behaviour; using sampling approaches that incorporate previous knowledge of response behaviour; incorporating heterogeneity in a model-based design; correctly specifying Bayesian priors; minimizing parameter prior variances; and using an appropriate method to create the DCE design for the research question. The simulation studies performed well in terms of reporting quality. Improvement is needed in regards to clearly specifying study objectives, number of failures, random number generators, starting seeds, and the software used.

Conclusion

These results identify the best approaches to structure a DCE. An investigator can manipulate design characteristics to help reduce response burden and increase statistical efficiency. Since studies varied in their objectives, conclusions were made on several design characteristics, however, the validity of each conclusion was limited. Further research should be conducted to explore all conclusions in various design settings and scenarios. Additional reviews to explore other statistical efficiency outcomes and databases can also be performed to enhance the conclusions identified from this review.

SUBMITTER: Vanniyasingam T

PROVIDER: S-EPMC5898574 | biostudies-literature | 2018 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Investigating the impact of design characteristics on statistical efficiency within discrete choice experiments: A systematic survey.

Vanniyasingam Thuva T Daly Caitlin C Jin Xuejing X Zhang Yuan Y Foster Gary G Cunningham Charles C Thabane Lehana L

Contemporary clinical trials communications 20180110

<h4>Objectives</h4>This study reviews simulation studies of discrete choice experiments to determine (i) how survey design features affect statistical efficiency, (ii) and to appraise their reporting quality.<h4>Outcomes</h4>Statistical efficiency was measured using relative design (D-) efficiency, D-optimality, or D-error.<h4>Methods</h4>For this systematic survey, we searched Journal Storage (JSTOR), Since Direct, PubMed, and OVID which included a search within EMBASE. Searches were conducted ...[more]

PMID: 29696154

Similar Datasets

Project description:BackgroundThe potential barriers and facilitators to change should guide the choice of implementation strategy. Implementation researchers believe that existing methods for the evaluation of potential barriers and facilitators are not satisfactory. Discrete choice experiments (DCE) are relatively new in the health care sector to investigate preferences, and may be of value in the field of implementation research. The objective of our study was to investigate the complementary value of DCE for the evaluation of barriers and facilitators in implementation research.MethodsClinical subject was the implementation of the guideline for breast cancer surgery in day care. We identified 17 potential barriers and facilitators to the implementation of this guideline. We used a traditional questionnaire that was made up of statements about the potential barriers and facilitators. Respondents answered 17 statements on a five-point scale ranging from one (fully disagree) to five (fully agree). The potential barriers and facilitators were included in the DCE as decision attributes. Data were gathered among anaesthesiologists, surgical oncologists, and breast care nurses by means of a paper-and-pencil questionnaire.ResultsThe overall response was 10%. The most striking finding was that the responses to the traditional questionnaire hardly differentiated between barriers. Forty-seven percent of the respondents thought that DCE is an inappropriate method. These respondents considered DCE too difficult and too time-consuming. Unlike the traditional questionnaire, the results of a DCE provide implementation researchers and clinicians with a relative attribute importance ranking that can be used to prioritize potential barriers and facilitators to change, and hence to better fine-tune the implementation strategies to the specific problems and challenges of a particular implementation process.ConclusionThe results of our DCE and traditional questionnaire would probably lead to different implementation strategies. Although there is no 'gold standard' for prioritising potential barriers and facilitators to the implementation of change, theoretically, DCE would be the method of choice. However, the feasibility of using DCE was less favourable. Further empirical applications should investigate whether DCE can really make a valuable contribution to the implementation science.

Project description:IntroductionDespite the recognised importance of participant understanding for valid and reliable discrete choice experiment (DCE) results, there has been limited assessment of whether, and how, people understand DCEs, and how 'understanding' is conceptualised in DCEs applied to a health context.ObjectivesOur aim was to identify how participant understanding is conceptualised in the DCE literature in a health context. Our research questions addressed how participant understanding is defined, measured, and used.MethodsSearches were conducted (June 2019) in the MEDLINE, EMBASE, PsychINFO and Econlit databases, as well as hand searching. Search terms were based on previous DCE systematic reviews, with additional understanding keywords used in a proximity-based search strategy. Eligible studies were peer-reviewed journal articles in the field of health, related to DCE or best-worst scaling type 3 (BWS3) studies, and reporting some consideration or assessment of participant understanding. A descriptive analytical approach was used to chart relevant data from each study, including publication year, country, clinical area, subject group, sample size, study design, numbers of attributes, levels and choice sets, definition of understanding, how understanding was tested, results of the understanding tests, and how the information about understanding was used. Each study was categorised based on how understanding was conceptualised and used within the study.ResultsOf 306 potentially eligible articles identified, 31 were excluded based on titles and abstracts, and 200 were excluded on full-text review, resulting in 75 included studies. Three categories of study were identified: applied DCEs (n = 52), pretesting studies (n = 7) and studies of understanding (n = 16). Typically, understanding was defined in relation to either the choice context, such as attribute terminology, or the concept of choosing. Very few studies considered respondents' engagement as a component of understanding. Understanding was measured primarily through qualitative pretesting, rationality or validity tests included in the survey, and participant self-report, however reporting and use of the results of these methods was inconsistent.ConclusionsThose conducting or using health DCEs should carefully select, justify, and report the measurement and potential impact of participant understanding in their specific choice context. There remains scope for research into the different components of participant understanding, particularly related to engagement, the impact of participant understanding on DCE validity and reliability, the best measures of understanding, and methods to maximise participant understanding.

Dataset Information

Investigating the impact of design characteristics on statistical efficiency within discrete choice experiments: A systematic survey.

Objectives

Outcomes

Methods

Results

Conclusion

Publications

Investigating the impact of design characteristics on statistical efficiency within discrete choice experiments: A systematic survey.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets