Dataset Information

Effect of response format for clinical vignettes on reporting quality of physician practice.

ABSTRACT:

Background

Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats - open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) - in rheumatologists' pre-treatment assessment for tumor-necrosis-factor (TNF) blocker therapy.

Methods

Study design

Prospective randomized study.

Setting

Rheumatologists attending the 2004 French Society of Rheumatology meeting. Physicians were given a vignette describing the history of a fictitious woman with active rheumatoid arthritis, who was a candidate for therapy with TNF blocking agents, and then were randomized to receive questionnaire A, B, or C, each containing the same four questions but with different response formats, that asked about their pretreatment assessment.

Measurements

Long (recommended items) and short (mandatory items) checklists were developed for pretreatment assessment for TNF-blocker therapy, and scores were expressed on the basis of responses to questionnaires A, B, and C as the percentage of respondents correctly choosing explicit items on these checklists.

Statistical analysis

Comparison of the selected items using pairwise Chi-square tests with Bonferonni correction for variables with statistically significant differences.

Results

Data for all surveys distributed (114 As, 118 Bs, and 118 Cs) were complete and available for analysis. The percentage of questionnaire A, B, and C respondents for whom data was correctly complete for the short checklist was 50.4%, 84.0% and 95.0%, respectively, and was 0%, 5.0% and 5.9%, respectively, for the long version. As an example, 65.8%, 85.7% and 95.8% of the respondents of A, B, and C questionnaires, respectively, correctly identified the need for tuberculin skin test (p < 0.0001).

Conclusion

In evaluating clinical practice with use of a clinical vignette, a multiple-choice format rather than an open-ended format overestimates physician performance. The insertion of deceptive response items mixed with correct items in closed-ended (multiple-choice) questionnaire failed to avoid this overestimation.

SUBMITTER: Pham T

PROVIDER: S-EPMC3224732 | biostudies-literature | 2009 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Effect of response format for clinical vignettes on reporting quality of physician practice.

Pham Thao T Roy Carine C Mariette Xavier X Lioté Fréderic F Durieux Pierre P Ravaud Philippe P

BMC health services research 20090728

<h4>Background</h4>Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats - open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) - in rh ...[more]

PMID: 19638231

Similar Datasets

Project description:Physicians often select clinical management strategies not strongly supported by evidence or guidelines. Our objective was to examine the likelihood of selecting, and rationale for pursuing, clinical management strategies with more or less guideline support among physicians using clinical vignettes of eight common medical admissions.We conducted a cross-sectional survey using clinical vignettes of attending physicians and housestaff at one internal medicine program in New York City. Each clinical vignette included a brief clinical scenario and a varying number of clinical management strategies: diagnostic tests, consultations, and treatments, some of which had strong evidence or guideline support (Level 1 strategies) while others had limited evidence or guideline support (Level 3 strategies). Likelihood of selecting a given management strategy was assessed using Likert scales and multiple response options were used to indicate rationale(s) for selections.Our sample included 79 physicians; 68 (86%) were younger than 40 years of age, 34 (43%) were female. There were 31 attending physicians (39%) and 48 housestaff (61%) and 39 (49%) had or planned to have primarily primary care internal medicine clinical responsibilities. Overall, physicians were more likely to select Level 1 strategies "always" or "most of the time" when compared with Level 3 strategies (82% vs. 43%; p?<?0.001), with wide variation across the eight medical admissions. There were no differences between attending and housestaff physician likelihood of selecting Level 3 strategies (47% vs. 45%, p?=?0.36). Supportive evidence and local practice patterns were the two most common rationales behind selections; supportive evidence was cited as the most common rationale for selecting Level 1 when compared with Level 3 strategies (63% versus 30%; p?<?0.001), whereas ruling out other severe conditions was cited most often for Level 3 strategies.For eight common medical admissions, physicians selected more than 80% of management strategies with strong evidence or guideline support, but also selected more than 40% of strategies for which there was limited evidence or guideline support. The promotion of evidence-based care, including the avoidance of care that is not strongly supported by evidence or guidelines, may require better evidence dissemination and educational outreach to physicians.

Dataset Information

Effect of response format for clinical vignettes on reporting quality of physician practice.

Background

Methods

Study design

Setting

Measurements

Statistical analysis

Results

Conclusion

Publications

Effect of response format for clinical vignettes on reporting quality of physician practice.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets