Dataset Information

Effect of response format for clinical vignettes on reporting quality of physician practice.

ABSTRACT:

Background

Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats - open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) - in rheumatologists' pre-treatment assessment for tumor-necrosis-factor (TNF) blocker therapy.

Methods

Study design

Prospective randomized study.

Setting

Rheumatologists attending the 2004 French Society of Rheumatology meeting. Physicians were given a vignette describing the history of a fictitious woman with active rheumatoid arthritis, who was a candidate for therapy with TNF blocking agents, and then were randomized to receive questionnaire A, B, or C, each containing the same four questions but with different response formats, that asked about their pretreatment assessment.

Measurements

Long (recommended items) and short (mandatory items) checklists were developed for pretreatment assessment for TNF-blocker therapy, and scores were expressed on the basis of responses to questionnaires A, B, and C as the percentage of respondents correctly choosing explicit items on these checklists.

Statistical analysis

Comparison of the selected items using pairwise Chi-square tests with Bonferonni correction for variables with statistically significant differences.

Results

Data for all surveys distributed (114 As, 118 Bs, and 118 Cs) were complete and available for analysis. The percentage of questionnaire A, B, and C respondents for whom data was correctly complete for the short checklist was 50.4%, 84.0% and 95.0%, respectively, and was 0%, 5.0% and 5.9%, respectively, for the long version. As an example, 65.8%, 85.7% and 95.8% of the respondents of A, B, and C questionnaires, respectively, correctly identified the need for tuberculin skin test (p < 0.0001).

Conclusion

In evaluating clinical practice with use of a clinical vignette, a multiple-choice format rather than an open-ended format overestimates physician performance. The insertion of deceptive response items mixed with correct items in closed-ended (multiple-choice) questionnaire failed to avoid this overestimation.

SUBMITTER: Pham T

PROVIDER: S-EPMC3224732 | biostudies-literature | 2009 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Effect of response format for clinical vignettes on reporting quality of physician practice.

Pham Thao T Roy Carine C Mariette Xavier X Lioté Fréderic F Durieux Pierre P Ravaud Philippe P

BMC health services research 20090728

<h4>Background</h4>Clinical vignettes have been used widely to compare quality of clinical care and to assess variation in practice, but the effect of different response formats has not been extensively evaluated. Our objective was to compare three clinical vignette-based survey response formats - open-ended questionnaire (A), closed-ended (multiple-choice) questionnaire with deceptive response items mixed with correct items (B), and closed-ended questionnaire with only correct items (C) - in rh ...[more]

PMID: 19638231

Similar Datasets

Project description:Physicians often select clinical management strategies not strongly supported by evidence or guidelines. Our objective was to examine the likelihood of selecting, and rationale for pursuing, clinical management strategies with more or less guideline support among physicians using clinical vignettes of eight common medical admissions.We conducted a cross-sectional survey using clinical vignettes of attending physicians and housestaff at one internal medicine program in New York City. Each clinical vignette included a brief clinical scenario and a varying number of clinical management strategies: diagnostic tests, consultations, and treatments, some of which had strong evidence or guideline support (Level 1 strategies) while others had limited evidence or guideline support (Level 3 strategies). Likelihood of selecting a given management strategy was assessed using Likert scales and multiple response options were used to indicate rationale(s) for selections.Our sample included 79 physicians; 68 (86%) were younger than 40 years of age, 34 (43%) were female. There were 31 attending physicians (39%) and 48 housestaff (61%) and 39 (49%) had or planned to have primarily primary care internal medicine clinical responsibilities. Overall, physicians were more likely to select Level 1 strategies "always" or "most of the time" when compared with Level 3 strategies (82% vs. 43%; p?<?0.001), with wide variation across the eight medical admissions. There were no differences between attending and housestaff physician likelihood of selecting Level 3 strategies (47% vs. 45%, p?=?0.36). Supportive evidence and local practice patterns were the two most common rationales behind selections; supportive evidence was cited as the most common rationale for selecting Level 1 when compared with Level 3 strategies (63% versus 30%; p?<?0.001), whereas ruling out other severe conditions was cited most often for Level 3 strategies.For eight common medical admissions, physicians selected more than 80% of management strategies with strong evidence or guideline support, but also selected more than 40% of strategies for which there was limited evidence or guideline support. The promotion of evidence-based care, including the avoidance of care that is not strongly supported by evidence or guidelines, may require better evidence dissemination and educational outreach to physicians.

Project description:BackgroundHealth practice guidelines (HPGs) are important tools for the translation of evidence into practice. Reporting Items for Practice Guidelines in HealThcare (RIGHT) checklist provides guidance on reporting health practice guidelines (HPGs). We assessed the reporting completeness and quality of a set of national (Croatian) and relevant transnational (European) HPGs.MethodsThe national sample included all HPGs published in the official journal of the Croatian Medical Association in 2014-2016. We searched PubMed to identify relevant European guidelines (n?=?24). Two independent reviewers assessed the adherence with the items on the RIGHT checklist. Kappa score was used to measure the level of agreement. Frequentist and Bayes statistics Bayes factor (BF10) was used to evaluate the differences between the national and transnational HPGs.ResultsOverall, Croatian and European HPGs adhered to less than 50% of RIGHT checklist items. Croatian HPGs reported a median of 14.0 (95% CI 13.0-15.0) RIGHT reporting items, and European counterparts reported a median of 16.0 (95% CI 14.0-17.2) out of the total of 35 checklist items (Mann Whitney U test, P?=?0.048; BF10?=?1.543). European HPGs were better than Croatian HPGs in reporting stakeholder involvement and values and preferences (BF10?=?80.63), as well as describing the implications of costs and resources (BF10?=?55.15). Croatian HPGs better reported HPGs specified aims (BF10?=?16.90), primary intended users (BF10?=?8.70), and sources of funding (BF10?=?122.90). Most insufficiently reported items for both HPG sets were defining the guideline questions and clear outcomes, quality assurance, management of funding and conflicts of interest, and guideline limitations.ConclusionsImportant methodological details are missing from most published HPGs at national and transnational levels. To ensure better quality and adequate use of HPGs, reporting guidelines should be endorsed and used by developers and users alike.

Project description:BACKGROUND:Migrants may carry with them communicable and non communicable diseases as they move to the host country. Screening migrants may help in improving their health status and in preventing the spread of infections to the host population. OBJECTIVE:To identify and assess the quality of published practice guidelines addressing migrants' health. METHODS:We included practice guidelines addressing migrants' health at the clinical, public health or health systems levels. We searched Medline, Embase, the National Guideline Clearinghouse and the Canadian Medical Association's Clinical Practice Guidelines Database. Two teams of two reviewers conducted in duplicate and independent manner study selection, data abstraction, assessment of the guideline quality (using the AGREE II instrument), and assessment of the quality of the reporting (using the RIGHT statement). RESULTS:Out of 2732 citations captured by the electronic search, we included 24 eligible practice guidelines, all addressing the level of post-arrival to the host country and published between 2011 and 2017. The majority of guidelines (57%) addressed non-communicable diseases, 95% addressed screening, while 52% addressed prevention and treatment respectively. The majority of the guidelines reported their funding sources. 86% used the GRADE approach as part of the development process. The included guidelines scored high on the majority of the items, and low on the following two domains of the AGREE II instrument: rigor of development and applicability. The mean number of the RIGHT checklist items met by the included guidelines was 27, out of a total of 35. Most of the guidelines were based on systematic reviews (95.6%). A minority of the included guidelines (26%) reported considering the values and preferences of the target populations or the costs and resource implications (30%) in the formulation of recommendations. CONCLUSION:We identified 23 practice guidelines addressing migrants' health, the majority of which addressed screening services. The vast majority of the captured guidelines targeted screening because the population of interest is migrants, meaning that the intention of the guidelines is to deal with additional factors than usual ones, such as prevalence of disease in country of origin, endemic diseases and others. The guidelines suffered limitations on two quality domains (rigor of development and applicability), and have room for improvement of their reporting.

Dataset Information

Effect of response format for clinical vignettes on reporting quality of physician practice.

Background

Methods

Study design

Setting

Measurements

Statistical analysis

Results

Conclusion

Publications

Effect of response format for clinical vignettes on reporting quality of physician practice.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure