Dataset Information

Using standardized videos to validate a measure of handoff quality: the handoff mini-clinical examination exercise.

ABSTRACT: The most recent iteration of the Accreditation Council for Graduate Medical Education duty-hour regulations includes language mandating handoff education for trainees and assessments of handoff quality by residency training programs. However, there is a lack of validated tools for the assessment of handoff quality and for use in trainee education.Faculty at 2 sites (University of Chicago and Yale University) were recruited to participate in a workshop on handoff education. Video-based scenarios were developed to represent varying levels of performance in the domains of communication, professionalism, and setting. Videos were shown in a random order, and faculty were instructed to use the Handoff Mini-Clinical Examination Exercise (CEX), a paper-based instrument with qualitative anchors defining each level of performance, to rate the handoffs.Forty-seven faculty members (14 at site 1; 33 at site 2) participated in the validation workshops, providing a total of 172 observations (of a possible 191 [96%]). Reliability testing revealed a Cronbach ? of 0.81 and Kendall coefficient of concordance of 0.59 (>0.6?=?high reliability). Faculty were able to reliably distinguish the different levels of performance in each domain in a statistically significant fashion (ie, unsatisfactory professionalism mean 2.42 vs satisfactory professionalism 4.81 vs superior professionalism 6.01, P?

SUBMITTER: Arora VM

PROVIDER: S-EPMC4079746 | biostudies-literature | 2014 Jul

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Using standardized videos to validate a measure of handoff quality: the handoff mini-clinical examination exercise.

Arora Vineet M VM Berhie Saba S Horwitz Leora I LI Saathoff Mark M Staisiunas Paul P Farnan Jeanne M JM

Journal of hospital medicine 20140324 7

<h4>Background</h4>The most recent iteration of the Accreditation Council for Graduate Medical Education duty-hour regulations includes language mandating handoff education for trainees and assessments of handoff quality by residency training programs. However, there is a lack of validated tools for the assessment of handoff quality and for use in trainee education.<h4>Methods</h4>Faculty at 2 sites (University of Chicago and Yale University) were recruited to participate in a workshop on handof ...[more]

PMID: 24665068

Similar Datasets

Project description:BackgroundCultural differences might challenge the acceptance of the implementation of assessment formats that are developed in other countries. Acceptance of assessment formats is essential for its effectiveness; therefore, we explored the views of students and specialists on the practicality and impact on learning of these formats. This study was conducted to explore Indonesian students' and specialists' appreciation of the implementation of the Mini-Clinical Evaluation Exercise (Mini-CEX) in Indonesian clerkships.MethodsThis study was conducted at the Universitas Gadjah Mada, Indonesia. Participants were 52 students and 21 specialists in neurology and 78 students and 50 specialists in internal medicine. They were asked to complete a 19-item questionnaire that covered the characteristics of the mini-CEX such as its practicality, and the impact on learning and professional development. We used a Mann-Whitney U test to analyse the data.ResultsIn total, 124 students (46 from neurology and 78 from internal medicine) and 38 specialists (13 from neurology and 25 from internal medicine) participated in this study. Students and specialists were positive about the practicality of the mini-CEX and the impact of this assessment format on learning and on professional development. The Mann-Whitney U test showed that there were no significant differences between students' and specialists' opinions on the mini-CEX, except for 2 items: specialists' appreciation of direct observation (mean rank = 93.16) was statistically significantly higher than students' appreciation of it (mean rank = 77.93; z = 2.065; p < 0.05), but students' appreciation of the item that students' past mini-CEX results affected their recent mini-CEX outcomes (mean rank = 85.29) was significantly higher than specialists' appreciation of it (mean rank = 69.12; z = 2140; p < 0.05).ConclusionStudents and specialists were positive about the mini-CEX in Indonesian clerkships, although it was developed and validated in another culture. We found only small differences between their appreciations, which could be explained by the patterns of specialist-student interaction in Indonesian culture as large power distance and low individualism country.

Project description:BackgroundStandardized patients (SPs) are essential stakeholders in the multiple mini interviews (MMIs) that are increasingly used to assess medical school applicants' interpersonal skills. However, there is little evidence for their inclusion in the development of instruments.ObjectiveThis study aimed to describe the process and evaluate the impact of having SPs co-design and cocreate a global measurement question that assesses medical school applicants' readiness for medical school and acceptance status.MethodsThis study used an exploratory, sequential, and mixed methods study design. First, we evaluated the initial MMI program and determined the next quality improvement steps. Second, we held a collaborative workshop with SPs to codevelop the assessment question and response options. Third, we evaluated the created question and the additional MMI rubric items through statistical tests based on 1084 applicants' data from 3 cohorts of applicants starting in the 2018-2019 academic year. The internal reliability of the MMI was measured using a Cronbach α test, and its prediction of admission status was tested using a forward stepwise binary logistic regression.ResultsProgram evaluation indicated the need for an additional quantitative question to assess applicant readiness for medical school. In total, 3 simulation specialists, 2 researchers, and 21 SPs participated in a workshop leading to a final global assessment question and responses. The Cronbach α's were >0.8 overall and in each cohort year. The final stepwise logistic model for all cohorts combined was statistically significant (P<.001), explained 9.2% (R2) of the variance in acceptance status, and correctly classified 65.5% (637/972) of cases. The final model consisted of 3 variables: empathy, rank of readiness, and opening the encounter.ConclusionsThe collaborative nature of this project between stakeholders, including nonacademics and researchers, was vital for the success of this project. The SP-created question had a significant impact on the final model predicting acceptance to medical school. This finding indicates that SPs bring a critical perspective that can improve the process of evaluating medical school applicants.

Project description:Objective: Exercise-induced bronchoconstriction (EIB) is a highly prevalent morbidity of childhood asthma and defined by a transient narrowing of the airways during or after physical exercise. An exercise challenge test (ECT) is the reference standard for the diagnosis of EIB. Video evaluation of EIB symptoms could be a practical alternative for the assessment of EIB. We studied the ability of pediatricians to assess EIB from post-exercise videos. Methods: A clinical assessment was performed in 20 asthmatic children (mean age 11.6 years) and EIB was measured with a standardized ECT performed in cold, dry air. EIB was defined as a fall in forced expiratory volume in 1 s (FEV1) of ?10% post-exercise. Children were filmed before and after exercise in frontal position and bare chested. The clinical assessment results and videos were shown to 20 pediatricians (mean experience 14.4 years). Each assessed EIB severity in 5 random children providing 100 assessments, scored on a continuous rating scale (0-10) and in severity classifications (no, mild, moderate, severe) using a scoring list including physical asthma symptoms. Correlations between predicted scores and objective scores were calculated with Spearman's rho and Cohen's Kappa. A generalized linear model was used to assess the relationship between physical symptoms and fall in FEV1. Results: Median fall in FEV1 after exercise was 15.1% (IQR 1.2-65.1). Pediatricians detected EIB with a sensitivity of 78% (95% CI 66-87%) and a specificity of 40% (95% CI 27-55%). The positive predictive value for a pediatricians' diagnosis of EIB was 61% (95% CI 50-72%). The negative predictive value was 60% (95% CI 42-76%). The agreement between predicted EIB severity classifications and the validated classifications based on the ECT's, was fair [Kappa = 0.36 (95% CI 0.23-0.48)]. The correlation between predicted EIB severity scored on a continuous rating scale and fall in FEV1 after exercise was weak (rs = 0.39, p < 0.001). Independent predictive variables for fall in FEV1 were wheezing (-11%), supraclavicular retractions (-8.4%) and a prolonged expiratory phase (-8.8%). Conclusion: The ability of pediatricians to assess EIB from post-exercise videos is fair at best, implicating that standardized ECT's are still vital in the assessment of EIB.

Dataset Information

Using standardized videos to validate a measure of handoff quality: the handoff mini-clinical examination exercise.

Publications

Using standardized videos to validate a measure of handoff quality: the handoff mini-clinical examination exercise.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets