Dataset Information

Standardized Video Interview Scores Correlate Poorly with Faculty and Patient Ratings.

ABSTRACT: The Standardized Video Interview (SVI) was developed by the Association of American Medical Colleges to assess professionalism, communication, and interpersonal skills of residency applicants. How SVI scores compare with other measures of these competencies is unknown. The goal of this study was to determine whether there is a correlation between the SVI score and both faculty and patient ratings of these competencies in emergency medicine (EM) applicants. This was a retrospective analysis of a prospectively collected dataset of medical students. Students enrolled in the fourth-year EM clerkship at our institution and who applied to the EM residency Match were included. We collected faculty ratings of the students' professionalism and patient care/communication abilities as well as patient ratings using the Communication Assessment Tool (CAT) from the clerkship evaluation forms. Following completion of the clerkship, students applying to EM were asked to voluntarily provide their SVI score to the study authors for research purposes. We compared SVI scores with the students' faculty and patient scores using Spearman's rank correlation. Of the 43 students from the EM clerkship who applied in EM during the 2017-2018 and 2018-2019 application cycles, 36 provided their SVI scores. All 36 had faculty evaluations and 32 had CAT scores available. We found that SVI scores did not correlate with faculty ratings of professionalism (rho = 0.09, p = 0.13), faculty assessment of patient care/communication (rho = 0.12, p = 0.04), or CAT scores (rho = 0.11, p = 0.06). Further studies are needed to validate the SVI and determine whether it is indeed a predictor of these competencies in residency.

SUBMITTER: Hall MM

PROVIDER: S-EPMC6948708 | biostudies-literature | 2019 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Standardized Video Interview Scores Correlate Poorly with Faculty and Patient Ratings.

Hall Matthew M MM Lewis Jason J JJ Joseph Joshua W JW Ketterer Andrew R AR Rosen Carlo L CL Dubosh Nicole M NM

The western journal of emergency medicine 20191219 1

The Standardized Video Interview (SVI) was developed by the Association of American Medical Colleges to assess professionalism, communication, and interpersonal skills of residency applicants. How SVI scores compare with other measures of these competencies is unknown. The goal of this study was to determine whether there is a correlation between the SVI score and both faculty and patient ratings of these competencies in emergency medicine (EM) applicants. This was a retrospective analysis of a ...[more]

PMID: 31913835

Similar Datasets

Project description:BackgroundThe Association of American Medical Colleges instituted a standardized video interview (SVI) for all applicants to emergency medicine (EM). It is unclear how the SVI affects a faculty reviewer's decision on likelihood to invite an applicant (LTI) for an interview.ObjectivesThe objective was to determine whether the SVI affects the LTI.MethodsNine Accreditation Council of Graduate Medication Education (ACGME)-accredited EM residency programs participated in this prospective, observational study. LTI was defined on a 5-point Likert scale as follows: 1 = definitely not invite, 2 = likely not invite, 3 = might invite, 4 = probably invite, 5 = definitely invite. LTI was recorded at three instances during each review: 1) after typical screening (blinded to the SVI), 2) after unblinding to the SVI score, and 3) after viewing the SVI video.ResultsSeventeen reviewers at nine ACGME-accredited residency programs participated. We reviewed 2,219 applications representing 1,424 unique applicants. After unblinding the SVI score, LTI did not change in 2,065 (93.1%), increased in 85 (3.8%) and decreased in 69 (3.1%; p = 0.22). In subgroup analyses, the effect of the SVI on LTI was unchanged by United States Medical Licensing Examination score. However, when examining subgroups of SVI scores, the percentage of applicants in whom the SVI score changed the LTI was significantly different in those that scored in the lower and upper subgroups (p < 0.0001). The SVI video was viewed in 816 (36.8%) applications. Watching the video did not change the LTI in 631 (77.3%); LTI increased in 106 (13.0%) and decreased in 79 (9.7%) applications (p = 0.04).ConclusionsThe SVI score changed the LTI in 7% of applications. In this group, the score was equally likely to increase or decrease the LTI. Lower SVI scores were more likely to decrease the LTI than higher scores were to increase the LTI. Watching the SVI video was more likely to increase the LTI than to decrease it.

Project description:Background:The Standardized Video Interview (SVI) was developed by the American Association of Medical Colleges to allow applicants to include objective data about professional behaviors and interpersonal and communication skills. Although the SVI pilot was administered to individuals applying to emergency medicine (EM) residency programs during the 2018 Electronic Residency Application Service (ERAS) cycle, little data have been published evaluating the applicant's perceptions. This survey aims to assess EM residency applicant attitudes toward the SVI. Methods:During the 2018 ERAS application season an anonymous survey was administered to interviewees at one urban Accreditation Council for Graduate Medical Education-approved EM residency. Respondents were asked questions regarding the production of their video interviews, thoughts regarding the additive value of the SVI, and individual demographic data such as ethnicity and sex. Participation was optional. Results:A total of 219 of 238 candidates completed the survey representing a 92% response rate. While the majority of applicants did not feel that their ethnicity impacted their application, 58.1% of those who did self-identified as African American or Asian. A total of 8.7% of respondents felt the SVI added information about their professional behaviors and 11% felt that it added information about interpersonal and communication skills. Only 2.8% of survey respondents felt the SVI should remain a portion of the ERAS application. Conclusions:Most respondents felt that the SVI was not an accurate representation of their interpersonal and communication skills or their professionalism and that it did not add value to their applications. While most cohorts were not concerned about bias regarding sex, ethnicity, sex, or age, a small subset felt that there was a potential for the SVI to bias the party reviewing their applications. Very few applicants felt the SVI should remain a part of the ERAS application. Applicant attitudes toward the SVI are largely negative and require further investigation prior to becoming a standard part of applicants' ERAS files.

Project description:BackgroundStudies suggest that black patients have better interactions, on average, with physicians of their own race. Whether this reflects greater "cultural competence" in race-concordant relationships, or other effects of race unrelated to physician behavior, is unclear. It is also unclear if physician race influences patient decision-making.ObjectiveTo determine whether physician race affects patients' ratings of physicians and decision-making, independent of physician behavior.DesignRandomized study using standardized video vignettes.ParticipantsPrimary care patients with coronary risk factors or disease.InterventionsEach participant viewed one of 16 vignettes depicting a physician reviewing cardiac catheterization results and recommending coronary artery bypass graft (CABG) surgery. Vignettes varied only in terms of physicians' race, gender, age, and communication style (high vs. low patient-centeredness).Main measuresParticipants rated the video physician's communication, interpersonal style, competence, trustworthiness, likability, and overall performance (0-4 Likert scales). They also rated the necessity of CABG (0-5 scale) and whether they would undergo CABG or obtain a second opinion if they were the video patient (0-3 scales).Key resultsParticipants included 107 black and 131 white patients (72% participation rate). Black participants viewing a black (vs. white) video physician gave higher ratings on all physician attributes (e.g., overall rating 3.22 vs. 2.34, p < 0.001) and were more likely to perceive CABG as necessary (4.05 vs. 3.72, p = 0.03) and say they would undergo CABG if they were the video patient (2.43 vs. 2.09, p = 0.004). Patient-centered communication style reduced, but did not eliminate, the impact of race concordance. Physician race was not associated with any outcomes among white patients.ConclusionsBlack patients viewed the doctor in a scripted vignette more positively, and were more receptive to the same recommendation, communicated in the same way, with a black vs. white physician. Patient-centered communication reduced but did not eliminate the effect of physician race.

Dataset Information

Standardized Video Interview Scores Correlate Poorly with Faculty and Patient Ratings.

Publications

Standardized Video Interview Scores Correlate Poorly with Faculty and Patient Ratings.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets