Dataset Information

Influence of response instructions and response format on applicant perceptions of a situational judgement test for medical school selection.

ABSTRACT:

Background

This study examined the influence of two Situational Judgement Test (SJT) design features (response instructions and response format) on applicant perceptions. Additionally, we investigated demographic subgroup differences in applicant perceptions of an SJT.

Methods

Medical school applicants (N?=?372) responded to an online survey on applicant perceptions, including a description and two example items of an SJT. Respondents randomly received one of four SJT versions (should do-rating, should do-pick-one, would do-rating, would do-pick-one). They rated overall favourability and items on four procedural justice factors (face validity, applicant differentiation, study relatedness and chance to perform) and ease-of-cheating. Additionally, applicant perceptions were compared for subgroups based on gender, ethnic background and first-generation university status.

Results

Applicants rated would-do instructions as easier to cheat than should-do instructions. Rating formats received more favourable judgements than pick-one formats on applicant differentiation, study-relatedness, chance to perform and ease of cheating. No significant main effect for demographic subgroup on applicant perceptions was found, but significant interaction effects showed that certain subgroups might have more pronounced preferences for certain SJT design features. Specifically, ethnic minority applicants - but not ethnic majority applicants - showed greater preference for should-do than would-do instructions. Additionally, first-generation university students - but not non-first-generation university students - were more favourable of rating formats than of pick-one formats.

Conclusions

Findings indicate that changing SJT design features may positively affect applicant perceptions by promoting procedural justice factors and reducing perceived ease of cheating and that response instructions and response format can increase the attractiveness of SJTs for minority applicants.

SUBMITTER: De Leng WE

PROVIDER: S-EPMC6258459 | biostudies-literature | 2018 Nov

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Influence of response instructions and response format on applicant perceptions of a situational judgement test for medical school selection.

De Leng Wendy E WE Stegers-Jager Karen M KM Born Marise Ph MP Themmen Axel P N APN

BMC medical education 20181126 1

<h4>Background</h4>This study examined the influence of two Situational Judgement Test (SJT) design features (response instructions and response format) on applicant perceptions. Additionally, we investigated demographic subgroup differences in applicant perceptions of an SJT.<h4>Methods</h4>Medical school applicants (N = 372) responded to an online survey on applicant perceptions, including a description and two example items of an SJT. Respondents randomly received one of four SJT versions (sh ...[more]

PMID: 30477494

Similar Datasets

Project description:ContextDespite their widespread use in medical school selection, there remains a lack of clarity on exactly what situational judgement tests (SJTs) measure.ObjectivesWe aimed to develop an SJT that measures integrity by combining critical incident interviews (inductive approach) with an innovative deductive approach. The deductive approach guided the development of the SJT according to two established theoretical models, of which one was positively related to integrity (honesty-humility [HH]) and one was negatively related to integrity (cognitive distortions [CD]). The Integrity SJT covered desirable (HH-based) and undesirable (CD-based) response options. We examined the convergent and discriminant validity of the Integrity SJT and compared the validity of the HH-based and CD-based subscores.MethodsThe Integrity SJT was administered to 402 prospective applicants at a Dutch medical school. The Integrity SJT consisted of 57 scenarios, each followed by four response options, of which two represented HH facets and two represented CD categories. Three SJT scores were computed, including a total, an HH-based and a CD-based score. The validity of these scores was examined according to their relationships with external integrity-related measures (convergent validity) and self-efficacy (discriminant validity).ResultsThe three SJT scores correlated significantly with all integrity-related measures and not with self-efficacy, indicating convergent and discriminant validity. In addition, the CD-based SJT score correlated significantly more strongly than the HH-based SJT score with two of the four integrity-related measures.ConclusionsAn SJT that assesses the ability to correctly recognise CD-based response options as inappropriate (i.e. what one should not do) seems to have stronger convergent validity than an SJT that assesses the ability to correctly recognise HH-based response options as appropriate (i.e. what one should do). This finding might be explained by the larger consensus on what is considered inappropriate than on what is considered appropriate in a challenging situation. It may be promising to focus an SJT on the ability to recognise what one should not do.

Project description:ImportanceThe ophthalmology residency application process is critical for applicants and residency programs, and knowledge about the preferences of applicants would assist both groups in improving the process.ObjectiveTo evaluate the experiences and preferences of ophthalmology residency applicants.Design, setting, and participantsThis cross-sectional, nonvalidated survey was conducted online. All applicants to the Bascom Palmer Eye Institute ophthalmology residency program during the 2018-2019 application cycle were invited to complete the survey. Data collection occurred from April 1, 2019, to April 30, 2019.Main outcomes and measuresApplicant demographics, application submissions, interview experiences, financial considerations, match results, and suggestions for improvement of the application process.ResultsResponses were received from 185 applicants (36.4%), including 77 women (41.6%). A successful match into an ophthalmology residency was achieved by 172 respondents (93.0%). There was a mean (SD) US Medical Licensing Examination Step 1 score of 245.8 (13.3) points. Respondents applied to a mean (SD) of 76.4 (23.5) ophthalmology residency programs, received 14.0 (9.0) invitations to interview, and attended 10.3 (4.4) interviews. Choices regarding applications and interviews were based mostly on program reputation, location, and advisor recommendation. A usual lead time of at least 3 weeks between the invitation and interview was reported by 126 respondents (69.2%), which was reduced to 14 respondents (15.1%) when a wait-list was involved. The ophthalmology residency application process cost a mean (SD) of $5704 ($2831) per applicant. Respondents reported that they were most able to reduce costs through housing choices (hotel stays or similar arrangements) and least able to reduce costs by limiting the number of programs to which they applied or at which they interviewed.Conclusions and relevanceThe ophthalmology residency application process is complex and poses substantial challenges to applicants and residency programs. These findings suggest that many current applicants have difficulty selecting programs to apply to, and most respondents desired changes to the current system of interview invitations and scheduling.

Project description:BackgroundThis paper explores local community perceptions of a relatively new rural medical school. For the purposes of this paper, community engagement is conceptualized as involvement in planning, delivering, and evaluating the medical program. Although there are several reviews of patient involvement in medical curricula development, this study was designed to pilot an approach to exploring the perspectives of well members of the community in the transition of institutional policy on community engagement to one medical school.MethodsAn advertisement in the local newspaper invited volunteers to participate in a telephone interview about the new medical school. An independent researcher external to the medical school conducted the interviews using a topic guide. Audio recordings were not made, but detailed notes including verbatim statements were recorded. At least two research team members analyzed interview records for emergent themes. Human research ethics approval was obtained.ResultsTwelve interviews were conducted. Participants offered rich imaginings on the role of the school and expectations and opportunities for students. Most participants expressed strong and positive views, especially in addressing long-term health workforce issues. It was considered important that students live, mix, and study in the community. Some participants had very clear ideas about the need of the school to address specified needs, such as indigenous health, obesity, aging, drug and alcohol problems, teenage pregnancy, ethnic diversity, and working with people of low socioeconomic status.ConclusionThis study has initiated a dialogue with potential partners in the community, which can be built upon to shape the medical school's mission and contribution to the society it serves. The telephone interview approach and thematic analysis yielded valuable insights and is recommended for further studies. Our study was limited by its small study size and the single recruitment source. The community is a rich resource for medical education, but there is a dearth of literature on the perspectives of the community and its role in medical education.

Dataset Information

Influence of response instructions and response format on applicant perceptions of a situational judgement test for medical school selection.

Background

Methods

Results

Conclusions

Publications

Influence of response instructions and response format on applicant perceptions of a situational judgement test for medical school selection.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets