Dataset Information

Sequential Objective Structured Clinical Examination based on item response theory in Iran.

ABSTRACT:

Purpose

In a sequential objective structured clinical examination (OSCE), all students initially take a short screening OSCE. Examinees who pass are excused from further testing, but an additional OSCE is administered to the remaining examinees. Previous investigations of sequential OSCE were based on classical test theory. We aimed to design and evaluate screening OSCEs based on item response theory (IRT).

Methods

We carried out a retrospective observational study. At each station of a 10-station OSCE, the students' performance was graded on a Likert-type scale. Since the data were polytomous, the difficulty parameters, discrimination parameters, and students' ability were calculated using a graded response model. To design several screening OSCEs, we identified the 5 most difficult stations and the 5 most discriminative ones. For each test, 5, 4, or 3 stations were selected. Normal and stringent cut-scores were defined for each test. We compared the results of each of the 12 screening OSCEs to the main OSCE and calculated the positive and negative predictive values (PPV and NPV), as well as the exam cost.

Results

A total of 253 students (95.1%) passed the main OSCE, while 72.6% to 94.4% of examinees passed the screening tests. The PPV values ranged from 0.98 to 1.00, and the NPV values ranged from 0.18 to 0.59. Two tests effectively predicted the results of the main exam, resulting in financial savings of 34% to 40%.

Conclusion

If stations with the highest IRT-based discrimination values and stringent cut-scores are utilized in the screening test, sequential OSCE can be an efficient and convenient way to conduct an OSCE.

SUBMITTER: Hejri SM

PROVIDER: S-EPMC5676019 | biostudies-literature | 2017

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Sequential Objective Structured Clinical Examination based on item response theory in Iran.

Hejri Sara Mortaz SM Jalili Mohammad M

Journal of educational evaluation for health professions 20170908

<h4>Purpose</h4>In a sequential objective structured clinical examination (OSCE), all students initially take a short screening OSCE. Examinees who pass are excused from further testing, but an additional OSCE is administered to the remaining examinees. Previous investigations of sequential OSCE were based on classical test theory. We aimed to design and evaluate screening OSCEs based on item response theory (IRT).<h4>Methods</h4>We carried out a retrospective observational study. At each statio ...[more]

PMID: 28881560

Similar Datasets

Project description:Introduction: The aim of the Objective Structured Clinical Examination (OSCE) is a standardized and fair assessment of clinical skills. Observing second clinical year medical students during a summative OSCE assessing a General Practice clerkship, we noticed that information exchange with peers led to a progressively faster and overly focused management of simulations. Therefore, we established a Multiple Scenario-OSCE (MS-OSCE) where all students had to manage the same chief complaint at a station but it's underlying scenarios being randomly changed during students' rotation through their parcours. We wanted to ensure they fully explore differential diagnosis instead of managing their task influenced by shared information. We wanted to assess if a MS-OSCE violates the assumption of objectivity and fairness given that students are not tested with the same scenarios. Methods: We developed and piloted five OSCE stations (chest pain, abdominal pain, back pain, fatigue and acute cough) with two or three different underlying scenarios each. At each station these scenarios randomly changed from student to student. Performance was assessed with a checklist and global rating. The effect of scenarios and raters on students' grades was assessed calculating the intraclass correlation coefficient with a fixed effect two level linear model. Results: A total of 169 students and 23 raters participated in the MS-OSCE. The internal consistency over all stations was 0.65 by Cronbach's alpha. The difference of the mean grades between the scenarios of a given chief complaint ranged from 0.03 to 0.4 on a 1 to 5 grading scale. The effect of scenarios on the variance of the final grades at each station ranged from 4% to 9% and of raters from 20% to 50% when adjusted for students' skills. Conclusions: The effect of different scenarios on the grades was relevant but small compared to the effect of raters on grades. Improving rater training is more important to ensure objectivity and fairness of MS-OSCE than providing the same scenario to all students.

Project description:BACKGROUND:Evaluation of the competence and practical skills of nursing students, using the objective structured clinical examination (OSCE), is an integral part of the nursing education program. However, their performance could be negatively influenced by a significant level of stress and anxiety prior to the test. The present study aimed to evaluate the effect of an anxiety coping program on the OSCE performance level of first-year nursing students in Shiraz, Iran. METHODS:The present quasi-experimental study was conducted among 76 nursing students; control group (n?=?35) and intervention group (n?=?41). To attain the study purpose, the intervention group received a pre-exam anxiety coping program that included relaxation and soothing techniques, diaphragmatic breathing training, and progressive muscle relaxation training accompanied by light instrumental music, while the control group received no intervention before the exam. Data collection instruments included a demographic characteristics form and State-Trait Anxiety Inventory (STAI) questionnaire. Demographic characteristics of the participants indicated an overall homogeneity within the study population. The STAI questionnaire was filled in before and after the OSCE and the results were compared with those of the control group. The data were analysed using SPSS software (version 22.0). P?<?0.05 was considered statistically significant. RESULTS:As a result of the anxiety coping program, a substantial reduction in the anxiety score (by 11.61?units) in the intervention group was observed. There was a significant difference in the pre- and post-exam anxiety scores between the control and intervention groups (P?<?0.001). The anxiety coping program improved the examination results of nursing students in the final exam compared to the midterm results (an increase of 0.9487?units, P?<?0.001). CONCLUSION:The anxiety coping program reduced the anxiety level among nursing students and improved their OSCE results. Our findings can be utilized to better evaluate clinical activities in different medical and paramedical groups. Moreover, educators can implement such coping programs prior to evaluations in order to effectively assess the knowledge, attitude, and performance of the students.

Project description:BACKGROUND:Objective structured clinical examinations (OSCEs) are a useful method to evaluate medical students' performance in the clerkship years. OSCEs are designed to assess skills and knowledge in a standardized clinical setting and through use of a preset standard grading sheet, so that clinical knowledge can be evaluated at a high level and in a reproducible way. OBJECTIVE:This study aimed to present our OSCE assessment tool designed specifically for radiology clerkship medical students, which we called the objective structured radiology examination (OSRE), with the intent to advance the assessment of clerkship medical students by providing an objective, structured, reproducible, and low-cost method to evaluate medical students' radiology knowledge and the reproducibility of this assessment tool. METHODS:We designed 9 different OSRE cases for radiology clerkship classes with participating third- and fourth-year medical students. Each examination comprises 1 to 3 images, a clinical scenario, and structured questions, along with a standardized scoring sheet that allows for an objective and low-cost assessment. Each medical student completed 3 of 9 random examination cases during their rotation. To evaluate for reproducibility of our scoring sheet assessment tool, we used 5 examiners to grade the same students. Reproducibility for each case and consistency for each grader were assessed with a two-way mixed effects intraclass correlation coefficient (ICC). An ICC below 0.4 was deemed poor to fair, an ICC of 0.41 to 0.60 was moderate, an ICC of 0.6 to 0.8 was substantial, and an ICC greater than 0.8 was almost perfect. We also assessed the correlation of scores and the students' clinical experience with a linear regression model and compared mean grades between third- and fourth-year students. RESULTS:A total of 181 students (156 third- and 25 fourth-year students) were included in the study for a full academic year. Moreover, 6 of 9 cases demonstrated average ICCs more than 0.6 (substantial correlation), and the average ICCs ranged from 0.36 to 0.80 (P<.001 for all the cases). The average ICC for each grader was more than 0.60 (substantial correlation). The average grade among the third-year students was 11.9 (SD 4.9), compared with 12.8 (SD 5) among the fourth-year students (P=.005). There was no correlation between clinical experience and OSRE grade (-0.02; P=.48), adjusting for the medical school year. CONCLUSIONS:Our OSRE is a reproducible assessment tool with most of our OSRE cases showing substantial correlation, except for 3 cases. No expertise in radiology is needed to grade these examinations using our scoring sheet. There was no correlation between scores and the clinical experience of the medical students tested.

Dataset Information

Sequential Objective Structured Clinical Examination based on item response theory in Iran.

Purpose

Methods

Results

Conclusion

Publications

Sequential Objective Structured Clinical Examination based on item response theory in Iran.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets