Dataset Information

Using the arthroscopic surgery skill evaluation tool as a pass-fail examination.

ABSTRACT:

Background

Examination of arthroscopic skill requires evaluation tools that are valid and reliable with clear criteria for passing. The Arthroscopic Surgery Skill Evaluation Tool was developed as a video-based assessment of technical skill with criteria for passing established by a panel of experts. The purpose of this study was to test the validity and reliability of the Arthroscopic Surgery Skill Evaluation Tool as a pass-fail examination of arthroscopic skill.

Methods

Twenty-eight residents and two sports medicine faculty members were recorded performing diagnostic knee arthroscopy on a left and right cadaveric specimen in our arthroscopic skills laboratory. Procedure videos were evaluated with use of the Arthroscopic Surgery Skill Evaluation Tool by two raters blind to subject identity. Subjects were considered to pass the Arthroscopic Surgery Skill Evaluation Tool when they attained scores of ≥ 3 on all eight assessment domains.

Results

The raters agreed on a pass-fail rating for fifty-five of sixty videos rated with an interclass correlation coefficient value of 0.83. Ten of thirty participants were assigned passing scores by both raters for both diagnostic arthroscopies performed in the laboratory. Receiver operating characteristic analysis demonstrated that logging more than eighty arthroscopic cases or performing more than thirty-five arthroscopic knee cases was predictive of attaining a passing Arthroscopic Surgery Skill Evaluation Tool score on both procedures performed in the laboratory.

Conclusions

The Arthroscopic Surgery Skill Evaluation Tool is valid and reliable as a pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory.

Clinical relevance

This study demonstrates that the Arthroscopic Surgery Skill Evaluation Tool may be a useful tool for pass-fail examination of diagnostic arthroscopy of the knee in the simulation laboratory. Further study is necessary to determine whether the Arthroscopic Surgery Skill Evaluation Tool can be used for the assessment of multiple arthroscopic procedures and whether it can be used to evaluate arthroscopic procedures performed in the operating room.

SUBMITTER: Koehler RJ

PROVIDER: S-EPMC4574927 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:BackgroundUnited States Medical Licensing Examination Step 1 will transition from numeric grading to pass/fail, sometime after January 2022. The aim of this study was to compare how program directors in orthopaedics and internal medicine perceive a pass/fail Step 1 will impact the residency application process.MethodsA 27-item survey was distributed through REDCap to 161 U.S. orthopaedic residency program directors and 548 U.S. internal medicine residency program directors. Program director emails were obtained from the American Medical Association's Fellowship and Residency Electronic Interactive Database.ResultsWe received 58 (36.0%) orthopaedic and 125 (22.8%) internal medicine program director responses. The majority of both groups disagree with the change to pass/fail, and felt that the decision was not transparent. Both groups believe that the Step 2 Clinical Knowledge exam and clerkship grades will take on more importance. Compared to internal medicine PDs, orthopaedic PDs were significantly more likely to emphasize research, letters of recommendation from known faculty, Alpha Omega Alpha membership, leadership/extracurricular activities, audition elective rotations, and personal knowledge of the applicant. Both groups believe that allopathic students from less prestigious medical schools, osteopathic students, and international medical graduates will be disadvantaged. Orthopaedic and internal medicine program directors agree that medical schools should adopt a graded pre-clinical curriculum, and that there should be a cap on the number of residency applications a student can submit.ConclusionOrthopaedic and internal medicine program directors disagree with the change of Step 1 to pass/fail. They also believe that this transition will make the match process more difficult, and disadvantage students from less highly-regarded medical schools. Both groups will rely more heavily on the Step 2 clinical knowledge exam score, but orthopaedics will place more importance on research, letters of recommendation, Alpha Omega Alpha membership, leadership/extracurricular activities, personal knowledge of the applicant, and audition electives.

Project description:BackgroundObjective Structured Clinical Examination is a standard method of testing declarative and process knowledge in clinical core competencies. It is desirable that students undergo Objective Structured Clinical Examination training before participating in the exam. However, establishing Objective Structured Clinical Examination training is resource intensive and therefore there is often limited practice time. Web-based immersive patient simulators such as ALICE (Artificial Learning Interface of Clinical Education) can possibly fill this gap as they allow for the training of complex medical procedures at the user's individual pace and with an adaptable number of repetitions at home. ALICE has previously been shown to positively influence knowledge gain and motivation.ObjectiveTherefore, the aim of this study was to develop a Web-based curriculum that teaches declarative and process knowledge and prepares students for a real Objective Structured Clinical Examination station. Furthermore, we wanted to test the influence of ALICE on knowledge gain and student motivation.MethodsA specific curriculum was developed in order to implement the relevant medical content of 2 surgical Objective Structured Clinical Examination stations into the ALICE simulator framework. A total of 160 medical students were included in the study, where 100 students had access to ALICE and their performance was compared to 60 students in a control group. The simulator performance was validated on different levels and students' knowledge gain and motivation were tested at different points during the study.ResultsThe curriculum was developed according to the Kern cycle. Four virtual clinical cases were implemented with different teaching methods (structured feedback, keynote speech, group discussion, and debriefing by a real instructor) in order to consolidate declarative and process knowledge. Working with ALICE had significant impact on declarative knowledge gain and Objective Structured Clinical Examination performance. Simulator validation was positive for face, content, construct, and predictive validity. Students showed high levels of motivation and enjoyed working with ALICE.ConclusionsALICE offers Web-based training for Objective Structured Clinical Examination preparation and can be used as a selective didactic intervention as it has positive effect on knowledge gain and student motivation.

Project description:PurposeThe expansion of the Hospital Corneal Retrieval Programme (HCRP) and the role of Eye Donation Counsellors (EDCs) have shown promising results in increasing donor cornea availability. To further enhance EDC competency and improve cornea availability for transplantation, the study proposes the Eye Donation Counsellor Skill Evaluation Tool (EDC-SET). Key words: Competency assessment, corneal blindness, Eye Donation Counsellors (EDCs), eye health, Hospital Corneal Retrieval Programme (HCRP), skill evaluation.MethodsA competency framework for EDCs and a tool for evaluation of the competency level (EDC-SET) was designed, along with scripts for emulating realistic counselling scenarios. Independent assessors were trained on using EDC-SET and evaluated EDCs participating in role-plays based on the pre-designed scripts. EDCs were briefed a priori on the EDC-SET but not on the role play scripts which were used during evaluation.ResultsWe piloted the tool at a tertiary eye hospital, and it was found to capture EDC competency reliably. The study evaluated seven EDCs each assessed by two independent assessors. Conformity, kappa, and intraclass correlation coefficient (ICC) values were calculated to estimate the parameter-wise reliability of the EDC-SET. The results suggest that the scores from both assessors closely matched for most of the parameters, and overall, their ratings showed moderate agreement. The difficulty level of each parameter as assessed by the two assessors was also calculated. It was observed that most participants found "Initiation of the Case" and "Empathy and Emotional Support" challenging. However, "Body Language," "Cultural Awareness," and "Attentiveness to Family Dynamics" were perceived as relatively easy or not difficult by participants.ConclusionEvaluation of EDCs using EDC-SET revealed the varying competency levels of EDCs and showed agreement on the evaluation between the assessors. Implementing the EDC-SET could help standardise the mapping and measurement of the skill set of EDCs, thereby enhancing their performance and reducing corneal blindness in India.