Dataset Information

Performance of the Ebel standard-setting method in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions.

ABSTRACT: PURPOSE:It aimed to know the performance of the Ebel standard-setting method in in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. Specifically followings were searched: the inter-rater agreement; the correlation between Ebel scores and item facility indices; raters' knowledge of correct answers' impact on the Ebel score; and affection of rater's specialty on theinter-rater agreement and Ebel scores. METHODS:Data were drawn from a Royal College of Physicians and Surgeons of Canada certification exam. Ebel's method was applied to 203 MCQs by 49 raters. Facility indices came from 194 candidates. We computed Fleiss' kappa and the Pearson correlation between Ebel scores and item facility indices. We investigated differences in the Ebel score (correct answers provided or not) and differences between internists and other specialists with t-tests. RESULTS:Kappa was below 0.15 for facility and relevance. The correlation between Ebel scores and facility indices was low when correct answers were provided and negligible when they were not. The Ebel score was the same, whether the correct answers were provided or not. Inter-rater agreement and Ebel scores was not differentbetween internists and other specialists. CONCLUSION:Inter-rater agreement and correlations between item Ebel scores and facility indices wee consistently low; furthermore, raters' knowledge of correct answer and rater specialty had no effect on Ebel scores in the present setting.

SUBMITTER: Bourque J

PROVIDER: S-EPMC7242791 | biostudies-literature | 2020

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Performance of the Ebel standard-setting method in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions.

Bourque Jimmy J Skinner Haley H Dupré Jonathan J Bacchus Maria M Ainslie Martha M Ma Irene W Y IWY Cole Gary G

Journal of educational evaluation for health professions 20200420

<h4>Purpose</h4>It aimed to know the performance of the Ebel standard-setting method in in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions. Specifically followings were searched: the inter-rater agreement; the correlation between Ebel scores and item facility indices; raters' knowledge of correct answers' impact on the Ebel score; and affection of rater's specialty on theinter-rater agreement and Eb ...[more]

PMID: 32306708

Similar Datasets

Project description:Prior structural studies of posterior subcapsular cataract (PSC) development in Royal College of Surgeons (RCS) rats suggest that migration of basal fiber ends was disrupted, ultimately resulting in a PSC. Therefore the goal of this study was to assess the overall migration patterns as well as changes to the structure and cytoskeleton of basal fiber ends during PSC development.Lenses from 48 RCS dystrophic rats (RCS/Lav) and 24 genetically matched control animals (RCS-rdy(+)/Lav) from 2 to 8 weeks old were examined. Equatorial diameters were measured and suture patterns were photographed immediately following enucleation/dissection. Right eye lenses were fixed and processed to visualize the actin cytoskeleton via laser scanning confocal microscopy (LSCM), left eye lenses were decapsulated, fixed and processed for scanning electron microscopy (SEM). Scaled 3D-computer assisted drawings (CADs) and animations were constructed from the data to depict the changes in suture patterns and fiber end architecture.At 2 weeks, dystrophic lenses displayed an inverted Y suture on the posterior, and by 3 weeks most lenses had at least one sub-branch. Additional sub-branches were observed with time, opacities being visible as early as 4 weeks and progressing into PSC plaques by 6 weeks. Control lenses displayed inverted Y sutures at all ages and were transparent. SEM of dystrophic lenses revealed fiber ends with normal size, shape, arrangement, and filopodia at 2 weeks; scattered areas of dome-shaped fiber ends and small filopodia were present at 3 weeks. At 4 weeks the irregularly arranged domed fiber ends had extremely long filopodia with 'boutons' at their tips. By 6 weeks all fiber ends within plaques displayed rounded or domed basal membranes and lacked filopodial extensions. Control lenses at all time points had comparable ultrastructure to the 2 week old dystrophic lenses. F-actin arrangement within the basal membrane complex (BMC) of control lenses showed the expected peripheral pattern of labeling at all ages. Dystrophic RCS lenses at 2 weeks were comparable to controls, however by 3-4 weeks they displayed scattered foci of F-actin within the BMC. At all time points thereafter, F-actin was rearranged into a 'rosette' pattern of prominent foci at cell vertices.The data are consistent with the hypothesis that migration of basal fiber ends is altered in a two stage process wherein initially, migration patterns undergo a rapid shift resulting in abnormal suture sub-branch formation. Subsequent cytological alterations are consistent with an eventual cessation of migration, precluding proper targeting of basal ends to their sutural destinations and leading to cataract plaque formation.

Project description:To determine whether an artificial intelligence candidate could pass the rapid (radiographic) reporting component of the Fellowship of the Royal College of Radiologists (FRCR) examination. Prospective multi-reader diagnostic accuracy study. United Kingdom. One artificial intelligence candidate (Smarturgences, Milvue) and 26 radiologists who had passed the FRCR examination in the preceding 12 months. Accuracy and pass rate of the artificial intelligence compared with radiologists across 10 mock FRCR rapid reporting examinations (each examination containing 30 radiographs, requiring 90% accuracy rate to pass). When non-interpretable images were excluded from the analysis, the artificial intelligence candidate achieved an average overall accuracy of 79.5% (95% confidence interval 74.1% to 84.3%) and passed two of 10 mock FRCR examinations. The average radiologist achieved an average accuracy of 84.8% (76.1-91.9%) and passed four of 10 mock examinations. The sensitivity for the artificial intelligence was 83.6% (95% confidence interval 76.2% to 89.4%) and the specificity was 75.2% (66.7% to 82.5%), compared with summary estimates across all radiologists of 84.1% (81.0% to 87.0%) and 87.3% (85.0% to 89.3%). Across 148/300 radiographs that were correctly interpreted by >90% of radiologists, the artificial intelligence candidate was incorrect in 14/148 (9%). In 20/300 radiographs that most (>50%) radiologists interpreted incorrectly, the artificial intelligence candidate was correct in 10/20 (50%). Most imaging pitfalls related to interpretation of musculoskeletal rather than chest radiographs. When special dispensation for the artificial intelligence candidate was provided (that is, exclusion of non-interpretable images), the artificial intelligence candidate was able to pass two of 10 mock examinations. Potential exists for the artificial intelligence candidate to improve its radiographic interpretation skills by focusing on musculoskeletal cases and learning to interpret radiographs of the axial skeleton and abdomen that are currently considered "non-interpretable."

Dataset Information

Performance of the Ebel standard-setting method in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions.

Publications

Performance of the Ebel standard-setting method in spring 2019 Royal College of Physicians and Surgeons of Canada internal medicine certification examination consisted of multiple-choice questions.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets