Dataset Information

The dialysis orders objective structured clinical examination (OSCE): a formative assessment for nephrology fellows.

ABSTRACT:

Background

Few quantitative nephrology-specific simulations assess fellow competency. We describe the development and initial validation of a formative objective structured clinical examination (OSCE) assessing fellow competence in ordering acute dialysis.

Methods

The three test scenarios were acute continuous renal replacement therapy, chronic dialysis initiation in moderate uremia and acute dialysis in end-stage renal disease-associated hyperkalemia. The test committee included five academic nephrologists and four clinically practicing nephrologists outside of academia. There were 49 test items (58 points). A passing score was 46/58 points. No item had median relevance less than 'important'. The content validity index was 0.91. Ninety-five percent of positive-point items were easy-medium difficulty. Preliminary validation was by 10 board-certified volunteers, not test committee members, a median of 3.5?years from graduation. The mean score was 49 [95% confidence interval (CI) 46-51], ??=?0.68 (95% CI 0.59-0.77), Cronbach's ??=?0.84.

Results

We subsequently administered the test to 25 fellows. The mean score was 44 (95% CI 43-45); 36% passed the test. Fellows scored significantly less than validators (P?ConclusionsThe OSCE may be used to formatively assess fellow proficiency in three common areas of acute dialysis practice. Further validation studies are in progress.

SUBMITTER: Prince LK

PROVIDER: S-EPMC5887504 | biostudies-literature | 2018 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

The dialysis orders objective structured clinical examination (OSCE): a formative assessment for nephrology fellows.

Prince Lisa K LK Campbell Ruth C RC Gao Sam W SW Kendrick Jessica J Lebrun Christopher J CJ Little Dustin J DJ Mahoney David L DL Maursetter Laura A LA Nee Robert R Saddler Mark M Watson Maura A MA Yuan Christina M CM

Clinical kidney journal 20170823 2

<h4>Background</h4>Few quantitative nephrology-specific simulations assess fellow competency. We describe the development and initial validation of a formative objective structured clinical examination (OSCE) assessing fellow competence in ordering acute dialysis.<h4>Methods</h4>The three test scenarios were acute continuous renal replacement therapy, chronic dialysis initiation in moderate uremia and acute dialysis in end-stage renal disease-associated hyperkalemia. The test committee included ...[more]

PMID: 29644053

Similar Datasets

Project description:Background and objectivesAcute kidney replacement therapy (KRT) prescription is a critical nephrology skill. We administered a formative objective structured clinical examination (OSCE) to nephrology fellows to assess acute KRT medical knowledge, patient care, and systems-based practice competencies.Design, setting, participants, & measurementsProspective cohort study of an educational test using the unified model of construct validity. We tested 117 fellows: 25 (four programs) in 2016 and 92 (15 programs) in 2017; 51 first-year and 66 second-year fellows. Using institutional protocols and order sets, fellows wrote orders and answered open-ended questions on a three-scenario OSCE, previously validated by board-certified, practicing clinical nephrologists. Outcomes were overall and scenario pass percentage and score; percent correctly answering predetermined, evidence-based questions; second-year score correlation with in-training examination score; and satisfaction survey.ResultsA total of 76% passed scenario 1 (acute continuous RRT): 92% prescribed a ≥20 ml/kg per hour effluent dose; 63% estimated clearance as effluent volume. Forty-two percent passed scenario 2 (maintenance dialysis initiation); 75% correctly prescribed 3-4 mEq/L K+ dialysate and 12% identified the two absolute, urgent indications for maintenance dialysis initiation (uremic encephalopathy and pericarditis). Six percent passed scenario 3 (acute life-threatening hyperkalemia); 20% checked for rebound hyperkalemia with two separate blood draws. Eighty-three percent correctly withheld intravenous sodium bicarbonate for acute hyperkalemia in a nonacidotic, volume-overloaded patient on maintenance dialysis, and 32% passed overall. Second-year versus first-year fellow overall score was 44.4±4 versus 42.7±5 (one-tailed P=0.02), with 39% versus 24% passing (P=0.08). Second-year in-training examination and OSCE scores were not significantly correlated (r=0.15; P=0.26). Seventy-seven percent of fellows agreed the OSCE was useful in assessing "proficiency in ordering" acute KRT. Limitations include lack of a validated criterion test, and unfamiliarity with open-ended question format.ConclusionsThe OSCE can provide quantitative data for formative Accreditation Council for Graduate Medical Education competency assessments and identify opportunities for dialysis curriculum development.PodcastThis article contains a podcast at https://www.asn-online.org/media/podcast/CJASN/2019_08_08_CJASNPodcast_19_09_.mp3.

Project description:Background: We previously described a nephrology-specific “Breaking Bad News” Objective Structured Clinical Examination (OSCE) assessing nephrology fellow communication and counseling skills in 3 scenarios: kidney replacement therapy (KRT) in kidney failure, urgent KRT in acute kidney injury (AKI), and kidney biopsy (KBx). Objective: The main objectives of this study is to adapt the OSCE to a virtual platform, simulating nephrology patient telemedicine encounters involving difficult conversations, and to assess fellow and faculty satisfaction with the virtual format. Design: Description of a formative telemedicine simulation for nephrology fellows. Setting: Fully virtual simulation conducted by 2 academic medical simulation centers. Participants: Nephrology faculty and fellows at 3 urban/suburban training programs in the eastern United States. Measurements: Description of the virtual OSCE process. Fellow and faculty satisfaction overall and for each scenario. Faculty and fellow estimates of frequency of virtual patient encounters in the past year. Methods: The OSCE consisted of 3 scenarios: KRT in kidney failure, urgent KRT in AKI, and KBx. Objective Structured Clinical Examinations were administered in May 2021. Each scenario lasted 20 minutes. The AKI scenario was audio only. Fellows telephoned a simulated patient surrogate for urgent KRT consent. Kidney failure and KBx scenarios were video encounters. Faculty observed while muted/video off. Immediately after the OSCE, fellows and faculty were anonymously surveyed regarding their satisfaction with each scenario, the OSCE overall, and their estimate of outpatient encounters and inpatient KRT counseling done virtually in the preceding year. Results: Seventeen fellows completed the OSCE at 2 centers (3 programs). Sixteen (94%) completed the survey. Almost 94% rated the OSCE as a good/very good approximation of telemedicine encounters. Those satisfied/very satisfied with each scenario are as follow: 100% for AKI, 75% for kidney failure, and 75% for KBx. Two commented that they often did urgent KRT counseling by telephone. Fellows estimated a median 20% (interquartile range: 175, 50%) of counseling for acute inpatient KRT and a median 50% (IQR: 33.75, 70%) of outpatient encounters were virtual in the prior year. Two (regarding the kidney failure and KBx scenarios) indicated they would not have counseled similar outpatients virtually. Limitations: The 15-minute interactions may be too short to allow the encounter to be completed comfortably. A small number of programs and fellows participated, and programs were located in urban/suburban areas on the east coast of the United States. Conclusions: Overall, fellows felt that the OSCE was a good approximation of virtual encounters. The OSCE is an opportunity for fellows to practice telemedicine communication skills.

Project description:This study aimed to investigate the impact of student e-learning on the development of clinical competencies. The study participants were 3rd year students (n = 43) at a private mid-sized medical school located in a South Korean suburb on a four-year medical program. Educational intervention was implemented to enhance student clinical performance. Students engaged in learning activities that intended to promote their self-directed learning abilities and clinical performances using e-learning resources. Intervention was conducted for the duration of six months during the 3rd year and its effectiveness was investigated by comparing student performances in OSCEs in a pre- and post- comparison format and also by comparing them with national scores. In addition, student perceptions of the impact of e-learning on their OSCE performances were assessed using a questionnaire, which included 36 items that elicited student perceptions of their experiences of e-learning and readiness for e-learning. Student OSCE scores improved significantly after educational intervention in all domains of clinical competencies assessed and for total scores (p < 0.001). Furthermore, students achieved higher OSCE scores than national average scores in the post-test, whereas they had performed lower than national average scores in the pre-test. Students showed neutral or slightly positive responses to the effectiveness of e-learning, and their perceptions of e-learning were not associated with their e-learning readiness scores. The study shows student OSCE performance improved significantly after educational intervention, which indicate the effectiveness of e-learning to support student learning of clinical performance. Despite significant improvements in student OSCE scores after e-learning, their perceptions of its effectiveness were neutral. Furthermore, student perceptions of e-learning were not associated with their readiness for it. Suggestions are made to help students use e-learning more effectively to enhance their clinical competencies.

Project description:PurposeEnsuring that examiners in different parallel circuits of objective structured clinical examinations (OSCEs) judge to the same standard is critical to the chain of validity. Recent work suggests examiner-cohort (i.e., the particular group of examiners) could significantly alter outcomes for some candidates. Despite this, examiner-cohort effects are rarely examined since fully nested data (i.e., no crossover between the students judged by different examiner groups) limit comparisons. In this study, the authors aim to replicate and further develop a novel method called Video-based Examiner Score Comparison and Adjustment (VESCA), so it can be used to enhance quality assurance of distributed or national OSCEs.MethodIn 2019, 6 volunteer students were filmed on 12 stations in a summative OSCE. In addition to examining live student performances, examiners from 8 separate examiner-cohorts scored the pool of video performances. Examiners scored videos specific to their station. Video scores linked otherwise fully nested data, enabling comparisons by Many Facet Rasch Modeling. Authors compared and adjusted for examiner-cohort effects. They also compared examiners' scores when videos were embedded (interspersed between live students during the OSCE) or judged later via the Internet.ResultsHaving accounted for differences in students' ability, different examiner-cohort scores for the same ability of student ranged from 18.57 of 27 (68.8%) to 20.49 (75.9%), Cohen's d = 1.3. Score adjustment changed the pass/fail classification for up to 16% of students depending on the modeled cut score. Internet and embedded video scoring showed no difference in mean scores or variability. Examiners' accuracy did not deteriorate over the 3-week Internet scoring period.ConclusionsExaminer-cohorts produced a replicable, significant influence on OSCE scores that was unaccounted for by typical assessment psychometrics. VESCA offers a promising means to enhance validity and fairness in distributed OSCEs or national exams. Internet-based scoring may enhance VESCA's feasibility.

Dataset Information

The dialysis orders objective structured clinical examination (OSCE): a formative assessment for nephrology fellows.

Background

Methods

Results

Publications

The dialysis orders objective structured clinical examination (OSCE): a formative assessment for nephrology fellows.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets