Dataset Information

Validation of a detailed scoring checklist for use during advanced cardiac life support certification.

ABSTRACT: Defining valid, reliable, defensible, and generalizable standards for the evaluation of learner performance is a key issue in assessing both baseline competence and mastery in medical education. However, before setting these standards of performance, the reliability of the scores yielding from a grading tool must be assessed. Accordingly, the purpose of this study was to assess the reliability of scores generated from a set of grading checklists used by nonexpert raters during simulations of American Heart Association (AHA) Megacodes.The reliability of scores generated from a detailed set of checklists, when used by 4 nonexpert raters, was tested by grading team leader performance in 8 Megacode scenarios. Videos of the scenarios were reviewed and rated by trained faculty facilitators and a group of nonexpert raters. The videos were reviewed "continuously" and "with pauses." The grading made by 2 content experts served as the reference standard, and 4 nonexpert raters were used to test the reliability of the checklists.Our results demonstrate that nonexpert raters are able to produce reliable grades when using the checklists under consideration, demonstrating excellent intrarater reliability and agreement with a reference standard. The results also demonstrate that nonexpert raters can be trained in the proper use of the checklist in a short amount of time, with no discernible learning curve thereafter. Finally, our results show that a single trained rater can achieve reliable scores of team leader performance during AHA Megacodes when using our checklist in a continuous mode because measures of agreement in total scoring were very strong [Lin's (Biometrics 1989;45:255-268) concordance correlation coefficient, 0.96; intraclass correlation coefficient, 0.97].We have shown that our checklists can yield reliable scores, are appropriate for use by nonexpert raters, and are able to be used during continuous assessment of team leader performance during the review of a simulated Megacode. This checklist may be more appropriate for use by advanced cardiac life support instructors during Megacode assessments than the current tools provided by the AHA.

SUBMITTER: McEvoy MD

PROVIDER: S-EPMC3467004 | biostudies-literature | 2012 Aug

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Validation of a detailed scoring checklist for use during advanced cardiac life support certification.

McEvoy Matthew D MD Smalley Jeremy C JC Nietert Paul J PJ Field Larry C LC Furse Cory M CM Blenko John W JW Cobb Benjamin G BG Walters Jenna L JL Pendarvis Allen A Dalal Nishita S NS Schaefer John J JJ

Simulation in healthcare : journal of the Society for Simulation in Healthcare 20120801 4

<h4>Introduction</h4>Defining valid, reliable, defensible, and generalizable standards for the evaluation of learner performance is a key issue in assessing both baseline competence and mastery in medical education. However, before setting these standards of performance, the reliability of the scores yielding from a grading tool must be assessed. Accordingly, the purpose of this study was to assess the reliability of scores generated from a set of grading checklists used by nonexpert raters duri ...[more]

PMID: 22863996

Similar Datasets

Project description:BackgroundEffective and timely delivery of cardiac arrest interventions during in-hospital cardiac arrest resuscitation is associated with greater survival. Whether a mobile app that provides timely reminders of critical interventions improves adherence to Advanced Cardiovascular Life Support (ACLS) guidelines among house officers, who may lack experience in leading resuscitations, remains unknown.ObjectiveThe aim of this study was to assess the impact of a commercially available, dynamic mobile app on house officers' adherence to ACLS guidelines.MethodsAs part of a quality improvement initiative, internal medicine house officers were invited to participate and randomized to lead 2 consecutive cardiac arrest simulations, one with a novel mobile app and one without a novel mobile app. All simulations included 4 cycles of cardiopulmonary resuscitation with different cardiac arrest rhythms and were video recorded. The coprimary end points were chest compression fraction and number of correct interventions in each simulation. The secondary end point was incorrect interventions, defined as interventions not indicated by the 2015 ACLS guidelines. Paired t tests compared performance with and without the mobile app.ResultsAmong 53 house officers, 26 house officers were randomized to lead the first simulation with the mobile app, and 27 house officers were randomized to do so without the app. Use of the mobile app was associated with a higher number of correct ACLS interventions (out of 7; mean 6.2 vs 5.1; absolute difference 1.1 [95% CI 0.6 to 1.6]; P<.001) as well as fewer incorrect ACLS interventions (mean 0.3 vs 1.0; absolute difference -0.7 [95% CI -0.3 to -1.0]; P<.001). Simulations with the mobile app also had a marginally higher chest compression fraction (mean 90.9% vs 89.0%; absolute difference 1.9% [95% CI 0.6% to 3.4%]; P=.007).ConclusionsThis proof-of-concept study suggests that this novel mobile app may improve adherence to ACLS protocols, but its effectiveness on survival in real-world resuscitations remains unknown.

Dataset Information

Validation of a detailed scoring checklist for use during advanced cardiac life support certification.

Publications

Validation of a detailed scoring checklist for use during advanced cardiac life support certification.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets