Dataset Information

Predicting electrocardiogram interpretation performance in Advanced Cardiovascular Life Support simulation: comparing knowledge tests and simulation performance among Mexican medical students.

ABSTRACT: Background:Simulation plays a key role in assessing performance in Advanced Cardiovascular Life Support (ACLS). Traditional knowledge tests are also important for assessing the cognitive elements of ACLS performance. However, the association between the two has not been established. In this study, we focus on one important element in ACLS-interpretation of electrocardiograms (ECG)-and the potential of knowledge tests to serve as predictors of improvement in ACLS performance. Methods:We looked at the correlation between Mexican medical students' improvement in ECG interpretation performance in ACLS megacode simulations (from the start of the semester to the end of the semester), and their scores on ECG interpretation knowledge tests. Results:We found significant improvement in ECG interpretation in ACLS megacode simulation (from pre-semester to post-semester), but this was not predicted by the ECG interpretation knowledge test scores. The correlation was .079 (p = 0.66). Conclusions:These results suggest that even cognitive tasks such as ECG interpretation can be expressed and assessed differently in simulation versus traditional knowledge testing.

SUBMITTER: Smith MW

PROVIDER: S-EPMC6422014 | biostudies-literature | 2019

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Predicting electrocardiogram interpretation performance in Advanced Cardiovascular Life Support simulation: comparing knowledge tests and simulation performance among Mexican medical students.

Smith Michael William MW Abarca Rondero David D

PeerJ 20190315

<h4>Background</h4>Simulation plays a key role in assessing performance in Advanced Cardiovascular Life Support (ACLS). Traditional knowledge tests are also important for assessing the cognitive elements of ACLS performance. However, the association between the two has not been established. In this study, we focus on one important element in ACLS-interpretation of electrocardiograms (ECG)-and the potential of knowledge tests to serve as predictors of improvement in ACLS performance.<h4>Methods</ ...[more]

PMID: 30891369

Similar Datasets

Project description:High-fidelity simulation (HFS) is a learning method which has proven effective in medical education for technical and non-technical skills. However, its effectiveness for knowledge acquisition is less validated. We performed a randomized study with the primary aim of investigating whether HFS, in association with frontal lessons, would improve knowledge about advanced life support (ALS), in comparison to frontal lessons only among medical students. The secondary aims were to evaluate the effect of HFS on knowledge acquisition of different sections of ALS and personal knowledge perception. Participants answered a pre-test questionnaire consisting of a subjective (evaluating personal perception of knowledge) and an objective section (measuring level of knowledge) containing 100 questions about algorithms, technical skills, team working/early warning scores/communication strategies according to ALS guidelines. All students participated in 3 frontal lessons before being randomized in group S, undergoing a HFS session, and group C, receiving no further interventions. After 10 days from the end of each intervention, both groups answered a questionnaire (post-test) with the same subjective section but a different objective one. The overall number of correct answers of the post-test was significantly higher in group S (mean 74.1, SD 11.2) than in group C (mean 65.5, SD 14.3), p = 0.0017, 95% C.I. 3.34 - 13.9. A significantly higher number of correct answers was reported in group S than in group C for questions investigating knowledge of algorithms (p = 0.0001; 95% C.I 2.22-5.99) and team working/early warning scores/communication strategies (p = 0.0060; 95% C.I 1.13-6.53). Students in group S showed a significantly higher score in the post-test subjective section (p = 0.0074). A lower proportion of students in group S confirmed their perception of knowledge compared to group C (p = 0.0079). HFS showed a beneficial effect on knowledge of ALS among medical students, especially for notions of algorithms and team working/early warning scores/communication.

Project description:ImportanceMillions of clinicians rely daily on automated preliminary electrocardiogram (ECG) interpretation. Critical comparisons of machine learning-based automated analysis against clinically accepted standards of care are lacking.ObjectiveTo use readily available 12-lead ECG data to train and apply an explainability technique to a convolutional neural network (CNN) that achieves high performance against clinical standards of care.Design, setting, and participantsThis cross-sectional study was conducted using data from January 1, 2003, to December 31, 2018. Data were obtained in a commonly available 12-lead ECG format from a single-center tertiary care institution. All patients aged 18 years or older who received ECGs at the University of California, San Francisco, were included, yielding a total of 365 009 patients. Data were analyzed from January 1, 2019, to March 2, 2021.ExposuresA CNN was trained to predict the presence of 38 diagnostic classes in 5 categories from 12-lead ECG data. A CNN explainability technique called LIME (Linear Interpretable Model-Agnostic Explanations) was used to visualize ECG segments contributing to CNN diagnoses.Main outcomes and measuresArea under the receiver operating characteristic curve (AUC), sensitivity, and specificity were calculated for the CNN in the holdout test data set against cardiologist clinical diagnoses. For a second validation, 3 electrophysiologists provided consensus committee diagnoses against which the CNN, cardiologist clinical diagnosis, and MUSE (GE Healthcare) automated analysis performance was compared using the F1 score; AUC, sensitivity, and specificity were also calculated for the CNN against the consensus committee.ResultsA total of 992 748 ECGs from 365 009 adult patients (mean [SD] age, 56.2 [17.6] years; 183 600 women [50.3%]; and 175 277 White patients [48.0%]) were included in the analysis. In 91 440 test data set ECGs, the CNN demonstrated an AUC of at least 0.960 for 32 of 38 classes (84.2%). Against the consensus committee diagnoses, the CNN had higher frequency-weighted mean F1 scores than both cardiologists and MUSE in all 5 categories (CNN frequency-weighted F1 score for rhythm, 0.812; conduction, 0.729; chamber diagnosis, 0.598; infarct, 0.674; and other diagnosis, 0.875). For 32 of 38 classes (84.2%), the CNN had AUCs of at least 0.910 and demonstrated comparable F1 scores and higher sensitivity than cardiologists, except for atrial fibrillation (CNN F1 score, 0.847 vs cardiologist F1 score, 0.881), junctional rhythm (0.526 vs 0.727), premature ventricular complex (0.786 vs 0.800), and Wolff-Parkinson-White (0.800 vs 0.842). Compared with MUSE, the CNN had higher F1 scores for all classes except supraventricular tachycardia (CNN F1 score, 0.696 vs MUSE F1 score, 0.714). The LIME technique highlighted physiologically relevant ECG segments.Conclusions and relevanceThe results of this cross-sectional study suggest that readily available ECG data can be used to train a CNN algorithm to achieve comparable performance to clinical cardiologists and exceed the performance of MUSE automated analysis for most diagnoses, with some exceptions. The LIME explainability technique applied to CNNs highlights physiologically relevant ECG segments that contribute to the CNN's diagnoses.

Project description:BackgroundSerious games enrich simulation-based health care trainings and improve knowledge, skills, and self-confidence of learners while entertaining them.ObjectiveA platform which can combine performance data from a basic life support (BLS) serious game app and hands-on data based on the same scoring system is not available in the market. The aim of this study was to create such a platform and investigate whether performance evaluation of BLS trainings would be more objective compared to conventional Objective Structured Clinical Examination (OSCE) examinations if these evaluations were carried out with the platform which combines OSCE scoring criteria with sensor data retrieved from the simulator's sensors.MethodsParticipants were 25 volunteers (11 men [44.0%] and 14 [56.0] women) among Ac?badem Mehmet Ali Ayd?nlar University students without prior knowledge of the BLS protocol. A serious game module has been created for teaching learners the European Resuscitation Council Basic Life Support 2015 protocol. A second module called the hands-on module was designed for educators. This module includes a checklist used for BLS OSCE examinations and can retrieve sensor data such as compression depth, compression frequency, and ventilation volume from the manikin (CPR Lilly; 3B Scientific GmbH) via Bluetooth. Data retrieved from the sensors of the manikin enable educators to evaluate learners in a more objective way. Performance data retrieved from the serious gaming module have been combined with the results of the hands-on module. Data acquired from the hands-on module have also been compared with the results of conventional OSCE scores of the participants, which were obtained by watching the videos of the same trainings.ResultsParticipants were considered successful in the game if they scored 80/100 or above. Overall, participants scored 80 or above in an average of 1.4 (SD 0.65) trials. The average BLS serious game score was 88.3/100 (SD 5.17) and hands-on average score was 70.7/100 (SD 17.3), whereas the OSCE average score was 84.4/100 (SD 12.9). There was no statistically significant correlation between success on trials (score ?80/100), serious game, hands-on training app, and OSCE scores (Spearman rho test, P>.05). The mean BLS serious game score of the participants was 88.3/100 (SD 5.17), whereas their mean hands-on training app score was 70.7/100 (SD 17.3) and OSCE score was 84.4/100 (SD 12.9).ConclusionsAlthough scoring criteria for OSCE and hands-on training app were identical, OSCE scores were 17% higher than hands-on training app scores. After analyzing the difference of scores between hands-on training app and OSCE, it has been revealed that these differences originate from scoring parameters such as compression depth, compression frequency, and ventilation volume. These data suggest that evaluation of BLS trainings would be more objective if these evaluations were carried out with the modality, which combines visual OSCE scoring criteria with sensor data retrieved from the simulator's sensors.Trial registrationClinicalTrials.gov NCT04533893; https://clinicaltrials.gov/ct2/show/NCT04533893.

Dataset Information

Predicting electrocardiogram interpretation performance in Advanced Cardiovascular Life Support simulation: comparing knowledge tests and simulation performance among Mexican medical students.

Publications

Predicting electrocardiogram interpretation performance in Advanced Cardiovascular Life Support simulation: comparing knowledge tests and simulation performance among Mexican medical students.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets