Dataset Information

Classification of 12-lead ECGs: the PhysioNet/Computing in Cardiology Challenge 2020.

ABSTRACT:

Objective

Vast 12-lead ECGs repositories provide opportunities to develop new machine learning approaches for creating accurate and automatic diagnostic systems for cardiac abnormalities. However, most 12-lead ECG classification studies are trained, tested, or developed in single, small, or relatively homogeneous datasets. In addition, most algorithms focus on identifying small numbers of cardiac arrhythmias that do not represent the complexity and difficulty of ECG interpretation. This work addresses these issues by providing a standard, multi-institutional database and a novel scoring metric through a public competition: the PhysioNet/Computing in Cardiology Challenge 2020.

Approach

A total of 66 361 12-lead ECG recordings were sourced from six hospital systems from four countries across three continents; 43 101 recordings were posted publicly with a focus on 27 diagnoses. For the first time in a public competition, we required teams to publish open-source code for both training and testing their algorithms, ensuring full scientific reproducibility.

Main results

A total of 217 teams submitted 1395 algorithms during the Challenge, representing a diversity of approaches for identifying cardiac abnormalities from both academia and industry. As with previous Challenges, high-performing algorithms exhibited significant drops ([Formula: see text]10%) in performance on the hidden test data.

Significance

Data from diverse institutions allowed us to assess algorithmic generalizability. A novel evaluation metric considered different misclassification errors for different cardiac abnormalities, capturing the outcomes and risks of different diagnoses. Requiring both trained models and code for training models improved the generalizability of submissions, setting a new bar in reproducibility for public data science competitions.

SUBMITTER: Perez Alday EA

PROVIDER: S-EPMC8015789 | biostudies-literature |

REPOSITORIES: biostudies-literature

ACCESS DATA

Similar Datasets

Project description:INTRODUCTION: We describe initial validation of a new system for digital to analog conversion (DAC) and reconstruction of 12-lead ECGs. The system utilizes an open and optimized software format with a commensurately optimized DAC hardware configuration to accurately reproduce, from digital files, the original analog electrocardiographic signals of previously instrumented patients. By doing so, the system also ultimately allows for transmission of data collected on one manufacturer's 12-lead ECG hardware/software into that of any other. MATERIALS AND METHODS: To initially validate the system, we compared original and post-DAC re-digitized 12-lead ECG data files (?5-minutes long) in two types of validation studies in 10 patients. The first type quantitatively compared the total waveform voltage differences between the original and re-digitized data while the second type qualitatively compared the automated electrocardiographic diagnostic statements generated by the original versus re-digitized data. RESULTS: The grand-averaged difference in root mean squared voltage between the original and re-digitized data was 20.8 µV per channel when re-digitization involved the same manufacturer's analog to digital converter (ADC) as the original digitization, and 28.4 µV per channel when it involved a different manufacturer's ADC. Automated diagnostic statements generated by the original versus reconstructed data did not differ when using the diagnostic algorithm from the same manufacturer on whose device the original data were collected, and differed only slightly for just 1 of 10 patients when using a third-party diagnostic algorithm throughout. CONCLUSION: Original analog 12-lead ECG signals can be reconstructed from digital data files with accuracy sufficient for clinical use. Such reconstructions can readily enable automated second opinions for difficult-to-interpret 12-lead ECGs, either locally or remotely through the use of dedicated or cloud-based servers.