Dataset Information

Crowdsourcing seizure detection: algorithm development and validation on human implanted device recordings.

ABSTRACT: There exist significant clinical and basic research needs for accurate, automated seizure detection algorithms. These algorithms have translational potential in responsive neurostimulation devices and in automatic parsing of continuous intracranial electroencephalography data. An important barrier to developing accurate, validated algorithms for seizure detection is limited access to high-quality, expertly annotated seizure data from prolonged recordings. To overcome this, we hosted a kaggle.com competition to crowdsource the development of seizure detection algorithms using intracranial electroencephalography from canines and humans with epilepsy. The top three performing algorithms from the contest were then validated on out-of-sample patient data including standard clinical data and continuous ambulatory human data obtained over several years using the implantable NeuroVista seizure advisory system. Two hundred teams of data scientists from all over the world participated in the kaggle.com competition. The top performing teams submitted highly accurate algorithms with consistent performance in the out-of-sample validation study. The performance of these seizure detection algorithms, achieved using freely available code and data, sets a new reproducible benchmark for personalized seizure detection. We have also shared a 'plug and play' pipeline to allow other researchers to easily use these algorithms on their own datasets. The success of this competition demonstrates how sharing code and high quality data results in the creation of powerful translational tools with significant potential to impact patient care.

SUBMITTER: Baldassano SN

PROVIDER: S-EPMC6075622 | biostudies-literature | 2017 Jun

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

Crowdsourcing seizure detection: algorithm development and validation on human implanted device recordings.

Baldassano Steven N SN Brinkmann Benjamin H BH Ung Hoameng H Blevins Tyler T Conrad Erin C EC Leyde Kent K Cook Mark J MJ Khambhati Ankit N AN Wagenaar Joost B JB Worrell Gregory A GA Litt Brian B

Brain : a journal of neurology 20170601 6

There exist significant clinical and basic research needs for accurate, automated seizure detection algorithms. These algorithms have translational potential in responsive neurostimulation devices and in automatic parsing of continuous intracranial electroencephalography data. An important barrier to developing accurate, validated algorithms for seizure detection is limited access to high-quality, expertly annotated seizure data from prolonged recordings. To overcome this, we hosted a kaggle.com ...[more]

PMID: 28459961

Similar Datasets

Project description:OBJECTIVE:Closed-loop implantable neural stimulators are an exciting treatment option for patients with medically refractory epilepsy, with a number of new devices in or nearing clinical trials. These devices must accurately detect a variety of seizure types in order to reliably deliver therapeutic stimulation. While effective, broadly-applicable seizure detection algorithms have recently been published, these methods are too computationally intensive to be directly deployed in an implantable device. We demonstrate a strategy that couples devices to cloud computing resources in order to implement complex seizure detection methods on an implantable device platform. APPROACH:We use a sensitive gating algorithm capable of running on-board a device to identify potential seizure epochs and transmit these epochs to a cloud-based analysis platform. A precise seizure detection algorithm is then applied to the candidate epochs, leveraging cloud computing resources for accurate seizure event detection. This seizure detection strategy was developed and tested on eleven human implanted device recordings generated using the NeuroVista Seizure Advisory System. MAIN RESULTS:The gating algorithm achieved high-sensitivity detection using a small feature set as input to a linear classifier, compatible with the computational capability of next-generation implantable devices. The cloud-based precision algorithm successfully identified all seizures transmitted by the gating algorithm while significantly reducing the false positive rate. Across all subjects, this joint approach detected 99% of seizures with a false positive rate of 0.03?h-1. SIGNIFICANCE:We present a novel framework for implementing computationally intensive algorithms on human data recorded from an implanted device. By using telemetry to intelligently access cloud-based computational resources, the next generation of neuro-implantable devices will leverage sophisticated algorithms with potential to greatly improve device performance and patient outcomes.

Project description:SEE MORMANN AND ANDRZEJAK DOI101093/BRAIN/AWW091 FOR A SCIENTIFIC COMMENTARY ON THIS ARTICLE : Accurate forecasting of epileptic seizures has the potential to transform clinical epilepsy care. However, progress toward reliable seizure forecasting has been hampered by lack of open access to long duration recordings with an adequate number of seizures for investigators to rigorously compare algorithms and results. A seizure forecasting competition was conducted on kaggle.com using open access chronic ambulatory intracranial electroencephalography from five canines with naturally occurring epilepsy and two humans undergoing prolonged wide bandwidth intracranial electroencephalographic monitoring. Data were provided to participants as 10-min interictal and preictal clips, with approximately half of the 60 GB data bundle labelled (interictal/preictal) for algorithm training and half unlabelled for evaluation. The contestants developed custom algorithms and uploaded their classifications (interictal/preictal) for the unknown testing data, and a randomly selected 40% of data segments were scored and results broadcasted on a public leader board. The contest ran from August to November 2014, and 654 participants submitted 17 856 classifications of the unlabelled test data. The top performing entry scored 0.84 area under the classification curve. Following the contest, additional held-out unlabelled data clips were provided to the top 10 participants and they submitted classifications for the new unseen data. The resulting area under the classification curves were well above chance forecasting, but did show a mean 6.54 ± 2.45% (min, max: 0.30, 20.2) decline in performance. The kaggle.com model using open access data and algorithms generated reproducible research that advanced seizure forecasting. The overall performance from multiple contestants on unseen data was better than a random predictor, and demonstrates the feasibility of seizure forecasting in canine and human epilepsy.media-1vid110.1093/brain/aww045_video_abstractaww045_video_abstract.

Project description:BackgroundHip fracture is the most common type of fracture in elderly individuals. Numerous deep learning (DL) algorithms for plain pelvic radiographs (PXRs) have been applied to improve the accuracy of hip fracture diagnosis. However, their efficacy is still undetermined.ObjectiveThe objective of this study is to develop and validate a human-algorithm integration (HAI) system to improve the accuracy of hip fracture diagnosis in a real clinical environment.MethodsThe HAI system with hip fracture detection ability was developed using a deep learning algorithm trained on trauma registry data and 3605 PXRs from August 2008 to December 2016. To compare their diagnostic performance before and after HAI system assistance using an independent testing dataset, 34 physicians were recruited. We analyzed the physicians' accuracy, sensitivity, specificity, and agreement with the algorithm; we also performed subgroup analyses according to physician specialty and experience. Furthermore, we applied the HAI system in the emergency departments of different hospitals to validate its value in the real world.ResultsWith the support of the algorithm, which achieved 91% accuracy, the diagnostic performance of physicians was significantly improved in the independent testing dataset, as was revealed by the sensitivity (physician alone, median 95%; HAI, median 99%; P<.001), specificity (physician alone, median 90%; HAI, median 95%; P<.001), accuracy (physician alone, median 90%; HAI, median 96%; P<.001), and human-algorithm agreement [physician alone κ, median 0.69 (IQR 0.63-0.74); HAI κ, median 0.80 (IQR 0.76-0.82); P<.001. With the help of the HAI system, the primary physicians showed significant improvement in their diagnostic performance to levels comparable to those of consulting physicians, and both the experienced and less-experienced physicians benefited from the HAI system. After the HAI system had been applied in 3 departments for 5 months, 587 images were examined. The sensitivity, specificity, and accuracy of the HAI system for detecting hip fractures were 97%, 95.7%, and 96.08%, respectively.ConclusionsHAI currently impacts health care, and integrating this technology into emergency departments is feasible. The developed HAI system can enhance physicians' hip fracture diagnostic performance.

Dataset Information

Crowdsourcing seizure detection: algorithm development and validation on human implanted device recordings.

Publications

Crowdsourcing seizure detection: algorithm development and validation on human implanted device recordings.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets