Performance metrics for online seizure prediction.
Ontology highlight
ABSTRACT:
SUBMITTER: Chen HH
PROVIDER: S-EPMC7340210 | biostudies-literature |
REPOSITORIES: biostudies-literature
Similar Datasets
Project description:Assay Performance Metrics for Low RNA Inputs and Single Cells
2012-02-27 | GSE34363 | GEO
Project description:The development of seizure prediction models is often based on long-term scalp electroencephalograms (EEGs) since they capture brain electrical activity, are non-invasive, and come at a relatively low-cost. However, they suffer from major shortcomings. First, long-term EEG is usually highly contaminated with artefacts. Second, changes in the EEG signal over long intervals, known as concept drift, are often neglected. We evaluate the influence of these problems on deep neural networks using EEG time series and on shallow neural networks using widely-used EEG features. Our patient-specific prediction models were tested in 1577 hours of continuous EEG, containing 91 seizures from 41 patients with temporal lobe epilepsy who were undergoing pre-surgical monitoring. Our results showed that cleaning EEG data, using a previously developed artefact removal method based on deep convolutional neural networks, improved prediction performance. We also found that retraining the models over time reduced false predictions. Furthermore, the results show that although deep neural networks processing EEG time series are less susceptible to false alarms, they may need more data to surpass feature-based methods. These findings highlight the importance of robust data denoising and periodic adaptation of seizure prediction models.
| S-EPMC10090199 | biostudies-literature
Project description:Software is a complex entity, and its development needs careful planning and a high amount of time and cost. To assess quality of program, software measures are very helpful. Amongst the existing measures, coupling is an important design measure, which computes the degree of interdependence among the entities of a software system. Higher coupling leads to cognitive complexity and thus a higher probability occurrence of faults. Well in time prediction of fault-prone modules assists in saving time and cost of testing. This paper aims to capture important aspects of coupling and then assess the effectiveness of these aspects in determining fault-prone entities in the software system. We propose two coupling metrics, i.e., Vovel-in and Vovel-out, that capture the level of coupling and the volume of information flow. We empirically evaluate the effectiveness of the Vovel metrics in determining the fault-prone classes using five projects, i.e., Eclipse JDT, Equinox framework, Apache Lucene, Mylyn, and Eclipse PDE UI. Model building is done using univariate logistic regression and later Spearman correlation coefficient is computed with the existing coupling metrics to assess the coverage of unique information. Finally, the least correlated metrics are used for building multivariate logistic regression with and without the use of Vovel metrics, to assess the effectiveness of Vovel metrics. The results show the proposed metrics significantly improve the predicting of fault prone classes. Moreover, the proposed metrics cover a significant amount of unique information which is not covered by the existing well-known coupling metrics, i.e., CBO, RFC, Fan-in, and Fan-out. This paper, empirically evaluates the impact of coupling metrics, and more specifically the importance of level and volume of coupling in software fault prediction. The results advocate the prudent addition of proposed metrics due to their unique information coverage and significant predictive ability.
| S-EPMC8205299 | biostudies-literature
Project description:Focus on predictive algorithm and its performance evaluation is extensively covered in most research studies to determine best or appropriate predictive model with Optimum prediction solution indicated by prediction accuracy score, precision, recall, f1score etc. Prediction accuracy score from performance evaluation has been used extensively as the main determining metric for performance recommendation. It is one of the most widely used metric for identifying optimal prediction solution irrespective of dataset class distribution context or nature of dataset and output class distribution between the minority and majority variables. The key research question however is the impact of class inequality on prediction accuracy score in such datasets with output class distribution imbalance as compared to balanced accuracy score in the determination of model performance in healthcare and other real-world application systems. Answering this question requires an appraisal of current state of knowledge in both prediction accuracy score and balanced accuracy score use in real-world applications where there is unequal class distribution. Review of related works that highlight the use of imbalanced class distribution datasets with evaluation metrics will assist in contextualizing this systematic review.
| S-EPMC10688675 | biostudies-literature
Project description:Epilepsy is a chronic non-communicable disorder of the brain that affects individuals of all ages. It is caused by a sudden abnormal discharge of brain neurons leading to temporary dysfunction. In this regard, if seizures could be predicted a reasonable period of time before their occurrence, epilepsy patients could take precautions against them and improve their safety and quality of life. However, the potential that permutation entropy(PE) can be applied in human epilepsy prediction from intracranial electroencephalogram (iEEG) recordings remains unclear. Here, we described the novel application of PE to track the dynamical changes of human brain activity from iEEG recordings for the epileptic seizure prediction. The iEEG signals of 19 patients were obtained from the Epilepsy Centre at the University Hospital of Freiburg. After preprocessing, PE was extracted in a sliding time window, and a support vector machine (SVM) was employed to discriminate cerebral state. Then a two-step post-processing method was applied for the purpose of prediction. The results showed that we obtained an average sensitivity (SS) of 94% and false prediction rates (FPR) with 0.111 h-1. The best results with SS of 100% and FPR of 0 h-1 were achieved for some patients. The average prediction horizon was 61.93 min, leaving sufficient treatment time before a seizure. These results indicated that applying PE as a feature to extract information and SVM for classification could predict seizures, and the presented method shows great potential in clinical seizure prediction for human.
| S-EPMC6060283 | biostudies-literature
Project description:Building accurate rainfall-runoff models is an integral part of hydrological science and practice. The variety of modeling goals and applications have led to a large suite of evaluation metrics for these models. Yet, hydrologists still put considerable trust into visual judgment, although it is unclear whether such judgment agrees or disagrees with existing quantitative metrics. In this study, we tasked 622 experts to compare and judge more than 14,000 pairs of hydrographs from 13 different models. Our results show that expert opinion broadly agrees with quantitative metrics and results in a clear preference for a Machine Learning model over traditional hydrological models. The expert opinions are, however, subject to significant amounts of inconsistency. Nevertheless, where experts agree, we can predict their opinion purely from quantitative metrics, which indicates that the metrics sufficiently encode human preferences in a small set of numbers. While there remains room for improvement of quantitative metrics, we suggest that the hydrologic community should reinforce their benchmarking efforts and put more trust in these metrics.
| S-EPMC10909531 | biostudies-literature
Project description:A subset of patients with epilepsy successfully self-predicted seizures in a paper diary study. We conducted an e-diary study to ensure that prediction precedes seizures, and to characterize the prodromal features and time windows that underlie self-prediction.Subjects 18 or older with localization-related epilepsy (LRE) and ?3 seizures per month maintained an e-diary, reporting a.m./p.m. data daily, including mood, premonitory symptoms, and all seizures. Self-prediction was rated by, "How likely are you to experience a seizure (time frame)?" Five choices ranged from almost certain (>95% chance) to very unlikely. Relative odds of seizure (odds ratio, OR) within time frames was examined using Poisson models with log normal random effects to adjust for multiple observations.Nineteen subjects reported 244 eligible seizures. OR for prediction choices within 6 h was as high as 9.31 (CI 1.92-45.23) for "almost certain." Prediction was most robust within 6 h of diary entry, and remained significant up to 12 h. For nine best predictors, average sensitivity was 50%. Older age contributed to successful self-prediction, and self-prediction appeared to be driven by mood and premonitory symptoms. In multivariate modeling of seizure occurrence, self-prediction (2.84; CI 1.68-4.81), favorable change in mood (0.82; CI 0.67-0.99), and number of premonitory symptoms (1.11; CI 1.00-1.24) were significant.Some persons with epilepsy can self-predict seizures. In these individuals, the odds of a seizure following a positive prediction are high. Predictions were robust, not attributable to recall bias, and were related to self-awareness of mood and premonitory features. The 6-h prediction window is suitable for the development of preemptive therapy.
| S-EPMC3833277 | biostudies-literature
Project description:In the context of the COVID-19 global pandemic, highly intense and frequent online teaching has leapt to be one of the dominant learning patterns and become an ordinary situation in university teaching practices. In recent years, progress in feature engineering and machine learning has made it possible for more effective educational data mining, which in turn has enhanced the performance of intelligent learning models. However, the potential impact of increasing and varying features on online instruction in this new situation makes it unclear whether the existing related findings and results are practical for teachers. In this article, we use various state-of-the-art machine learning techniques to predict students' performance. Based on the validation of the rationality of the built models, the importance of features under different feature selection techniques are calculated separately for the datasets of two groups and compared with the features before and at the beginning of the pandemic. The results show that in the current new state of highly intense online learning, without considering student information such as demographic information, campus attributes (administrative class and teaching class) and learning behavior (completion of online learning tasks and stage tests) these dynamic features are more likely to discriminate students' academic performances, which deserves more attention than demographics for teachers in the guidance of students' learning. In addition, it is suggested that further improvements and refinements should be made to the existing features, such as classifying features more precisely and expanding in these feature categories, and taking into account the statistics about students' in-class performances as well as their subjective understanding of what they have learned. Our findings are in line with the new situation under the pandemic and provide more implications to teachers' teaching guidance.
| S-EPMC10703107 | biostudies-literature
Project description: Not available
| S-EPMC8531996 | biostudies-literature
Project description:ImportanceIn 2020, the Centers for Medicare & Medicaid Services (CMS) issued a final rule updating the Organ Procurement Organization (OPO) Conditions for Coverage. This rule evaluates OPO performance based on an unadjusted donation rate and an age-adjusted transplant rate; however, neither considers other underlying population differences.ObjectiveTo evaluate whether adjusting for age and/or area deprivation index yields the same tier assignments as the cause, age, and location consistent (CALC) tier used by CMS.Design, setting, and participantsThis retrospective cross-sectional study examined the performance of 58 OPOs from 2018 to 2020 across the entire US. A total of 12 041 778 death records were examined from the 2017 to 2020 National Center for Health Statistics' Restricted Vital Statistics Detailed Multiple Cause of Death files; 399 530 of these met the definition of potential deceased donor. Information about 42 572 solid organ donors from the Scientific Registry of Transplant Recipients was also used. Statistical analysis was performed from January 2017 to December 2020.ExposureArea deprivation of donation service areas and age of potential donors.Main outcome and measuresOPO performance as measured by donation and transplant rates.ResultsA total of 399 530 potential deceased donors and 42 572 actual solid donor organs were assigned to 1 of 58 OPOs. Age and ADI adjustment resulted in 19.0% (11 of 58) to 31.0% (18 of 58) reclassification of tier ratings for the OPOs, with 46.6% of OPOs (27 of 58) changing tier ranking at least once during the 3-year period. Between 6.9% (4 of 58) and 12.1% (7 of 58) moved into tier 1 and up to 8.6% (5 of 58) moved into tier 3.Conclusions and relevanceThis cross-sectional study of population characteristics and OPO performance metrics found that adjusting for area deprivation and age significantly changed OPO measured performance and tier classifications. These findings suggest that underlying population characteristics may alter processes of care and characterize donation and transplant rates independent of OPO performance. Risk adjustment accounting for population characteristics warrants consideration in prospective policy and further evaluation of quality metrics.
| S-EPMC10548299 | biostudies-literature