ASSESSING ROBUSTNESS OF CLASSIFICATION USING ANGULAR BREAKDOWN POINT.
Ontology highlight
ABSTRACT:
SUBMITTER: Zhao J
PROVIDER: S-EPMC6168219 | biostudies-literature | 2018 Dec
REPOSITORIES: biostudies-literature
Publications
ASSESSING ROBUSTNESS OF CLASSIFICATION USING ANGULAR BREAKDOWN POINT.
Zhao Junlong J Yu Guan G Liu Yufeng Y
Annals of statistics 20180911 6B
Robustness is a desirable property for many statistical techniques. As an important measure of robustness, breakdown point has been widely used for regression problems and many other settings. Despite the existing development, we observe that the standard breakdown point criterion is not directly applicable for many classification problems. In this paper, we propose a new breakdown point criterion, namely angular breakdown point, to better quantify the robustness of different classification meth ...[more]
Similar Datasets
Project description:There is an increasing awareness that replication should become common practice in empirical studies. However, study results might fail to replicate for various reasons. The robustness of published study results can be assessed using the relatively new multiverse-analysis methodology, in which the robustness of the effect estimates against data analytical decisions is assessed. However, the uptake of multiverse analysis in empirical studies remains low, which might be due to the scarcity of guidance available on performing multiverse analysis. Researchers might experience difficulties in identifying data analytical decisions and in summarizing the large number of effect estimates yielded by a multiverse analysis. These difficulties are amplified when applying multiverse analysis to assess the robustness of the effect estimates from a mediation analysis, as a mediation analysis involves more data analytical decisions than a bivariate analysis. The aim of this paper is to provide an overview and worked example of the use of multiverse analysis to assess the robustness of the effect estimates from a mediation analysis. We showed that the number of data analytical decisions in a mediation analysis is larger than in a bivariate analysis. By using a real-life data example from the Longitudinal Aging Study Amsterdam, we demonstrated the application of multiverse analysis to a mediation analysis. This included the use of specification curves to determine the impact of data analytical decisions on the magnitude and statistical significance of the direct, indirect, and total effect estimates. Although the multiverse analysis methodology is still relatively new and future research is needed to further advance this methodology, this paper shows that multiverse analysis is a useful method for the assessment of the robustness of the direct, indirect, and total effect estimates in a mediation analysis and thereby to inform replication studies.
| S-EPMC9283158 | biostudies-literature
Project description:In this work, we present an extensive dataset of laser-induced breakdown spectroscopy (LIBS) spectra for the pre-training and evaluation of LIBS classification models. LIBS is a well-established spectroscopic method for in-situ and industrial applications, where LIBS is primarily applied for clustering and classification tasks. As such, our dataset is aimed at helping with the development and testing of classification and clustering methodologies. Moreover, the dataset could be used to pre-train classification models for applications where the amount of available data is limited. The dataset consists of LIBS spectra of 138 soil samples belonging to 12 distinct classes. The spectra were acquired with a state-of-the-art LIBS system. Lastly, the composition of each sample is also provided, including estimated uncertainties.
| S-EPMC7018695 | biostudies-literature
Project description:Image features need to be robust against differences in positioning, acquisition and segmentation to ensure reproducibility. Radiomic models that only include robust features can be used to analyse new images, whereas models with non-robust features may fail to predict the outcome of interest accurately. Test-retest imaging is recommended to assess robustness, but may not be available for the phenotype of interest. We therefore investigated 18 combinations of image perturbations to determine feature robustness, based on noise addition (N), translation (T), rotation (R), volume growth/shrinkage (V) and supervoxel-based contour randomisation (C). Test-retest and perturbation robustness were compared for combined total of 4032 morphological, statistical and texture features that were computed from the gross tumour volume in two cohorts with computed tomography imaging: I) 31 non-small-cell lung cancer (NSCLC) patients; II): 19 head-and-neck squamous cell carcinoma (HNSCC) patients. Robustness was determined using the 95% confidence interval (CI) of the intraclass correlation coefficient (1, 1). Features with CI???0:90 were considered robust. The NTCV, TCV, RNCV and RCV perturbation chain produced similar results and identified the fewest false positive robust features (NSCLC: 0.2-0.9%; HNSCC: 1.7-1.9%). Thus, these perturbation chains may be used as an alternative to test-retest imaging to assess feature robustness.
| S-EPMC6345842 | biostudies-other
Project description:With adverse industrial effects on the global landscape, climate change is imploring the global economy to adopt sustainable solutions. The ongoing evolution of energy efficiency targets massive data collection and Artificial Intelligence (AI) for big data analytics. Besides, emerging on the Internet of Energy (IoE) paradigm, edge computing is playing a rising role in liberating private data from cloud centralization. In this direction, a creative visual approach to understanding energy data is introduced. Building upon micro-moments, which are timeseries of small contextual data points, the power of pictorial representations to encapsulate rich information in a small two-dimensional (2D) space is harnessed through a novel Gramian Angular Fields (GAF) classifier for energy micro-moments. Designed with edge computing efficiency in mind, current testing results on the ODROID-XU4 can classify up to 7 million GAF-converted datapoints with ~ 90% accuracy in less than 30 s, paving the path towards industrial adoption of edge IoE.Supplementary informationThe online version contains supplementary material available at 10.1007/s10586-022-03704-1.
| S-EPMC9387409 | biostudies-literature
Project description:BackgroundTranscriptional regulation is primarily mediated by the binding of factors to non-coding regions in DNA. Identification of these binding regions enhances understanding of tissue formation and potentially facilitates the development of gene therapies. However, successful identification of binding regions is made difficult by the lack of a universal biological code for their characterisation.ResultsWe extend an alignment-based method, changept, and identify clusters of biological significance, through ontology and de novo motif analysis. Further, we apply a Bayesian method to estimate and combine binary classifiers on the clusters we identify to produce a better performing composite.ConclusionsThe analysis we describe provides a computational method for identification of conserved binding sites in the human genome and facilitates an alternative interrogation of combinations of existing data sets with alignment data.
| S-EPMC8790847 | biostudies-literature
Project description:Five species of bacteria including Escherichia coli, Mycobacterium smegmatis, Pseudomonas aeruginosa, Staphylococcus epidermidis, and Enterobacter cloacae were deposited from suspensions of various titers onto disposable nitrocellulose filter media for analysis by laser-induced breakdown spectroscopy (LIBS). Bacteria were concentrated and isolated in the center of the filter media during centrifugation using a simple and convenient sample preparation step. Summing all the single-shot LIBS spectra acquired from a given bacterial deposition provided perfectly sensitive and specific discrimination from sterile water control specimens in a partial least squares discriminant analysis (PLS-DA). Use of the single-shot spectra provided only a 0.87 and 0.72 sensitivity and specificity, respectively. To increase the statistical validity of chemometric analyses, a library of pseudodata was created by adding Gaussian noise to the measured intensity of every emission line in an averaged spectrum of each bacterium. The normally distributed pseudodata, consisting of 4995 spectra, were used to compare the performance of the PLS-DA with a discriminant function analysis (DFA) and an artificial neural network (ANN). For the highly similar bacterial data, no algorithm showed significantly superior performance, although the PLS-DA performed least accurately with a classification error of 0.21 compared to 0.16 and 0.17 for ANN and DFA, respectively. Single-shot LIBS spectra from all of the bacterial species were classified in a DFA model tested with a tenfold cross-validation. Classification errors ranging from 20% to 31% were measured due to repeatability limitations in the single-shot data.
| S-EPMC9411782 | biostudies-literature
Project description:Recent work reframes direct effects of covariates on items in mixture models as differential item functioning (DIF) and shows that, when present in the data but omitted from the fitted latent class model, DIF can lead to overextraction of classes. However, less is known about the effects of DIF on model performance-including parameter bias, classification accuracy, and distortion of class-specific response profiles-once the correct number of classes is chosen. First, we replicate and extend prior findings relating DIF to class enumeration using a comprehensive simulation study. In a second simulation study using the same parameters, we show that, while the performance of LCA is robust to the misspecification of DIF effects, it is degraded when DIF is omitted entirely. Moreover, the robustness of LCA to omitted DIF differs widely based on the degree of class separation. Finally, simulation results are contextualized by an empirical example.
| S-EPMC7247772 | biostudies-literature
Project description:Spatial light modulator (SLM) is widely used in imaging applications for modulating light intensity and phase delay. In this paper, we report a novel device concept termed angular light modulator (ALM). Different from the SLM, the reported ALM employs a tunable blind structure to modulate the angular components of the incoming light waves. For spatial-domain light modulation, the ALM can be directly placed in front of an image sensor for selecting different angular light components. In this case, we can sweep the slat angle of the blind structure and capture multiple images corresponding to different perspectives. These images can then be back-projected for 3D tomographic refocusing. By using a fixed slat angle, we can also convert the incident-angle information into intensity variations for wavefront sensing or introduce a translational shift to the defocused object for high-speed autofocusing. For Fourier-domain light modulation, the ALM can be placed at the pupil plane of an optical system for reinforcing the light propagating trajectories. We show that a pupil-plane-modulated system is able to achieve a better resolution for out-of-focus objects while maintaining the same resolution for in-focus objects. The reported ALM can be fabricated on the chip level and controlled by an external magnetic field. It may provide new insights for developing novel imaging and vision devices.
| S-EPMC5234498 | biostudies-literature
Project description:The analysis of gene network robustness to noise and mutation is important for fundamental and practical reasons. Robustness refers to the stability of the equilibrium expression state of a gene network to variations of the initial expression state and network topology. Numerical simulation of these variations is commonly used for the assessment of robustness. Since there exists a great number of possible gene network topologies and initial states, even millions of simulations may be still too small to give reliable results. When the initial and equilibrium expression states are restricted to being saturated (i.e., their elements can only take values 1 or -1 corresponding to maximum activation and maximum repression of genes), an analytical gene network robustness assessment is possible. We present this analytical treatment based on determination of the saturated fixed point attractors for sigmoidal function models. The analysis can determine (a) for a given network, which and how many saturated equilibrium states exist and which and how many saturated initial states converge to each of these saturated equilibrium states and (b) for a given saturated equilibrium state or a given pair of saturated equilibrium and initial states, which and how many gene networks, referred to as viable, share this saturated equilibrium state or the pair of saturated equilibrium and initial states. We also show that the viable networks sharing a given saturated equilibrium state must follow certain patterns. These capabilities of the analytical treatment make it possible to properly define and accurately determine robustness to noise and mutation for gene networks. Previous network research conclusions drawn from performing millions of simulations follow directly from the results of our analytical treatment. Furthermore, the analytical results provide criteria for the identification of model validity and suggest modified models of gene network dynamics. The yeast cell-cycle network is used as an illustration of the practical application of this analytical treatment.
| S-EPMC3998189 | biostudies-literature
Project description:The development of brain imaging techniques, in particular functional magnetic resonance imaging (fMRI), made it possible to non-invasively study the hemispheric lateralization of cognitive brain functions in large cohorts. Comprehensive models of hemispheric lateralization are, however, still missing and should not only account for the hemispheric specialization of individual brain functions, but also for the interactions among different lateralized cognitive processes (e.g., language and visuospatial processing). This calls for robust and reliable paradigms to study hemispheric lateralization for various cognitive functions. While numerous reliable imaging paradigms have been developed for language, which represents the most prominent left-lateralized brain function, the reliability of imaging paradigms investigating typically right-lateralized brain functions, such as visuospatial processing, has received comparatively less attention. In the present study, we aimed to establish an fMRI paradigm that robustly and reliably identifies right-hemispheric activation evoked by visuospatial processing in individual subjects. In a first study, we therefore compared three frequently used paradigms for assessing visuospatial processing and evaluated their utility to robustly detect right-lateralized brain activity on a single-subject level. In a second study, we then assessed the test-retest reliability of the so-called Landmark task-the paradigm that yielded the most robust results in study 1. At the single-voxel level, we found poor reliability of the brain activation underlying visuospatial attention. This suggests that poor signal-to-noise ratios can become a limiting factor for test-retest reliability. This represents a common detriment of fMRI paradigms investigating visuospatial attention in general and therefore highlights the need for careful considerations of both the possibilities and limitations of the respective fMRI paradigm-in particular, when being interested in effects at the single-voxel level. Notably, however, when focusing on the reliability of measures of hemispheric lateralization (which was the main goal of study 2), we show that hemispheric dominance (quantified by the lateralization index, LI, with |LI| >0.4) of the evoked activation could be robustly determined in more than 62% and, if considering only two categories (i.e., left, right), in more than 93% of our subjects. Furthermore, the reliability of the lateralization strength (LI) was "fair" to "good". In conclusion, our results suggest that the degree of right-hemispheric dominance during visuospatial processing can be reliably determined using the Landmark task, both at the group and single-subject level, while at the same time stressing the need for future refinements of experimental paradigms and more sophisticated fMRI data acquisition techniques.
| S-EPMC5653292 | biostudies-literature