A high-order statistical tensor based algorithm for anomaly detection in hyperspectral imagery.
Ontology highlight
ABSTRACT:
SUBMITTER: Geng X
PROVIDER: S-EPMC4219173 | biostudies-other | 2014
REPOSITORIES: biostudies-other
Similar Datasets
Project description:Hyperspectral images include information from a wide range of spectral bands deemed valuable for computer vision applications in various domains such as agriculture, surveillance, and reconnaissance. Anomaly detection in hyperspectral images has proven to be a crucial component of change and abnormality identification, enabling improved decision-making across various applications. These abnormalities/anomalies can be detected using background estimation techniques that do not require the prior knowledge of outliers. However, each hyperspectral anomaly detection (HS-AD) algorithm models the background differently. These different assumptions may fail to consider all the background constraints in various scenarios. We have developed a new approach called Greedy Ensemble Anomaly Detection (GE-AD) to address this shortcoming. It includes a greedy search algorithm to systematically determine the suitable base models from HS-AD algorithms and hyperspectral unmixing for the first stage of a stacking ensemble and employs a supervised classifier in the second stage of a stacking ensemble. It helps researchers with limited knowledge of the suitability of the HS-AD algorithms for the application scenarios to select the best methods automatically. Our evaluation shows that the proposed method achieves a higher average F1-macro score with statistical significance compared to the other individual methods used in the ensemble. This is validated on multiple datasets, including the Airport-Beach-Urban (ABU) dataset, the San Diego dataset, the Salinas dataset, the Hydice Urban dataset, and the Arizona dataset. The evaluation using the airport scenes from the ABU dataset shows that GE-AD achieves a 14.97% higher average F1-macro score than our previous method (HUE-AD), at least 17.19% higher than the individual methods used in the ensemble, and at least 28.53% higher than the other state-of-the-art ensemble anomaly detection algorithms. As using the combination of greedy algorithm and stacking ensemble to automatically select suitable base models and associated weights have not been widely explored in hyperspectral anomaly detection, we believe that our work will expand the knowledge in this research area and contribute to the wider application of this approach.
| S-EPMC11204925 | biostudies-literature
Project description:The recent advent of tensor tomography techniques has enabled tomographic investigations of the 3D nanostructure organization of biological and material science samples. These techniques extended the concept of conventional X-ray tomography by reconstructing not only a scalar value such as the attenuation coefficient per voxel, but also a set of parameters that capture the local anisotropy of nanostructures within every voxel of the sample. Tensor tomography data sets are intrinsically large as each pixel of a conventional X-ray projection is substituted by a scattering pattern, and projections have to be recorded at different sample angular orientations with several tilts of the rotation axis with respect to the X-ray propagation direction. Currently available reconstruction approaches for such large data sets are computationally expensive. Here, a novel, fast reconstruction algorithm, named iterative reconstruction tensor tomography (IRTT), is presented to simplify and accelerate tensor tomography reconstructions. IRTT is based on a second-rank tensor model to describe the anisotropy of the nanostructure in every voxel and on an iterative error backpropagation reconstruction algorithm to achieve high convergence speed. The feasibility and accuracy of IRTT are demonstrated by reconstructing the nanostructure anisotropy of three samples: a carbon fiber knot, a human bone trabecula specimen and a fixed mouse brain. Results and reconstruction speed were compared with those obtained by the small-angle scattering tensor tomography (SASTT) reconstruction method introduced by Liebi et al. [Nature (2015), 527, 349-352]. The principal orientation of the nanostructure within each voxel revealed a high level of agreement between the two methods. Yet, for identical data sets and computer hardware used, IRTT was shown to be more than an order of magnitude faster. IRTT was found to yield robust results, it does not require prior knowledge of the sample for initializing parameters, and can be used in cases where simple anisotropy metrics are sufficient, i.e. the tensor approximation adequately captures the level of anisotropy and the dominant orientation within a voxel. In addition, by greatly accelerating the reconstruction, IRTT is particularly suitable for handling large tomographic data sets of samples with internal structure or as a real-time analysis tool during the experiment for online feedback during data acquisition. Alternatively, the IRTT results might be used as an initial guess for models capturing a higher complexity of structural anisotropy such as spherical harmonics based SASTT in Liebi et al. (2015), improving both overall convergence speed and robustness of the reconstruction.
| S-EPMC6396401 | biostudies-literature
Project description:Hyperspectral imaging combined with chemometric approaches is proven to be a powerful tool for the quality evaluation and control of fruits. In fruit defect-detection scenarios, developing an unsupervised anomaly detection framework is vital, as defect sample preparation is labor-intensive and time-consuming, especially for exploring potential defects. In this paper, a spectral-spatial, information-based, self-supervised anomaly detection (SSAD) approach is proposed. During training, an auxiliary classifier is proposed to identify the projection axes of principal component (PC) images that were transformed from the hyperspectral data cubes. In test time, the fully connected layer of the learned classifier was used as a 'spectral-spatial' feature extractor, and the feature similarity metric was adopted as the score function for the downstream anomaly evaluation task. The proposed network was evaluated with two fruit data sets: a strawberry data set with bruised, infected, chilling-injured, and contaminated test samples and a blueberry data set with bruised, infected, chilling-injured, and wrinkled samples as anomalies. The results show that the SSAD yielded the best anomaly detection performance (AUC = 0.923 on average) over the baseline methods, and the visualization results further confirmed its advantage in extracting effective 'spectral-spatial' latent representation. Moreover, the robustness of SSAD is verified with the data pollution experiment; it performed significantly better than the baselines when a portion of anomalous samples was involved in the training process.
| S-EPMC10378827 | biostudies-literature
Project description:Objective:Malfunctions in Clinical Decision Support (CDS) systems occur due to a multitude of reasons, and often go unnoticed, leading to potentially poor outcomes. Our goal was to identify malfunctions within CDS systems. Methods:We evaluated 6 anomaly detection models: (1) Poisson Changepoint Model, (2) Autoregressive Integrated Moving Average (ARIMA) Model, (3) Hierarchical Divisive Changepoint (HDC) Model, (4) Bayesian Changepoint Model, (5) Seasonal Hybrid Extreme Studentized Deviate (SHESD) Model, and (6) E-Divisive with Median (EDM) Model and characterized their ability to find known anomalies. We analyzed 4 CDS alerts with known malfunctions from the Longitudinal Medical Record (LMR) and Epic® (Epic Systems Corporation, Madison, WI, USA) at Brigham and Women's Hospital, Boston, MA. The 4 rules recommend lead testing in children, aspirin therapy in patients with coronary artery disease, pneumococcal vaccination in immunocompromised adults and thyroid testing in patients taking amiodarone. Results:Poisson changepoint, ARIMA, HDC, Bayesian changepoint and the SHESD model were able to detect anomalies in an alert for lead screening in children and in an alert for pneumococcal conjugate vaccine in immunocompromised adults. EDM was able to detect anomalies in an alert for monitoring thyroid function in patients on amiodarone. Conclusions:Malfunctions/anomalies occur frequently in CDS alert systems. It is important to be able to detect such anomalies promptly. Anomaly detection models are useful tools to aid such detections.
| S-EPMC6016695 | biostudies-literature
Project description:Hyperspectral image (HSI) consists of hundreds of narrow spectral band components with rich spectral and spatial information. Extreme Learning Machine (ELM) has been widely used for HSI analysis. However, the classical ELM is difficult to use for sparse feature leaning due to its randomly generated hidden layer. In this paper, we propose a novel unsupervised sparse feature learning approach, called Evolutionary Multiobjective-based ELM (EMO-ELM), and apply it to HSI feature extraction. Specifically, we represent the task of constructing the ELM Autoencoder (ELM-AE) as a multiobjective optimization problem that takes the sparsity of hidden layer outputs and the reconstruction error as two conflicting objectives. Then, we adopt an Evolutionary Multiobjective Optimization (EMO) method to solve the two objectives, simultaneously. To find the best solution from the Pareto solution set and construct the best trade-off feature extractor, a curvature-based method is proposed to focus on the knee area of the Pareto solutions. Benefited from the EMO, the proposed EMO-ELM is less prone to fall into a local minimum and has fewer trainable parameters than gradient-based AEs. Experiments on two real HSIs demonstrate that the features learned by EMO-ELM not only preserve better sparsity but also achieve superior separability than many existing feature learning methods.
| S-EPMC7085560 | biostudies-literature
Project description:High-yield rice cultivation is an effective way to address the increasing food demand worldwide. Correct classification of high-yield rice is a key step of breeding. However, manual measurements within breeding programs are time consuming and have high cost and low throughput, which limit the application in large-scale field phenotyping. In this study, we developed an accurate large-scale approach and presented the potential usage of hyperspectral data for rice yield measurement using the XGBoost algorithm to speed up the rice breeding process for many breeders. In total, 13 japonica rice lines in regional trials in northern China were divided into different categories according to the manual measurement of yield. Using an Unmanned Aerial Vehicle (UAV) platform equipped with a hyperspectral camera to capture images over multiple time series, a rice yield classification model based on the XGBoost algorithm was proposed. Four comparison experiments were carried out through the intraline test and the interline test considering lodging characteristics at the midmature stage or not. The result revealed that the degree of lodging in the midmature stage was an important feature affecting the classification accuracy of rice. Thus, we developed a low-cost, high-throughput phenotyping and nondestructive method by combining UAV-based hyperspectral measurements and machine learning for estimation of rice yield to improve rice breeding efficiency.
| S-EPMC8028843 | biostudies-literature
Project description:The operational monitoring of forest decline requires the development of remote sensing methods that are sensitive to the spatiotemporal variations of pigment degradation and canopy defoliation. In this context, the red-edge spectral region (RESR) was proposed in the past due to its combined sensitivity to chlorophyll content and leaf area variation. In this study, the temporal dimension of the RESR was evaluated as a function of forest decline using a radiative transfer method with the PROSPECT and 3D FLIGHT models. These models were used to generate synthetic pine stands simulating decline and recovery processes over time and explore the temporal rate of change of the red-edge chlorophyll index (CI) as compared to the trajectories obtained for the structure-related Normalized Difference Vegetation Index (NDVI). The temporal trend method proposed here consisted of using synthetic spectra to calculate the theoretical boundaries of the subspace for healthy and declining pine trees in the temporal domain, defined by CItime=n/CItime=n+1 vs. NDVItime=n/NDVItime=n+1. Within these boundaries, trees undergoing decline and recovery processes showed different trajectories through this subspace. The method was then validated using three high-resolution airborne hyperspectral images acquired at 40 cm resolution and 260 spectral bands of 6.5 nm full-width half-maximum (FWHM) over a forest with widespread tree decline, along with field-based monitoring of chlorosis and defoliation (i.e., 'decline' status) in 663 trees between the years 2015 and 2016. The temporal rate of change of chlorophyll vs. structural indices, based on reflectance spectra extracted from the hyperspectral images, was different for trees undergoing decline, and aligned towards the decline baseline established using the radiative transfer models. By contrast, healthy trees over time aligned towards the theoretically obtained healthy baseline. The applicability of this temporal trend method to the red-edge bands of the MultiSpectral Imager (MSI) instrument on board Sentinel-2a for operational forest status monitoring was also explored by comparing the temporal rate of change of the Sentinel-2-derived CI over areas with declining and healthy trees. Results demonstrated that the Sentinel-2a red-edge region was sensitive to the temporal dimension of forest condition, as the relationships obtained for pixels in healthy condition deviated from those of pixels undergoing decline.
| S-EPMC5848051 | biostudies-other
Project description:MotivationMixed molecular data combines continuous and categorical features of the same samples, such as OMICS profiles with genotypes, diagnoses, or patient sex. Like all high-dimensional molecular data, it is prone to incorrect values that can stem from various sources for example the technical limitations of the measurement devices, errors in the sample preparation, or contamination. Most anomaly detection algorithms identify complete samples as outliers or anomalies. However, in most cases, not all measurements of those samples are erroneous but only a few one-dimensional features within the samples are incorrect. These one-dimensional data errors are continuous measurements that are either located outside or inside the normal ranges of their features but in both cases show atypical values given all other continuous and categorical features in the sample. Additionally, categorical anomalies can occur for example when the genotype or diagnosis was submitted wrongly.ResultsWe introduce ADMIRE (Anomaly Detection using MIxed gRaphical modEls), a novel approach for the detection and correction of anomalies in mixed high-dimensional data. Hereby, we focus on the detection of single (one-dimensional) data errors in the categorical and continuous features of a sample. For that the joint distribution of continuous and categorical features is learned by mixed graphical models, anomalies are detected by the difference between measured and model-based estimations and are corrected using imputation. We evaluated ADMIRE in simulation and by screening for anomalies in one of our own metabolic datasets. In simulation experiments, ADMIRE outperformed the state-of-the-art methods of Local Outlier Factor, stray, and Isolation Forest.Availability and implementationAll data and code is available at https://github.com/spang-lab/adadmire. ADMIRE is implemented in a Python package called adadmire which can be found at https://pypi.org/project/adadmire.
| S-EPMC10457663 | biostudies-literature
Project description:Extracellular vesicles (EVs) have gained significant attention in the field of extracellular communication, holding promise as intercellular messengers and diagnostic biomarkers. However, the challenges in isolating and purifying EVs from complex biological samples have hindered their widespread utility. This study presents a novel EV isolation strategy leveraging bioinformatics, utilizing the pH-low insertion peptide (pHLIP) to capture and release EVs under controlled conditions. The pHLIP-magnetic bead approach was developed to achieve efficient EV isolation, addressing the persistent issue of EV isolation method comparability. EVs from human plasma were isolated and characterized using various techniques, offering insights into the proposed isolation method's efficacy. A data science-driven statistical metric integrating multiple characterizations was applied to enable direct comparisons between different isolation methods. Next generation sequencing was applied to examine how the observed RNA-population differences across isolated EVs may lead researchers to different conclusions. This research contributes to the advancement of reliable and rigorous EV isolation methods and their potential clinical applications.
2024-01-31 | GSE246925 | GEO
Project description:Event-based cameras are bio-inspired novel sensors that asynchronously record changes in illumination in the form of events. This principle results in significant advantages over conventional cameras, such as low power utilization, high dynamic range, and no motion blur. Moreover, by design, such cameras encode only the relative motion between the scene and the sensor and not the static background to yield a very sparse data structure. In this paper, we leverage these advantages of an event camera toward a critical vision application-video anomaly detection. We propose an anomaly detection solution in the event domain with a conditional Generative Adversarial Network (cGAN) made up of sparse submanifold convolution layers. Video analytics tasks such as anomaly detection depend on the motion history at each pixel. To enable this, we also put forward a generic unsupervised deep learning solution to learn a novel memory surface known as Deep Learning (DL) memory surface. DL memory surface encodes the temporal information readily available from these sensors while retaining the sparsity of event data. Since there is no existing dataset for anomaly detection in the event domain, we also provide an anomaly detection event dataset with a set of anomalies. We empirically validate our anomaly detection architecture, composed of sparse convolutional layers, on this proposed and online dataset. Careful analysis of the anomaly detection network reveals that the presented method results in a massive reduction in computational complexity with good performance compared to previous state-of-the-art conventional frame-based anomaly detection networks.
| S-EPMC8358807 | biostudies-literature