Browse
Submit Data
Databases
API
Help

Dataset Information

0 Views

0 Connections

0 Citations

0 Reanalyses

0 Downloads

Omics score: 0

A note on factor normalization for deep neural network models.

ABSTRACT: Deep neural network (DNN) models often involve high-dimensional features. In most cases, these high-dimensional features can be decomposed into two parts: a low-dimensional factor and residual features with much-reduced variability and inter-feature correlation. This decomposition has several interesting theoretical implications for DNN training. Based on these implications, we develop a novel factor normalization method for better performance. The proposed method leads to a new deep learning model with two important characteristics. First, it allows factor-related feature extraction, and second, it allows for adaptive learning rates for factors and residuals. These model features improve the convergence speed on both training and testing datasets. Multiple empirical experiments are presented to demonstrate the model's superior performance.

SUBMITTER: Qi H

PROVIDER: S-EPMC8993835 | biostudies-literature | 2022 Apr

REPOSITORIES: biostudies-literature

ACCESS DATA

Json Xml

Publications

A note on factor normalization for deep neural network models.

Qi Haobo H Zhou Jing J Wang Hansheng H

Scientific reports 20220408 1

Deep neural network (DNN) models often involve high-dimensional features. In most cases, these high-dimensional features can be decomposed into two parts: a low-dimensional factor and residual features with much-reduced variability and inter-feature correlation. This decomposition has several interesting theoretical implications for DNN training. Based on these implications, we develop a novel factor normalization method for better performance. The proposed method leads to a new deep learning mo ...[more]

PMID: 35396523

Similar Datasets

Technical Note: Deriving ventilation imaging from 4DCT by deep convolutional neural network.

Project description:PurposeVentilation images can be derived from four-dimensional computed tomography (4DCT) by analyzing the change in HU values and deformable vector fields between different respiration phases of computed tomography (CT). As deformable image registration (DIR) is involved, accuracy of 4DCT-derived ventilation image is sensitive to the choice of DIR algorithms. To overcome the uncertainty associated with DIR, we develop a method based on deep convolutional neural network (CNN) to derive ventilation images directly from the 4DCT without explicit image registration.MethodsA total of 82 sets of 4DCT and ventilation images from patients with lung cancer were used in this study. In the proposed CNN architecture, the CT two-channel input data consist of CT at the end of exhale and the end of inhale phases. The first convolutional layer has 32 different kernels of size 5 × 5 × 5, followed by another eight convolutional layers each of which is equipped with an activation layer (ReLU). The loss function is the mean-squared-error (MSE) to measure the intensity difference between the predicted and reference ventilation images.ResultsThe predicted images were comparable to the label images of the test data. The similarity index, correlation coefficient, and Gamma index passing rate averaged over the tenfold cross validation were 0.880 ± 0.035, 0.874 ± 0.024, and 0.806 ± 0.014, respectively.ConclusionsThe results demonstrate that deep CNN can generate ventilation imaging from 4DCT without explicit deformable image registration, reducing the associated uncertainty.

| S-EPMC7098066 | biostudies-literature

Individual differences among deep neural network models.

Project description:Deep neural networks (DNNs) excel at visual recognition tasks and are increasingly used as a modeling framework for neural computations in the primate brain. Just like individual brains, each DNN has a unique connectivity and representational profile. Here, we investigate individual differences among DNN instances that arise from varying only the random initialization of the network weights. Using tools typically employed in systems neuroscience, we show that this minimal change in initial conditions prior to training leads to substantial differences in intermediate and higher-level network representations despite similar network-level classification performance. We locate the origins of the effects in an under-constrained alignment of category exemplars, rather than misaligned category centroids. These results call into question the common practice of using single networks to derive insights into neural information processing and rather suggest that computational neuroscientists working with DNNs may need to base their inferences on groups of multiple network instances.

| S-EPMC7665054 | biostudies-literature

Deep neural network models for computational histopathology: A survey.

Project description:Histopathological images contain rich phenotypic information that can be used to monitor underlying mechanisms contributing to disease progression and patient survival outcomes. Recently, deep learning has become the mainstream methodological choice for analyzing and interpreting histology images. In this paper, we present a comprehensive review of state-of-the-art deep learning approaches that have been used in the context of histopathological image analysis. From the survey of over 130 papers, we review the field's progress based on the methodological aspect of different machine learning strategies such as supervised, weakly supervised, unsupervised, transfer learning and various other sub-variants of these methods. We also provide an overview of deep learning based survival models that are applicable for disease-specific prognosis tasks. Finally, we summarize several existing open datasets and highlight critical challenges and limitations with current deep learning approaches, along with possible avenues for future research.

| S-EPMC7725956 | biostudies-literature

Identification of robust deep neural network models of longitudinal clinical measurements.

Project description:Deep learning (DL) from electronic health records holds promise for disease prediction, but systematic methods for learning from simulated longitudinal clinical measurements have yet to be reported. We compared nine DL frameworks using simulated body mass index (BMI), glucose, and systolic blood pressure trajectories, independently isolated shape and magnitude changes, and evaluated model performance across various parameters (e.g., irregularity, missingness). Overall, discrimination based on variation in shape was more challenging than magnitude. Time-series forest-convolutional neural networks (TSF-CNN) and Gramian angular field(GAF)-CNN outperformed other approaches (P < 0.05) with overall area-under-the-curve (AUCs) of 0.93 for both models, and 0.92 and 0.89 for variation in magnitude and shape with up to 50% missing data. Furthermore, in a real-world assessment, the TSF-CNN model predicted T2D with AUCs reaching 0.72 using only BMI trajectories. In conclusion, we performed an extensive evaluation of DL approaches and identified robust modeling frameworks for disease prediction based on longitudinal clinical measurements.

| S-EPMC9329311 | biostudies-literature

Estimating and interpreting nonlinear receptive field of sensory neural responses with deep neural network models.

Project description:Our understanding of nonlinear stimulus transformations by neural circuits is hindered by the lack of comprehensive yet interpretable computational modeling frameworks. Here, we propose a data-driven approach based on deep neural networks to directly model arbitrarily nonlinear stimulus-response mappings. Reformulating the exact function of a trained neural network as a collection of stimulus-dependent linear functions enables a locally linear receptive field interpretation of the neural network. Predicting the neural responses recorded invasively from the auditory cortex of neurosurgical patients as they listened to speech, this approach significantly improves the prediction accuracy of auditory cortical responses, particularly in nonprimary areas. Moreover, interpreting the functions learned by neural networks uncovered three distinct types of nonlinear transformations of speech that varied considerably from primary to nonprimary auditory regions. The ability of this framework to capture arbitrary stimulus-response mappings while maintaining model interpretability leads to a better understanding of cortical processing of sensory signals.

| S-EPMC7347387 | biostudies-literature

Deep neural network models for identifying incident dementia using claims and EHR datasets.

Project description:This study investigates the use of deep learning methods to improve the accuracy of a predictive model for dementia, and compares the performance to a traditional machine learning model. With sufficient accuracy the model can be deployed as a first round screening tool for clinical follow-up including neurological examination, neuropsychological testing, imaging and recruitment to clinical trials. Seven cohorts with two years of data, three to eight years prior to index date, and an incident cohort were created. Four trained models for each cohort, boosted trees, feed forward network, recurrent neural network and recurrent neural network with pre-trained weights, were constructed and their performance compared using validation and test data. The incident model had an AUC of 94.4% and F1 score of 54.1%. Eight years removed from index date the AUC and F1 scores were 80.7% and 25.6%, respectively. The results for the remaining cohorts were between these ranges. Deep learning models can result in significant improvement in performance but come at a cost in terms of run times and hardware requirements. The results of the model at index date indicate that this modeling can be effective at stratifying patients at risk of dementia. At this time, the inability to sustain this quality at longer lead times is more an issue of data availability and quality rather than one of algorithm choices.

| S-EPMC7514098 | biostudies-literature

Contrasting action and posture coding with hierarchical deep neural network models of proprioception.

Project description:Biological motor control is versatile, efficient, and depends on proprioceptive feedback. Muscles are flexible and undergo continuous changes, requiring distributed adaptive control mechanisms that continuously account for the body's state. The canonical role of proprioception is representing the body state. We hypothesize that the proprioceptive system could also be critical for high-level tasks such as action recognition. To test this theory, we pursued a task-driven modeling approach, which allowed us to isolate the study of proprioception. We generated a large synthetic dataset of human arm trajectories tracing characters of the Latin alphabet in 3D space, together with muscle activities obtained from a musculoskeletal model and model-based muscle spindle activity. Next, we compared two classes of tasks: trajectory decoding and action recognition, which allowed us to train hierarchical models to decode either the position and velocity of the end-effector of one's posture or the character (action) identity from the spindle firing patterns. We found that artificial neural networks could robustly solve both tasks, and the networks' units show tuning properties similar to neurons in the primate somatosensory cortex and the brainstem. Remarkably, we found uniformly distributed directional selective units only with the action-recognition-trained models and not the trajectory-decoding-trained models. This suggests that proprioceptive encoding is additionally associated with higher-level functions such as action recognition and therefore provides new, experimentally testable hypotheses of how proprioception aids in adaptive motor control.

| S-EPMC10361732 | biostudies-literature

Deep sequential neural network models improve stratification of suicide attempt risk among US veterans.

Project description:ObjectiveTo apply deep neural networks (DNNs) to longitudinal EHR data in order to predict suicide attempt risk among veterans. Local explainability techniques were used to provide explanations for each prediction with the goal of ultimately improving outreach and intervention efforts.Materials and methodsThe DNNs fused demographic information with diagnostic, prescription, and procedure codes. Models were trained and tested on EHR data of approximately 500 000 US veterans: all veterans with recorded suicide attempts from April 1, 2005, through January 1, 2016, each paired with 5 veterans of the same age who did not attempt suicide. Shapley Additive Explanation (SHAP) values were calculated to provide explanations of DNN predictions.ResultsThe DNNs outperformed logistic and linear regression models in predicting suicide attempts. After adjusting for the sampling technique, the convolutional neural network (CNN) model achieved a positive predictive value (PPV) of 0.54 for suicide attempts within 12 months by veterans in the top 0.1% risk tier. Explainability methods identified meaningful subgroups of high-risk veterans as well as key determinants of suicide attempt risk at both the group and individual level.Discussion and conclusionThe deep learning methods employed in the present study have the potential to significantly enhance existing suicide risk models for veterans. These methods can also provide important clues to explore the relative value of long-term and short-term intervention strategies. Furthermore, the explainability methods utilized here could also be used to communicate to clinicians the key features which increase specific veterans' risk for attempting suicide.

| S-EPMC10746318 | biostudies-literature

Deep Neural Network Models for Predicting Chemically Induced Liver Toxicity Endpoints From Transcriptomic Responses.

Project description:Improving the accuracy of toxicity prediction models for liver injuries is a key element in evaluating the safety of drugs and chemicals. Mechanism-based information derived from expression (transcriptomic) data, in combination with machine-learning methods, promises to improve the accuracy and robustness of current toxicity prediction models. Deep neural networks (DNNs) have the advantage of automatically assembling the relevant features from a large number of input features. This makes them especially suitable for modeling transcriptomic data, which typically contain thousands of features. Here, we gaged gene- and pathway-level feature selection schemes using single- and multi-task DNN approaches in predicting chemically induced liver injuries (biliary hyperplasia, fibrosis, and necrosis) from whole-genome DNA microarray data. The single-task DNN models showed high predictive accuracy and endpoint specificity, with Matthews correlation coefficients for the three endpoints on 10-fold cross validation ranging from 0.56 to 0.89, with an average of 0.74 in the best feature sets. The DNN models outperformed Random Forest models in cross validation and showed better performance than Support Vector Machine models when tested in the external validation datasets. In the cross validation studies, the effect of the feature selection scheme was negligible among the studied feature sets. Further evaluation of the models on their ability to predict the injury phenotype per se for non-chemically induced injuries revealed the robust performance of the DNN models across these additional external testing datasets. Thus, the DNN models learned features specific to the injury phenotype contained in the gene expression data.

| S-EPMC6370634 | biostudies-literature

Brain-optimized deep neural network models of human visual areas learn non-hierarchical representations.

Project description:Deep neural networks (DNNs) optimized for visual tasks learn representations that align layer depth with the hierarchy of visual areas in the primate brain. One interpretation of this finding is that hierarchical representations are necessary to accurately predict brain activity in the primate visual system. To test this interpretation, we optimized DNNs to directly predict brain activity measured with fMRI in human visual areas V1-V4. We trained a single-branch DNN to predict activity in all four visual areas jointly, and a multi-branch DNN to predict each visual area independently. Although it was possible for the multi-branch DNN to learn hierarchical representations, only the single-branch DNN did so. This result shows that hierarchical representations are not necessary to accurately predict human brain activity in V1-V4, and that DNNs that encode brain-like visual representations may differ widely in their architecture, ranging from strict serial hierarchies to multiple independent branches.

| S-EPMC10247700 | biostudies-literature

OmicsDI is part of the ELIXIR infrastructure

OmicsDI is an Elixir interoperability service. Learn more ›

Tweets

OmicsDI Databases

PRIDE
PeptideAtlas
MassIVE
JPOST Repository
Physiome Model Repository

EGA
EVA
ENA
LINCS
PAXDB
Cell Collective

MetaboLights
Metabolomics Workbench
MetabolomeExpress
GNPS
BioModels
FAIRDOMHub

ArrayExpress
dbGaP
ExpressionAtlas
GEO
NODE

Information

Databases
Help
API
Contact us
Code on GitHub
Terms of use
Submit Data