Dataset Information

An Adaptive Empirical Bayesian Method for Sparse Deep Learning.

ABSTRACT: We propose a novel adaptive empirical Bayesian (AEB) method for sparse deep learning, where the sparsity is ensured via a class of self-adaptive spike-and-slab priors. The proposed method works by alternatively sampling from an adaptive hierarchical posterior distribution using stochastic gradient Markov Chain Monte Carlo (MCMC) and smoothly optimizing the hyperparameters using stochastic approximation (SA). We further prove the convergence of the proposed method to the asymptotically correct distribution under mild conditions. Empirical applications of the proposed method lead to the state-of-the-art performance on MNIST and Fashion MNIST with shallow convolutional neural networks (CNN) and the state-of-the-art compression performance on CIFAR10 with Residual Networks. The proposed method also improves resistance to adversarial attacks.

SUBMITTER: Deng W

PROVIDER: S-EPMC7687285 | biostudies-literature | 2019 Dec

REPOSITORIES: biostudies-literature

ACCESS DATA

Publications

An Adaptive Empirical Bayesian Method for Sparse Deep Learning.

Deng Wei W Zhang Xiao X Liang Faming F Lin Guang G

Advances in neural information processing systems 20191201

We propose a novel adaptive empirical Bayesian (AEB) method for sparse deep learning, where the sparsity is ensured via a class of self-adaptive spike-and-slab priors. The proposed method works by alternatively sampling from an adaptive hierarchical posterior distribution using stochastic gradient Markov Chain Monte Carlo (MCMC) and smoothly optimizing the hyperparameters using stochastic approximation (SA). We further prove the convergence of the proposed method to the asymptotically correct di ...[more]

PMID: 33244209

Similar Datasets

Project description:BackgroundTime-resolved 4D cone beam-computed tomography (4D-CBCT) allows a daily assessment of patient anatomy and respiratory motion. However, 4D-CBCTs suffer from imaging artifacts that affect the CT number accuracy and prevent accurate proton dose calculations. Deep learning can be used to correct CT numbers and generate synthetic CTs (sCTs) that can enable CBCT-based proton dose calculations.PurposeIn this work, sparse view 4D-CBCTs were converted into 4D-sCT utilizing a deep convolutional neural network (DCNN). 4D-sCTs were evaluated in terms of image quality and dosimetric accuracy to determine if accurate proton dose calculations for adaptive proton therapy workflows of lung cancer patients are feasible.MethodsA dataset of 45 thoracic cancer patients was utilized to train and evaluate a DCNN to generate 4D-sCTs, based on sparse view 4D-CBCTs reconstructed from projections acquired with a 3D acquisition protocol. Mean absolute error (MAE) and mean error were used as metrics to evaluate the image quality of single phases and average 4D-sCTs against 4D-CTs acquired on the same day. The dosimetric accuracy was checked globally (gamma analysis) and locally for target volumes and organs-at-risk (OARs) (lung, heart, and esophagus). Furthermore, 4D-sCTs were also compared to 3D-sCTs. To evaluate CT number accuracy, proton radiography simulations in 4D-sCT and 4D-CTs were compared in terms of range errors. The clinical suitability of 4D-sCTs was demonstrated by performing a 4D dose reconstruction using patient specific treatment delivery log files and breathing signals.Results4D-sCTs resulted in average MAEs of 48.1 ± 6.5 HU (single phase) and 37.7 ± 6.2 HU (average). The global dosimetric evaluation showed gamma pass ratios of 92.3% ± 3.2% (single phase) and 94.4% ± 2.1% (average). The clinical target volume showed high agreement in D98 between 4D-CT and 4D-sCT, with differences below 2.4% for all patients. Larger dose differences were observed in mean doses of OARs (up to 8.4%). The comparison with 3D-sCTs showed no substantial image quality and dosimetric differences for the 4D-sCT average. Individual 4D-sCT phases showed slightly lower dosimetric accuracy. The range error evaluation revealed that lung tissues cause range errors about three times higher than the other tissues.ConclusionIn this study, we have investigated the accuracy of deep learning-based 4D-sCTs for daily dose calculations in adaptive proton therapy. Despite image quality differences between 4D-sCTs and 3D-sCTs, comparable dosimetric accuracy was observed globally and locally. Further improvement of 3D and 4D lung sCTs could be achieved by increasing CT number accuracy in lung tissues.

Project description:Protein secondary structure prediction provides insight into protein function and is a valuable preliminary step for predicting the 3D structure of a protein. Dynamic Bayesian networks (DBNs) and support vector machines (SVMs) have been shown to provide state-of-the-art performance in secondary structure prediction. As the size of the protein database grows, it becomes feasible to use a richer model in an effort to capture subtle correlations among the amino acids and the predicted labels. In this context, it is beneficial to derive sparse models that discourage over-fitting and provide biological insight.In this paper, we first show that we are able to obtain accurate secondary structure predictions. Our per-residue accuracy on a well established and difficult benchmark (CB513) is 80.3%, which is comparable to the state-of-the-art evaluated on this dataset. We then introduce an algorithm for sparsifying the parameters of a DBN. Using this algorithm, we can automatically remove up to 70-95% of the parameters of a DBN while maintaining the same level of predictive accuracy on the SD576 set. At 90% sparsity, we are able to compute predictions three times faster than a fully dense model evaluated on the SD576 set. We also demonstrate, using simulated data, that the algorithm is able to recover true sparse structures with high accuracy, and using real data, that the sparse model identifies known correlation structure (local and non-local) related to different classes of secondary structure elements.We present a secondary structure prediction method that employs dynamic Bayesian networks and support vector machines. We also introduce an algorithm for sparsifying the parameters of the dynamic Bayesian network. The sparsification approach yields a significant speed-up in generating predictions, and we demonstrate that the amino acid correlations identified by the algorithm correspond to several known features of protein secondary structure. Datasets and source code used in this study are available at http://noble.gs.washington.edu/proj/pssp.

Project description:Nanophotonics exploits the best of photonics and nanotechnology which has transformed optics in recent years by allowing subwavelength structures to enhance light-matter interactions. Despite these breakthroughs, design, fabrication, and characterization of such exotic devices have remained through iterative processes which are often computationally costly, memory-intensive, and time-consuming. In contrast, deep learning approaches have recently shown excellent performance as practical computational tools, providing an alternate avenue for speeding up such nanophotonics simulations. This study presents a DNN framework for transmission, reflection, and absorption spectra predictions by grasping the hidden correlation between the independent nanostructure properties and their corresponding optical responses. The proposed DNN framework is shown to require a sufficient amount of training data to achieve an accurate approximation of the optical performance derived from computational models. The fully trained framework can outperform a traditional EM solution using on the COMSOL Multiphysics approach in terms of computational cost by three orders of magnitude. Furthermore, employing deep learning methodologies, the proposed DNN framework makes an effort to optimise design elements that influence the geometrical dimensions of the nanostructure, offering insight into the universal transmission, reflection, and absorption spectra predictions at the nanoscale. This paradigm improves the viability of complicated nanostructure design and analysis, and it has a lot of potential applications involving exotic light-matter interactions between nanostructures and electromagnetic fields. In terms of computational times, the designed algorithm is more than 700 times faster as compared to conventional FEM method (when manual meshing is used). Hence, this approach paves the way for fast yet universal methods for the characterization and analysis of the optical response of nanophotonic systems.

Dataset Information

An Adaptive Empirical Bayesian Method for Sparse Deep Learning.

Publications

An Adaptive Empirical Bayesian Method for Sparse Deep Learning.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets