Dataset Information

Learning Scalable Deep Kernels with Recurrent Structure.

ABSTRACT: Many applications in speech, robotics, finance, and biology deal with sequential data, where ordering matters and recurrent structures are common. However, this structure cannot be easily captured by standard kernel functions. To model such structure, we propose expressive closed-form kernel functions for Gaussian processes. The resulting model, GP-LSTM, fully encapsulates the inductive biases of long short-term memory (LSTM) recurrent networks, while retaining the non-parametric probabilistic advantages of Gaussian processes. We learn the properties of the proposed kernels by optimizing the Gaussian process marginal likelihood using a new provably convergent semi-stochastic gradient procedure, and exploit the structure of these kernels for scalable training and prediction. This approach provides a practical representation for Bayesian LSTMs. We demonstrate state-of-the-art performance on several benchmarks, and thoroughly investigate a consequential autonomous driving application, where the predictive uncertainties provided by GP-LSTM are uniquely valuable.

SUBMITTER: Al-Shedivat M

PROVIDER: S-EPMC6334642 | biostudies-other | 2017 Jan

REPOSITORIES: biostudies-other

ACCESS DATA

Publications

Learning Scalable Deep Kernels with Recurrent Structure.

Al-Shedivat Maruan M Wilson Andrew Gordon AG Saatchi Yunus Y Hu Zhiting Z Xing Eric P EP

Journal of machine learning research : JMLR 20170101 1

Many applications in speech, robotics, finance, and biology deal with sequential data, where ordering matters and recurrent structures are common. However, this structure cannot be easily captured by standard kernel functions. To model such structure, we propose expressive closed-form kernel functions for Gaussian processes. The resulting model, GP-LSTM, fully encapsulates the inductive biases of long short-term memory (LSTM) recurrent networks, while retaining the non-parametric probabilistic a ...[more]

PMID: 30662374

Similar Datasets

Project description:AimUnderstanding how spatial scale of study affects observed dispersal patterns can provide insights into spatiotemporal population dynamics, particularly in systems with significant long-distance dispersal (LDD). We aimed to investigate the dispersal gradients of two rusts of wheat with spores of similar size, mass, and shape, over multiple spatial scales. We hypothesized that a single dispersal kernel could fit the dispersal from all spatial scales well, and that it would be possible to obtain similar results in spatiotemporal increase of disease when modeling based on differing scales.LocationCentral Oregon and St. Croix Island.TaxaPuccinia striiformis f. sp. tritici, Puccinia graminis f. sp. tritici, Triticum aestivum.MethodsWe compared empirically-derived primary disease gradients of cereal rust across three spatial scales: local (inoculum source and sampling unit = 0.0254 m, spatial extent = 1.52m) field-wide (inoculum source = 1.52 m, sampling unit = 0.305 m, and spatial extent = 91.44 m), and regional (inoculum source and sampling unit = 152 m, spatial extent = 10.7 km). We then examined whether disease spread in spatially explicit simulations depended upon the scale at which data were collected by constructing a compartmental time-step model.ResultsThe three data sets could be fit well by a single inverse-power law dispersal kernel. Simulating epidemic spread at different spatial resolutions resulted in similar patterns of spatiotemporal spread. Dispersal kernel data obtained at one spatial scale can be used to represent spatiotemporal disease spread at a larger spatial scale.Main conclusionsOrganisms spread by aerially dispersed small propagules that exhibit LDD may follow similar dispersal patterns over a several hundred- or thousand-fold expanse of spatial scale. Given that the primary mechanisms driving aerial dispersal remain constant, it may be possible to extrapolate across scales when empirical data are unavailable at a scale of interest.

Dataset Information

Learning Scalable Deep Kernels with Recurrent Structure.

Publications

Learning Scalable Deep Kernels with Recurrent Structure.

Similar Datasets

OmicsDI is part of the ELIXIR infrastructure

Tweets