Published on Fri May 31 2019

Greedy inference with structure-exploiting lazy maps

Michael C. Brennan, Daniele Bigoni, Olivier Zahm, Alessio Spantini, Youssef Marzouk

We propose a framework for solving high-dimensional Bayesian inference problems using low-dimensional transport maps. These maps are confined to a low- dimensional subspace (hence, lazy), and the subspace is identified by minimizing an upper bound on the Kullback--Leibler divergence.

0
0
0
Abstract

We propose a framework for solving high-dimensional Bayesian inference problems using \emph{structure-exploiting} low-dimensional transport maps or flows. These maps are confined to a low-dimensional subspace (hence, lazy), and the subspace is identified by minimizing an upper bound on the Kullback--Leibler divergence (hence, structured). Our framework provides a principled way of identifying and exploiting low-dimensional structure in an inference problem. It focuses the expressiveness of a transport map along the directions of most significant discrepancy from the posterior, and can be used to build deep compositions of lazy maps, where low-dimensional projections of the parameters are iteratively transformed to match the posterior. We prove weak convergence of the generated sequence of distributions to the posterior, and we demonstrate the benefits of the framework on challenging inference problems in machine learning and differential equations, using inverse autoregressive flows and polynomial maps as examples of the underlying density estimators.

Fri Jun 26 2020
Machine Learning
Relative gradient optimization of the Jacobian term in unsupervised deep learning
Deep density models have been widely used for this task, but their maximum likelihood based training requires estimating the log-determinant of the Jacobian. We exploit the matrix structure of neural network parameters to compute updates efficiently even in high-dimensional spaces.
0
0
0
Tue Jun 01 2021
Machine Learning
Large-Scale Wasserstein Gradient Flows
Wasserstein gradient flows provide a powerful means of understanding and solving many diffusion equations. Fokker-Planck equations can be understood as gradient descent over entropy functionals in Wasserstein space. Solving the optimization problem associated with each JKO step presents serious computational challenges.
2
2
16
Tue Jun 12 2018
Machine Learning
Approximate inference with Wasserstein gradient flows
The time-dependent density of the diffusion is derived as the limit of implicit Euler steps that follow the gradients of a particular free energy functional. Existing methods for computing Wasserstein gradient flowsrely on discretization of the domain of the diffuse process. We propose instead a
0
0
0
Tue Jun 20 2017
Machine Learning
Inference in Deep Networks in High Dimensions
Deep generative networks provide a powerful tool for modeling complex data in a wide range of applications. In inverse problems that use these networks, one must often perform inference of the inputs of the networks from the outputs. Inference is also required for sampling during stochastic training on these models.
0
0
0
Wed Jan 08 2020
Machine Learning
Learning Generative Models using Denoising Density Estimators
Learn probabilistic models that can estimate the density of a given set ofamples, and generate samples from that density. We introduce a new generative model based on denoising density estimators (DDEs)
0
0
0
Fri May 29 2020
Machine Learning
OT-Flow: Fast and Accurate Continuous Normalizing Flows via Optimal Transport
A normalizing flow is an invertible mapping between an arbitrary probability distribution and a standard normal distribution. The log-determinant of the flow's Jacobian can be obtained by integrating the trace of the dynamics' Jacobian along the flow. The proposed OT-Flow approach tackles two critical computational challenges.
1
0
0