Published on Sat Aug 22 2020

Informative Neural Ensemble Kalman Learning

Margaret Trautner, Gabriel Margolis, Sai Ravela

Informative approaches select key measurement or decision variables that maximize information gain to enhance the efficacy of model-related inferences. Here, we propose Informative Ensemble Kalman Learning, which replaces backpropagation with an adaptive Filter.

0
0
0
Abstract

In stochastic systems, informative approaches select key measurement or decision variables that maximize information gain to enhance the efficacy of model-related inferences. Neural Learning also embodies stochastic dynamics, but informative Learning is less developed. Here, we propose Informative Ensemble Kalman Learning, which replaces backpropagation with an adaptive Ensemble Kalman Filter to quantify uncertainty and enables maximizing information gain during Learning. After demonstrating Ensemble Kalman Learning's competitive performance on standard datasets, we apply the informative approach to neural structure learning. In particular, we show that when trained from the Lorenz-63 system's simulations, the efficaciously learned structure recovers the dynamical equations. To the best of our knowledge, Informative Ensemble Kalman Learning is new. Results suggest that this approach to optimized Learning is promising.

Tue Nov 27 2018
Machine Learning
Bayesian Neural Network Ensembles
Ensembles of neural networks (NNs) have long been used to estimate predictive uncertainty. The appeal of ensembling stems from being a collection of regular NNs. They have achieved strong empirical results in recent years.
0
0
0
Tue Jul 14 2020
Machine Learning
Supervised learning from noisy observations: Combining machine-learning techniques with data assimilation
0
0
0
Fri May 20 2016
Machine Learning
Bayesian Hyperparameter Optimization for Ensemble Learning
In this paper, we bridge the gap between hyperparameter optimization and ensemble learning by performing Bayesian optimization of an ensemble. Our method consists in building a fixed-size ensemble, optimizing the configuration of one classifier of the ensemble at each iteration of the algorithm.
0
0
0
Fri Aug 10 2018
Machine Learning
Ensemble Kalman Inversion: A Derivative-Free Technique For Machine Learning Tasks
The standard probabilistic perspective on machine learning gives rise to risk-minimization tasks that are frequently solved by stochastic gradient descent. We propose an efficient, gradient-free algorithm for finding a solution to these problems using ensemble Kalman inversion (EKI)
0
0
0
Wed Mar 08 2017
Machine Learning
Unsupervised Ensemble Regression
We develop methods to detect the best and worst regressors. We derive U-PCR, a novel principal components approach for unsupervised ensemble regression. We demonstrate its improved accuracy over the ensemble mean and median on a variety of problems.
0
0
0
Tue Feb 01 2005
Neural Networks
Neural network ensembles: Evaluation of aggregation algorithms
0
0
0