Published on Thu Jun 27 2019

From self-tuning regulators to reinforcement learning and back again

Nikolai Matni, Alexandre Proutiere, Anders Rantzer, Stephen Tu

Machine and reinforcement learning are increasingly being applied to plan and control the behavior of autonomous systems. Examples include self-driving vehicles, distributed sensor networks, and agile robots. The goal of this tutorial paper is to provide a starting point for control theorists wishing to work on learning related problems.

0
0
0
Abstract

Machine and reinforcement learning (RL) are increasingly being applied to plan and control the behavior of autonomous systems interacting with the physical world. Examples include self-driving vehicles, distributed sensor networks, and agile robots. However, when machine learning is to be applied in these new settings, the algorithms had better come with the same type of reliability, robustness, and safety bounds that are hallmarks of control theory, or failures could be catastrophic. Thus, as learning algorithms are increasingly and more aggressively deployed in safety critical settings, it is imperative that control theorists join the conversation. The goal of this tutorial paper is to provide a starting point for control theorists wishing to work on learning related problems, by covering recent advances bridging learning and control theory, and by placing these results within an appropriate historical context of system identification and adaptive control.

Mon Jun 25 2018
Machine Learning
A Tour of Reinforcement Learning: The View from Continuous Control
This manuscript surveys reinforcement learning from the perspective of optimization and control with a focus on continuous control applications. In order to compare the relative merits of various techniques, this survey presents a case study of the Linear Quadratic Regulator (LQR) with unknown dynamics.
0
0
0
Wed May 02 2018
Machine Learning
Reinforcement Learning and Control as Probabilistic Inference: Tutorial and Review
Reinforcement learning or optimal control provides amathematical formalization of intelligent decision making that is powerful and broadly applicable. The connection between reinforcement learning and inference in probabilistic models is not immediately immediately obvious. In this article, we will discuss how a generalization of reinforcement learning
1
0
0
Fri Nov 20 2020
Machine Learning
MRAC-RL: A Framework for On-Line Policy Adaptation Under Parametric Model Uncertainty
Reinforcement learning (RL) algorithms have been successfully used to develop control policies for dynamical systems. For many such systems, these policies are trained in a simulated environment. RL trained policies often fail to generalize and adapt appropriately when deployed in the real-world.
0
0
0
Sun Jan 19 2020
Artificial Intelligence
A Survey of Reinforcement Learning Techniques: Strategies, Recent Development, and Future Directions
Reinforcement learning is one of the core components in designing an artificial intelligent system emphasizing real-time response. In this paper, we present a comprehensive study on Reinforcement Learning focusing on various dimensions.
0
0
0
Sat Jun 13 2020
Artificial Intelligence
Reinforcement Learning as Iterative and Amortised Inference
There are several ways to categorise reinforcement learning (RL) algorithms. This paper outlines a novel classification scheme based on amortised and iterative inference.
0
0
0
Fri Jan 03 2020
Artificial Intelligence
Making Sense of Reinforcement Learning and Probabilistic Inference
Reinforcement learning (RL) combines a control problem with statistical estimation. In all but the most simple settings, the resulting inference is computationally intractable. We demonstrate that the popular `RL as inference' approximation can perform poorly in even very basic problems.
0
0
0