Published on Sun Feb 21 2021

Learning Efficient Navigation in Vortical Flow Fields

Peter Gunnarson, Ioannis Mandralis, Guido Novati, Petros Koumoutsakos, John O. Dabiri

Efficient point-to-point navigation in the presence of a background flow field is important for robotic applications such as ocean surveying. In such applications, robots may only have knowledge of their immediate surroundings or be faced with time-varying currents.

0
0
0
Abstract

Efficient point-to-point navigation in the presence of a background flow field is important for robotic applications such as ocean surveying. In such applications, robots may only have knowledge of their immediate surroundings or be faced with time-varying currents, which limits the use of optimal control techniques for planning trajectories. Here, we apply a novel Reinforcement Learning algorithm to discover time-efficient navigation policies to steer a fixed-speed swimmer through an unsteady two-dimensional flow field. The algorithm entails inputting environmental cues into a deep neural network that determines the swimmer's actions, and deploying Remember and Forget Experience replay. We find that the resulting swimmers successfully exploit the background flow to reach the target, but that this success depends on the type of sensed environmental cue. Surprisingly, a velocity sensing approach outperformed a bio-mimetic vorticity sensing approach by nearly two-fold in success rate. Equipped with local velocity measurements, the reinforcement learning algorithm achieved near 100% success in reaching the target locations while approaching the time-efficiency of paths found by a global optimal control planner.

Mon Jan 30 2017
Machine Learning
Flow Navigation by Smart Microswimmers via Reinforcement Learning
Smart active particles can acquire some limited knowledge of the fluid environment from simple mechanical cues. Their goal is to learn the best way to navigate by exploiting the underlying flow whenever possible. Areinforcement learning algorithm allows particles to learn effective strategies even in difficult situations.
0
0
0
Wed Jun 17 2020
Machine Learning
Deep Reinforcement Learning Controller for 3D Path-following and Collision Avoidance by Autonomous Underwater Vehicles
In complex systems, such as autonomous underwater vehicles performing the dual objective of path-following and collision avoidance, decision making becomes non-trivial. We propose a solution using state-of-the-art Deep Reinforcement Learning techniques.
0
0
0
Sat Feb 27 2021
Machine Learning
Optimal control of point-to-point navigation in turbulent time-dependent flows using Reinforcement Learning
We present theoretical and numerical results concerning the problem to find the path that minimizes the time to navigate between two given points in acomplex fluid. We contrast deterministic Optimal Navigation (ON) control with stochastic policies obtained by Reinforcement Learning (RL) algorithms.
0
0
0
Wed Jul 17 2019
Artificial Intelligence
Zermelo's problem: Optimal point-to-point navigation in 2D turbulent flows using Reinforcement Learning
To find the path that minimizes the time to navigate between two given points is known as Zermelo's problem. We show that an Actor-Critic RL algorithm is able to find quasi-optimal solutions for both time-independent and chaotically evolving configurations.
0
0
0
Fri Jan 10 2020
Artificial Intelligence
Deep Interactive Reinforcement Learning for Path Following of Autonomous Underwater Vehicle
Autonomous underwater vehicle (AUV) plays an increasingly important role in ocean exploration. Existing AUVs are usually not fully autonomous and generally limited to pre-planning or pre-programming tasks. Reinforcement learning (RL) and deep reinforcement learning have been introduced into the AUV design and
0
0
0
Wed Feb 07 2018
Artificial Intelligence
Efficient collective swimming by harnessing vortices through deep reinforcement learning
Fish can improve their propulsive efficiency by actively following, and judiciously intercepting, vortices in the wake of other swimmers. This swimming strategy progressivelyleads to collective energy-savings.
0
0
0