Published on Sun May 12 2019

Diagnosing Reinforcement Learning for Traffic Signal Control

Guanjie Zheng, Xinshi Zang, Nan Xu, Hua Wei, Zhengyao Yu, Vikash Gayah, Kai Xu, Zhenhui Li

The re is an emerging trend of employing reinforcement learning (RL) for traffic signal control. A key question for applying RL is how to define the reward and state. We propose to re-examine the RL approaches through the lens of classic transportation theory.

0
0
0
Abstract

With the increasing availability of traffic data and advance of deep reinforcement learning techniques, there is an emerging trend of employing reinforcement learning (RL) for traffic signal control. A key question for applying RL to traffic signal control is how to define the reward and state. The ultimate objective in traffic signal control is to minimize the travel time, which is difficult to reach directly. Hence, existing studies often define reward as an ad-hoc weighted linear combination of several traffic measures. However, there is no guarantee that the travel time will be optimized with the reward. In addition, recent RL approaches use more complicated state (e.g., image) in order to describe the full traffic situation. However, none of the existing studies has discussed whether such a complex state representation is necessary. This extra complexity may lead to significantly slower learning process but may not necessarily bring significant performance gain. In this paper, we propose to re-examine the RL approaches through the lens of classic transportation theory. We ask the following questions: (1) How should we design the reward so that one can guarantee to minimize the travel time? (2) How to design a state representation which is concise yet sufficient to obtain the optimal solution? Our proposed method LIT is theoretically supported by the classic traffic signal control methods in transportation field. LIT has a very simple state and reward design, thus can serve as a building block for future RL approaches to traffic signal control. Extensive experiments on both synthetic and real datasets show that our method significantly outperforms the state-of-the-art traffic signal control methods.

Thu Nov 14 2019
Machine Learning
Deep Reinforcement Learning for Adaptive Traffic Signal Control
Many existing traffic signal controllers are either simple adaptive controllers based on sensors placed around traffic intersections, or optimized by traffic engineers on a fixed schedule. Optimizing traffic controllers is time consuming and usually require experienced traffic engineers. Recent research has demonstrated the potential of using deep reinforcement learning in this context.
0
0
0
Sat May 02 2020
Machine Learning
Deep Reinforcement Learning for Intelligent Transportation Systems: A Survey
Latest technological improvements increased the quality of transportation. Combining data-driven applications with transportation systems plays a key role in recent transportation applications.
0
0
0
Sun Sep 01 2019
Artificial Intelligence
An Open-Source Framework for Adaptive Traffic Signal Control
Research contributes a framework for developing and evaluating different adaptive traffic signal controller models in simulation. Sub-optimal control policies in transportation systems negatively impact mobility, the environment and human health.
0
0
0
Mon Aug 06 2018
Artificial Intelligence
An Efficient Deep Reinforcement Learning Model for Urban Traffic Control
Urban Traffic Control (UTC) plays an essential role in Intelligent Transportation System (ITS) but remains difficult. New algorithm relaxes the fixed traffic demand pattern assumption and reduces human invention.
0
0
0
Mon Mar 09 2020
Machine Learning
Efficiency and Equity are Both Essential: A Generalized Traffic Signal Controller with Deep Reinforcement Learning
Traffic signal controllers play an essential role in today's traffic system. The majority of them currently is not sufficiently flexible or adaptive to generate optimal traffic schedules. In this paper we present an approach to learning policies for signal controllers using deep reinforcement.
0
0
0
Thu Feb 27 2020
Artificial Intelligence
Learning Scalable Multi-Agent Coordination by Spatial Differentiation for Traffic Signal Control
The intelligent control of the traffic signal is critical to the optimization of transportation systems. We propose the Spatial Differentiation method for coordination which uses the temporal-spatial information in the buffer to amend the reward of each action.
0
0
0