Published on Fri Aug 28 2020

Real-world Video Adaptation with Reinforcement Learning

Hongzi Mao, Shannon Chen, Drew Dimmery, Shaun Singh, Drew Blaisdell, Yuandong Tian, Mohammad Alizadeh, Eytan Bakshy
0
0
0
Abstract

Client-side video players employ adaptive bitrate (ABR) algorithms to optimize user quality of experience (QoE). We evaluate recently proposed RL-based ABR methods in Facebook's web-based video streaming platform. Real-world ABR contains several challenges that requires customized designs beyond off-the-shelf RL algorithms -- we implement a scalable neural network architecture that supports videos with arbitrary bitrate encodings; we design a training method to cope with the variance resulting from the stochasticity in network conditions; and we leverage constrained Bayesian optimization for reward shaping in order to optimize the conflicting QoE objectives. In a week-long worldwide deployment with more than 30 million video streaming sessions, our RL approach outperforms the existing human-engineered ABR algorithms.

Tue Aug 06 2019
Artificial Intelligence
Comyco: Quality-Aware Adaptive Video Streaming via Imitation Learning
Learn-based Adaptive Bit Rate~(ABR) method has become one of the research hotspots for adaptive streaming. It typically suffers from several issues, including low sample efficiency and lack of awareness of the video quality information. In this paper, we propose Comyco, a video quality-aware
0
0
0
Fri Aug 21 2020
Machine Learning
NANCY: Neural Adaptive Network Coding methodologY for video distribution over wireless networks
NANCY trains a neural network model with rewards formulated as quality of experience (QoE) metrics. NANCY provides 29.91% and 60.34% higher average QoE than Pensieve and robustMPC, respectively.
0
0
0
Wed Dec 04 2019
Machine Learning
Reinforcement learning for bandwidth estimation and congestion control in real-time communications
Bandwidth estimation and congestion control for real-time communications (i.e., audio and video conferencing) remains a difficult problem, despite many years of research. Achieving high quality of experience (QoE) for end users requires continual updates due to changing network architectures.
0
0
0
Fri Aug 24 2018
Machine Learning
Towards Machine Learning-Based Optimal HAS
Mobile video consumption is increasing and sophisticated video quality adaptation strategies are required to deal with mobile throughput fluctuations. This paper proposes a novel methodology for the design of machine learning-based adaptation logics named HASBRAIN.
0
0
0
Thu Sep 06 2018
Machine Learning
Model-Based Regularization for Deep Reinforcement Learning with Transcoder Networks
This paper proposes a new optimization objective for value-based deep reinforcement learning. We extend conventional Deep Q-Networks by adding a model-learning component. The prediction errors for the model are included in the basic DQN loss as additional regularizers.
0
0
0
Sat Mar 21 2020
Machine Learning
Accelerating Deep Reinforcement Learning With the Aid of Partial Model: Energy-Efficient Predictive Video Streaming
The goal is to minimize accumulated energy consumption of each base station over a complete video streaming session under the constraint that avoids video playback interruptions. To handle the continuous state and action spaces, we resort to a deep deterministic policy gradient algorithm.
0
0
0