Published on Tue Mar 13 2018

Multi-Frame Quality Enhancement for Compressed Video

Ren Yang, Mai Xu, Zulin Wang, Tianyi Li

Heavy quality fluctuation exists across compressed video frames. Low quality frames can be enhanced using the neighboring high quality frames, seen as Multi-Frame Quality Enhancement (MFQE) This paper proposes an MFQE approach for compressed video, as a first attempt.

0
0
0
Abstract

The past few years have witnessed great success in applying deep learning to enhance the quality of compressed image/video. The existing approaches mainly focus on enhancing the quality of a single frame, ignoring the similarity between consecutive frames. In this paper, we investigate that heavy quality fluctuation exists across compressed video frames, and thus low quality frames can be enhanced using the neighboring high quality frames, seen as Multi-Frame Quality Enhancement (MFQE). Accordingly, this paper proposes an MFQE approach for compressed video, as a first attempt in this direction. In our approach, we firstly develop a Support Vector Machine (SVM) based detector to locate Peak Quality Frames (PQFs) in compressed video. Then, a novel Multi-Frame Convolutional Neural Network (MF-CNN) is designed to enhance the quality of compressed video, in which the non-PQF and its nearest two PQFs are as the input. The MF-CNN compensates motion between the non-PQF and PQFs through the Motion Compensation subnet (MC-subnet). Subsequently, the Quality Enhancement subnet (QE-subnet) reduces compression artifacts of the non-PQF with the help of its nearest PQFs. Finally, the experiments validate the effectiveness and generality of our MFQE approach in advancing the state-of-the-art quality enhancement of compressed video. The code of our MFQE approach is available at https://github.com/ryangBUAA/MFQE.git

Tue Feb 26 2019
Computer Vision
MFQE 2.0: A New Approach for Multi-frame Quality Enhancement on Compressed Video
This paper proposes an MFQE approach for compressed video. It uses a novel Multi-Frame.Convolutional Neural Network (MF-CNN) to enhance the quality of compressed video, in which the non-PQF and its nearest two PQFs are the input.
0
0
0
Mon Mar 11 2019
Computer Vision
Quality-Gated Convolutional LSTM for Enhancing Compressed Video
The paper proposes the Quality-Gated Convolutional Long Short-Term Memory (QG-ConvLSTM) network. It uses a bi-directional recurrent structure to fully exploit the advantageous information in a large range of frames.
0
0
0
Mon May 31 2021
Computer Vision
Boosting the Performance of Video Compression Artifact Reduction with Reference Frame Proposals and Frequency Domain Information
0
0
0
Wed Mar 04 2020
Computer Vision
Learning for Video Compression with Hierarchical Quality and Recurrent Enhancement
Hierarchical Learned Video Compression (HLVC) has three hierarchical quality layers and a recurrent enhancement network. In our HLVC approach, the hierarchical quality benefits the coding efficiency, since the high quality information facilitates the compression and enhancement of low quality frames.
0
0
0
Fri Nov 30 2018
Computer Vision
DVC: An End-to-end Deep Video Compression Framework
Conventional video compression approaches use the predictive coding architecture and encode the corresponding motion information and residual information. In this paper, we propose the first end-to-end video compression deep model that jointly optimizes all the components for video compression.
0
0
0
Thu Nov 22 2018
Computer Vision
MGANet: A Robust Model for Quality Enhancement of Compressed Video
In video compression, most of the existing deep learning approaches concentrate on the visual quality of a single frame. We propose a multi-frame guided attention network (MGANet) to enhance the quality of compressed videos.
0
0
0