Towards coding for VoD application: An enhanced video compression system with a content-fitted recursive restoration network

• Published in: Digital signal processing Vol. 122; p. 103368
• Main Authors: Xu, Li, Wu, Chang, Huang, Linyi, Wei, Shengyu, He, Gang, Li, Yunsong, Lai, Xinquan
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 15-04-2022
• Subjects: Content-fitting; Deep neural network; Video compression; Video on demand; Video compression; Deep neural network; Video on demand; Content-fitting
• ISSN: 1051-2004; 1095-4333
• DOI: 10.1016/j.dsp.2021.103368

The video on demand (VoD) applications require real-time decoding for customers while pursuing a better compression performance. Recent work has demonstrated the great potential of deep learning techniques in video compression. Building a generic model to accommodate various videos and meet the real-time decoding is an arduous task currently. However, the ground truth, i.e. raw video herein, is available during the video encoding process in practice, which means generalization is unnecessary. Inspired by this, we propose an enhanced video compression system (EVCS) using a content-fitted recursive restoration network (CRRN) for VoD. On the encoder side, a lightweight CRRN adopting the recursive restoration strategy is trained for a group of consecutive frames to be well-fitted to this group and achieve a strong restoration ability. After that, the learned parameters of CRRN are transmitted to the decoder with the encoded bitstream. On the decoder side, CRRN can perform the same robust restoration operation on the decoded frames. The proposed framework can be compatible with all of the existing video coding systems. Experimental results show that the compression performance can be significantly improved in terms of 5.938% - 24.413% BD-BR reduction by integrating CRRN with VTM, HM, X265, X264, NVENC, and two end-to-end DNN codecs DVC, M-LVC. This allows the H.264 and H.265 codec IPs developed with a lot of effort to be reused. Compared with other DNN based restoration works over HM, this work achieves the best compression performance improvement with fewer FLOPs for DNN computation. EVCS can satisfy the real-time decoding for 1080p 30fps videos on the AI edge platform.