Attention-driven tile splitting method for improved efficiency of omnidirectional versatile video coding

• Published in: 2021 IEEE International Conference on Image Processing (ICIP) pp. 2149 - 2153
• Main Authors: Carreira, J., de Faria, Sergio M. M., Tavora, Luis M. N., Navarro, Antonio, Assuncao, Pedro A.
• Format: Conference Proceeding
• Language: English
• Published: IEEE 19-09-2021
• Subjects: adaptive 360° streaming; Bandwidth; Bit rate; Image coding; Minimization; Omnidirectional video coding; Streaming media; tile-based coding; Video coding; Visualization
• ISSN: 2381-8549
• DOI: 10.1109/ICIP42928.2021.9506358

A common approach used in omnidirectional video coding is based on frame splitting into tiles, allowing partial delivery of only the subset of tiles that is necessary to render the user's current viewing region, defined as a specific viewport or Field-of-View (FoV). Since tiles can be independently encoded, such mechanism provides a flexible solution for encoding planar representations with ultra-high definition (UHD), such as the Equirectangular Projection (ERP), using Versatile Video Coding (VVC). By only selecting and transmitting the coded data that is required to render the necessary FoV, rather than the full 360°, a great deal of bandwidth can be saved. While current solutions are based on splitting the omnidirectional video frames into tiles of equal size, this paper proposes a new approach based on adaptive tile size, driven by visual attention. Those regions where the visual attention is higher are partitioned in smaller tiles to obtain higher bit rate granularity, allowing to decode the most frequent FoVs with minimum out-of-FoV pixels and reduced bandwidth. Optimal tile boundaries are found by solving a lagrangian minimisation problem with a cost function that achieves the best tradeoff between the standard deviation and the average attention-weighted bit rate per tile. The experimental results show that an average of 7.17% and 17.73% of bit rate savings is obtained in comparison with conventional tilling methods for the commonly used FoVs of 90^{\circ} \times 90^{\circ} and 45^{\circ} \times 45^{\circ}, respectively.