No-Reference Quality Assessment for View Synthesis Using DoG-Based Edge Statistics and Texture Naturalness

• Published in: IEEE transactions on image processing Vol. 28; no. 9; pp. 4566 - 4579
• Main Authors: Zhou, Yu, Li, Leida, Wang, Shiqi, Wu, Jinjian, Fang, Yuming, Gao, Xinbo
• Format: Journal Article
• Language: English
• Published: United States IEEE 01-09-2019; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Algorithms; Computer simulation; Difference-of-Gaussian; Distortion; Dogs; Eye (anatomy); Feature extraction; gray level gradient co-occurrence matrix; Image acquisition; Image edge detection; Image quality; Measurement; orientation selective statistics; Quality; Quality assessment; quality evaluation; Regression models; Rendering; Rendering (computer graphics); Retina; Statistics; Synthesis; Texture; texture naturalness; View synthesis
• ISSN: 1057-7149; 1941-0042
• DOI: 10.1109/TIP.2019.2912463

View synthesis is a key technique in free-viewpoint video, which renders virtual views based on texture and depth images. The distortions in synthesized views come from two stages, i.e., the stage of the acquisition and processing of texture and depth images, and the rendering stage using depth-image-based-rendering (DIBR) algorithms. The existing view synthesis quality metrics are designed for the distortions caused by a single stage, which cannot accurately evaluate the quality of the entire view synthesis process. With the considerations that the distortions introduced by two stages both cause edge degradation and texture unnaturalness, and the Difference-of-Gaussian (DoG) representation is powerful in capturing image edge and texture characteristics by simulating the center-surrounding receptive fields of retinal ganglion cells of human eyes, this paper presents a no-reference quality index for Synthesized views using DoG-based Edge statistics and Texture naturalness (SET). To mimic the multi-scale property of the human visual system (HVS), DoG images are first calculated at multiple scales. Then, the orientation selective statistics features and the texture naturalness features are calculated on the DoG images and the coarsest scale image, producing two groups of quality-aware features. Finally, the quality model is learnt from these features using the random forest regression model. The experimental results on two view synthesis image databases demonstrate that the proposed metric is advantageous over the relevant state of the art in dealing with the distortions in the whole view synthesis process.