Pixel-Level Non-local Image Smoothing With Objective Evaluation

• Published in: IEEE transactions on multimedia Vol. 23; pp. 4065 - 4078
• Main Authors: Xu, Jun, Liu, Zhi-Ang, Hou, Ying-Kun, Zhen, Xian-Tong, Shao, Ling, Cheng, Ming-Ming
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2021; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ablation; Algorithms; benchmark dataset; Benchmark testing; Data smoothing; Datasets; Image edge detection; Image enhancement; Image processing; Image smoothing; Measurement; Optimization; performance evaluation; pixel-level non-local self similarity; Pixels; Self-similarity; Smoothing methods; Task analysis; Visualization
• ISSN: 1520-9210; 1941-0077
• DOI: 10.1109/TMM.2020.3037535

Recently, imagesmoothing has gained increasing attention due to its prerequisite role in other image processing tasks, e.g., image enhancement and editing. However, the evaluation of image smoothing algorithms is usually performed by subjective observation on images without corresponding ground truths. To promote the development of image smoothing algorithms, in this paper, we construct a novel Nankai Smoothing (NKS) dataset containing 200 images blended by versatile structure images and natural textures. The structure images are inherently smooth and naturally taken as ground truths. On our NKS dataset, we comprehensively evaluate 14 popular image smoothing algorithms. Moreover, we propose a Pixel-level Non-Local Smoothing (PNLS) method to well preserve the structure of the smoothed images, by exploiting the pixel-level non-local self-similarity prior of natural images. Extensive experiments on several benchmark datasets demonstrate that our PNLS outperforms previous algorithms on the image smoothing task. Ablation studies also reveal the work mechanism of our PNLS on image smoothing. To further show its effectiveness, we apply our PNLS on several applications such as semantic region smoothing, detail/edge enhancement, and image abstraction. The dataset and code are available at https://github.com/zal0302/PNLS .