Recursive Multi-Scale Image Deraining With Sub-Pixel Convolution Based Feature Fusion and Context Aggregation

• Published in: IEEE access Vol. 8; pp. 177495 - 177505
• Main Authors: Sharma, Suvash, Tang, Bo, Ball, John E., Carruth, Daniel W., Dabbiru, Lalitha
• Format: Journal Article
• Language: English
• Published: Piscataway IEEE 2020; The Institute of Electrical and Electronics Engineers, Inc. (IEEE)
• Subjects: Ablation; Agglomeration; Algorithms; Artificial neural networks; Computer architecture; Computer vision; Context; Convolution; feature fusion; Generators; Image reconstruction; Modules; multi-scale context aggregation; Optimization; Pixels; Rain; Signal to noise ratio; Single image deraining; sub-pixel convolution; Training; Weight reduction
• ISSN: 2169-3536; 2169-3536
• DOI: 10.1109/ACCESS.2020.3024542

Along with several other low-vision based computer vision problems, single image deraining is also taken as a challenging one due to its ill-posedness. Several algorithms based on convolutional neural networks are devised that are either too simple to provide acceptable deraining results due to under-deraining or have a complex architectures that may result into over-deraining. In this paper we propose a deraining algorithm that is capable of boosting the reconstruction/deraining quality without the problem of over or under-deraining. Along with the originally proposed network, two of its' light-weight versions with reduced computational costs are also devised. Basically, we propose a recursively trained architecture that has two major components: a front-end module and a refinement module. The front-end module is based on dense fusion of lower label features followed by sub-pixel convolutions (pixel shuffling based convolutions). To refine and generate the enhanced deraining results further, we cascade a refinement module to the front-end module using multi-scale Context Aggregation Network (CAN) which includes feature fusion and pixel shuffling based convolutions. We present the deraining results in terms of Structural Similarity Index (SSIM) and Peak Signal to Noise Ratio (PSNR) on several benchmarks and compare with current state-of-the-art algorithms. With comprehensive experiments on both real-world and synthetic datasets and extensive ablation study, we demonstrate that our approach produces better results compared to existing methods.