Automatic corrections of human body depth maps using deep neural networks

• Published in: Serbian journal of electrical engineering Vol. 17; no. 3; pp. 285 - 296
• Main Authors: Gojic, Gorana, Turovic, Radovan, Dragan, Dinu, Gajic, Dusan, Petrovic, Veljko
• Format: Journal Article
• Language: English
• Published: Faculty of Technical Sciences in Cacak 01-01-2020
• Subjects: deep learning; depth map refinement; human body; photogrammetry; segmentation
• ISSN: 1451-4869; 2217-7183
• DOI: 10.2298/SJEE2003285G

This paper presents an approach to correcting misclassified pixels in depth
maps representing parts of the human body. A misclassified pixel is a pixel
of a depth map which, incorrectly, has the ?background? value and does not
accurately reflect the distance from the sensor to the body being scanned. A
completely automatic, deep learning based solution for depth map correction
is proposed. As an input, the solution requires a color image and a
corresponding erroneous depth map. The input color image is segmented using
deep neural network for human body segmentation. The extracted segments are
further used as guidance to find and amend the misclassified pixels on the
depth map using a simple average based filter. Unlike other depth map
refinement solutions, this paper designs a method for the improvement of the
input depth map in terms of completeness instead of precision. The proposed
method does not exclude the application of other refinement methods.
Instead, it can be used as the first step in a depth map enhancement
pipeline to determine approximate depths for erroneous pixels, while other
refinement methods can be applied in a second step to improve the accuracy
of the recovered depths.