Improving the evidential value of low-quality face images with aggregation of deep neural network embeddings

• Published in: Science & justice Vol. 64; no. 5; pp. 509 - 520
• Main Authors: Ribeiro, Rafael Oliveira, Neves, João C., Ruifrok, Arnout, de Barros Vidal, Flavio
• Format: Journal Article
• Language: English
• Published: England Elsevier B.V 01-09-2024
• Subjects: Embedding aggregation; Face comparisons; Forensic evaluation; Likelihood ratio; Likelihood ratio; Forensic evaluation; Face comparisons; Embedding aggregation
• ISSN: 1355-0306; 1876-4452; 1876-4452
• DOI: 10.1016/j.scijus.2024.07.006

•An effective strategy to improve face recognition of low-quality facial images.•Validation of this strategy under forensically relevant challenging scenarios.•Cleaned versions of the BFW and Adience datasets are provided.•A new verification protocol for the Quis-Campi dataset.
In forensic facial comparison, questioned-source images are usually captured in uncontrolled environments, with non-uniform lighting, and from non-cooperative subjects. The poor quality of such material usually compromises their value as evidence in legal proceedings. On the other hand, in forensic casework, multiple images of the person of interest are usually available. In this paper, we propose to aggregate deep neural network embeddings from various images of the same person to improve the performance in forensic comparison of facial images. We observe significant performance improvements, especially for low-quality images. Further improvements are obtained by aggregating embeddings of more images and by applying quality-weighted aggregation. We demonstrate the benefits of this approach in forensic evaluation settings with the development and validation of common-source likelihood ratio systems and report improvements in Cllr both for CCTV images and for social media images.