MRG-T: Mask-Relation-Guided Transformer for Remote Vision-Based Pedestrian Attribute Recognition in Aerial Imagery

• Published in: Remote sensing (Basel, Switzerland) Vol. 16; no. 7; p. 1216
• Main Authors: Zhang, Shun, Li, Yupeng, Wu, Xiao, Chu, Zunheng, Li, Lingfei
• Format: Journal Article
• Language: English
• Published: Basel MDPI AG 01-04-2024
• Subjects: aerial imagery; Aerial photography; Artificial intelligence; Benchmarks; Comparative analysis; Computer vision; Drone aircraft; Identification and classification; Imagery; Labels; Machine learning; Mapping; masked attention mechanism; Methods; Modelling; Modules; Neural networks; pedestrian attribute recognition; Pedestrians; Photography, Aerial; Recognition; Redundancy; relation modeling; Semantic relations; Semantics; Sensory integration; Spatial data; Surveillance; Transformers; Unmanned aerial vehicles; China
• ISSN: 2072-4292; 2072-4292
• DOI: 10.3390/rs16071216

Nowadays, with the rapid development of consumer Unmanned Aerial Vehicles (UAVs), utilizing UAV platforms for visual surveillance has become very attractive, and a key part of this is remote vision-based pedestrian attribute recognition. Pedestrian Attribute Recognition (PAR) is dedicated to predicting multiple attribute labels of a single pedestrian image extracted from surveillance videos and aerial imagery, which presents significant challenges in the computer vision community due to factors such as poor imaging quality and substantial pose variations. Despite recent studies demonstrating impressive advancements in utilizing complicated architectures and exploring relations, most of them may fail to fully and systematically consider the inter-region, inter-attribute, and region-attribute mapping relations simultaneously and be stuck in the dilemma of information redundancy, leading to the degradation of recognition accuracy. To address the issues, we construct a novel Mask-Relation-Guided Transformer (MRG-T) framework that consists of three relation modeling modules to fully exploit spatial and semantic relations in the model learning process. Specifically, we first propose a Masked Region Relation Module (MRRM) to focus on precise spatial attention regions to extract more robust features with masked random patch training. To explore the semantic association of attributes, we further present a Masked Attribute Relation Module (MARM) to extract intrinsic and semantic inter-attribute relations with an attribute label masking strategy. Based on the cross-attention mechanism, we finally design a Region and Attribute Mapping Module (RAMM) to learn the cross-modal alignment between spatial regions and semantic attributes. We conduct comprehensive experiments on three public benchmarks such as PETA, PA-100K, and RAPv1, and conduct inference on a large-scale airborne person dataset named PRAI-1581. The extensive experimental results demonstrate the superior performance of our method compared to state-of-the-art approaches and validate the effectiveness of mask-relation-guided modeling in the remote vision-based PAR task.