LTVAL: Label Transfer Virtual Adversarial Learning framework for source-free facial expression recognition

• Published in: Multimedia tools and applications Vol. 83; no. 2; pp. 5207 - 5228
• Main Authors: Guo, Zhe, Liu, Yang, Liu, Xuewen, Pan, Zhaojun, Liu, Shiya, Fan, Yangyu
• Format: Journal Article
• Language: English
• Published: New York Springer US 2024; Springer Nature B.V
• Subjects: Clustering; Computer Communication Networks; Computer Science; Constraint modelling; Data Structures and Information Theory; Datasets; Deep learning; Entropy; Face recognition; Labels; Learning; Multiculturalism & pluralism; Multimedia; Multimedia Information Systems; Neural networks; Optimization; Special Purpose and Application-Based Systems; Facial expression recognition; Information maximization; Virtual adversarial training; Source-free facial expression recognition
• ISSN: 1380-7501; 1573-7721
• DOI: 10.1007/s11042-023-15297-x

Previous research on cross-domain Facial Expression Recognition (FER) mainly focused on metric learning or adversarial learning, which presupposes access to source domain data to find domain invariant information. However, in practical applications, due to the high privacy and sensitivity of face data, it is often impossible to directly obtain source domain data. In this case, these methods cannot be effectively applied. In order to better apply the cross-domain FER method to the real scenarios, this paper proposes a source-free FER method called Label Transfer Virtual Adversarial Learning (LTVAL), which does not need to directly access source domain data. First, we train the target domain model based on the information maximization constraint, and obtain the pseudo-labels of the target domain data through deep clustering to achieve label transfer. Secondly, the perturbation is added to the target domain samples, and the perturbed samples and the original samples are together used for virtual adversarial training with local distributed smoothing constraints. Finally, a joint loss function is constructed to optimize the target domain model. Using the source domain model trained on RAF-DB, experiments on four public datasets FER2013, JAFFE, CK+, and EXPW as target domain datasets show that our approach achieves much higher performance than the state-of-the-art cross-domain FER methods that require access to source domain data.