Out-of-Distribution Generalization via Style and Spuriousness Eliminating

• Published in: 2024 IEEE International Conference on Multimedia and Expo (ICME) pp. 1 - 6
• Main Authors: Jin, Kaiyu, Wu, Chenwang, Lian, Defu
• Format: Conference Proceeding
• Language: English
• Published: IEEE 15-07-2024
• Subjects: Buildings; Causality; Computer Vision; Correlation; Data models; Deep learning; Domain Generalization; Training data
• ISSN: 1945-788X
• DOI: 10.1109/ICME57554.2024.10687911

The deep learning model's performance may be compromised when the test data's distribution shifts from the training data. Distribution shifts can be categorized into the correlation shift and the diversity shift, and existing methods typically address only one. To this end, we propose a novel causal graph that consists of three latent variables: causal variable, spurious variable, and style variable, to explain both types of distribution shift. We provide formal definitions of these latent variables and find that the spurious and style variables contribute to the correlation and diversity shifts, respectively. Building upon this, we present the Style and Spuriousness Eliminating (SSE) method to simultaneously tackle both distribution shifts. Specifically, we utilize style intervention and feature alignment to remove the influence of style factors, and employ adversarial mask learning to mitigate the impact of spurious factors, ultimately retaining causal features for prediction. Experimental results demonstrate that our SSE method outperforms previous approaches on datasets with different distribution shifts.