A Concept-Based Explainability Framework for Large Multimodal Models

• Main Authors: Parekh, Jayneel, Khayatan, Pegah, Shukor, Mustafa, Newson, Alasdair, Cord, Matthieu
• Format: Journal Article
• Language: English
• Published: 12-06-2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language; Computer Science - Computer Vision and Pattern Recognition; Computer Science - Learning
• DOI: 10.48550/arxiv.2406.08074

Large multimodal models (LMMs) combine unimodal encoders and large language
models (LLMs) to perform multimodal tasks. Despite recent advancements towards
the interpretability of these models, understanding internal representations of
LMMs remains largely a mystery. In this paper, we present a novel framework for
the interpretation of LMMs. We propose a dictionary learning based approach,
applied to the representation of tokens. The elements of the learned dictionary
correspond to our proposed concepts. We show that these concepts are well
semantically grounded in both vision and text. Thus we refer to these as
``multi-modal concepts''. We qualitatively and quantitatively evaluate the
results of the learnt concepts. We show that the extracted multimodal concepts
are useful to interpret representations of test samples. Finally, we evaluate
the disentanglement between different concepts and the quality of grounding
concepts visually and textually. Our implementation is publicly available.