Towards Faithful Model Explanation in NLP: A Survey

• Main Authors: Lyu, Qing, Apidianaki, Marianna, Callison-Burch, Chris
• Format: Journal Article
• Language: English
• Published: 22-09-2022
• Subjects: Computer Science - Computation and Language
• DOI: 10.48550/arxiv.2209.11326

End-to-end neural Natural Language Processing (NLP) models are notoriously
difficult to understand. This has given rise to numerous efforts towards model
explainability in recent years. One desideratum of model explanation is
faithfulness, i.e. an explanation should accurately represent the reasoning
process behind the model's prediction. In this survey, we review over 110 model
explanation methods in NLP through the lens of faithfulness. We first discuss
the definition and evaluation of faithfulness, as well as its significance for
explainability. We then introduce recent advances in faithful explanation,
grouping existing approaches into five categories: similarity-based methods,
analysis of model-internal structures, backpropagation-based methods,
counterfactual intervention, and self-explanatory models. For each category, we
synthesize its representative studies, strengths, and weaknesses. Finally, we
summarize their common virtues and remaining challenges, and reflect on future
work directions towards faithful explainability in NLP.