Multi-resolution Interpretation and Diagnostics Tool for Natural Language Classifiers

• Main Authors: Jalali, Peyman, Zhou, Nengfeng, Yu, Yufei
• Format: Journal Article
• Language: English
• Published: 06-03-2023
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computation and Language
• DOI: 10.48550/arxiv.2303.03542

Developing explainability methods for Natural Language Processing (NLP)
models is a challenging task, for two main reasons. First, the high
dimensionality of the data (large number of tokens) results in low coverage and
in turn small contributions for the top tokens, compared to the overall model
performance. Second, owing to their textual nature, the input variables, after
appropriate transformations, are effectively binary (presence or absence of a
token in an observation), making the input-output relationship difficult to
understand. Common NLP interpretation techniques do not have flexibility in
resolution, because they usually operate at word-level and provide fully local
(message level) or fully global (over all messages) summaries. The goal of this
paper is to create more flexible model explainability summaries by segments of
observation or clusters of words that are semantically related to each other.
In addition, we introduce a root cause analysis method for NLP models, by
analyzing representative False Positive and False Negative examples from
different segments. At the end, we illustrate, using a Yelp review data set
with three segments (Restaurant, Hotel, and Beauty), that exploiting
group/cluster structures in words and/or messages can aid in the interpretation
of decisions made by NLP models and can be utilized to assess the model's
sensitivity or bias towards gender, syntax, and word meanings.