SetRank: Learning a Permutation-Invariant Ranking Model for Information Retrieval

• Main Authors: Pang, Liang, Xu, Jun, Ai, Qingyao, Lan, Yanyan, Cheng, Xueqi, Wen, Jirong
• Format: Journal Article
• Language: English
• Published: 12-12-2019
• Subjects: Computer Science - Information Retrieval
• DOI: 10.48550/arxiv.1912.05891

In learning-to-rank for information retrieval, a ranking model is
automatically learned from the data and then utilized to rank the sets of
retrieved documents. Therefore, an ideal ranking model would be a mapping from
a document set to a permutation on the set, and should satisfy two critical
requirements: (1)~it should have the ability to model cross-document
interactions so as to capture local context information in a query; (2)~it
should be permutation-invariant, which means that any permutation of the
inputted documents would not change the output ranking. Previous studies on
learning-to-rank either design uni-variate scoring functions that score each
document separately, and thus failed to model the cross-document interactions;
or construct multivariate scoring functions that score documents sequentially,
which inevitably sacrifice the permutation invariance requirement. In this
paper, we propose a neural learning-to-rank model called SetRank which directly
learns a permutation-invariant ranking model defined on document sets of any
size. SetRank employs a stack of (induced) multi-head self attention blocks as
its key component for learning the embeddings for all of the retrieved
documents jointly. The self-attention mechanism not only helps SetRank to
capture the local context information from cross-document interactions, but
also to learn permutation-equivariant representations for the inputted
documents, which therefore achieving a permutation-invariant ranking model.
Experimental results on three large scale benchmarks showed that the SetRank
significantly outperformed the baselines include the traditional
learning-to-rank models and state-of-the-art Neural IR models.