Keywords extraction with deep neural network model

• Published in: Neurocomputing (Amsterdam) Vol. 383; pp. 113 - 121
• Main Authors: Zhang, Yu, Tuo, Mingxiang, Yin, Qingyu, Qi, Le, Wang, Xuxiang, Liu, Ting
• Format: Journal Article
• Language: English
• Published: Elsevier B.V 28-03-2020
• Subjects: Attention mechanism; Deep learning; Keyword extraction; LSTM; Two-stage training; Deep learning; Attention mechanism; Keyword extraction; LSTM; Two-stage training
• ISSN: 0925-2312; 1872-8286
• DOI: 10.1016/j.neucom.2019.11.083

Keywords can express the main content of an article or a sentence. Keywords extraction is a critical issue in many Natural Language Processing (NLP) applications and can improve the performance of many NLP systems. The traditional methods of keywords extraction are based on machine learning or graph model. The performance of these methods is influenced by the feature selection and the manually defined rules. In recent years, with the emergence of deep learning technology, learning features automatically with the deep learning algorithm can improve the performance of many tasks. In this paper, we propose a deep neural network model for the task of keywords extraction. We make two extensions on the basis of traditional LSTM model. First, to better utilize both the historic and following contextual information of the given target word, we propose a target center-based LSTM model (TC-LSTM), which learns to encode the target word by considering its contextual information. Second, on the basis of TC-LSTM model, we apply the self-attention mechanism, which enables our model has an ability to focus on informative parts of the associated text. In addition, we also introduce a two-stage training method, which takes advantage of large-scale pseudo training data. Experimental results show the advantage of our method, our model beats all the baseline systems all across the board. And also, the two-stage training method is of great significance for improving the effectiveness of the model.