Using the Maximum Entropy Method for Natural Language Processing: Category Estimation, Feature Extraction, and Error Correction

• Published in: Cognitive computation Vol. 2; no. 4; pp. 272 - 279
• Main Authors: Murata, Masaki, Uchimoto, Kiyotaka, Utiyama, Masao, Ma, Qing, Nishimura, Ryo, Watanabe, Yasuhiko, Doi, Kouichi, Torisawa, Kentaro
• Format: Journal Article
• Language: English
• Published: New York Springer-Verlag 01-12-2010; Springer Nature B.V
• Subjects: Accuracy; Artificial Intelligence; Bilingualism; Biomedical and Life Sciences; Biomedicine; Computation by Abstract Devices; Computational Biology/Bioinformatics; Datasets; Error correction; Feature extraction; Information retrieval; Machine learning; Machine translation; Maximum entropy method; Natural language processing; Neurosciences; Software; Supervised learning; Translations; Category estimation; Error correction; Important feature; Maximum entropy
• ISSN: 1866-9956; 1866-9964
• DOI: 10.1007/s12559-010-9046-3

The maximum entropy (ME) method is a powerful supervised machine learning technique that is useful for various tasks. In this paper, we introduce new studies that successfully employ ME for natural language processing (NLP) problems including machine translation and information extraction. Specifically, we demonstrate, using simulation results, three applications of ME for NLP: estimation of categories, extraction of important features, and correction of error data items. We also evaluate the comparative performance of the proposed ME methods with other state-of-the-art approaches.