Bayesian Naïve Bayes classifiers to text classification

• Published in: Journal of information science Vol. 44; no. 1; pp. 48 - 59
• Main Author: Xu, Shuo
• Format: Journal Article
• Language: English
• Published: London, England SAGE Publications 01-02-2018; Bowker-Saur Ltd
• Subjects: Bayesian analysis; Classification; Classifiers; Text categorization; Naïve Bayes classifier; text classification; Bayesian Naïve Bayes classifier; event model
• ISSN: 0165-5515; 1741-6485
• DOI: 10.1177/0165551516677946

Text classification is the task of assigning predefined categories to natural language documents, and it can provide conceptual views of document collections. The Naïve Bayes (NB) classifier is a family of simple probabilistic classifiers based on a common assumption that all features are independent of each other, given the category variable, and it is often used as the baseline in text classification. However, classical NB classifiers with multinomial, Bernoulli and Gaussian event models are not fully Bayesian. This study proposes three Bayesian counterparts, where it turns out that classical NB classifier with Bernoulli event model is equivalent to Bayesian counterpart. Finally, experimental results on 20 newsgroups and WebKB data sets show that the performance of Bayesian NB classifier with multinomial event model is similar to that of classical counterpart, but Bayesian NB classifier with Gaussian event model is obviously better than classical counterpart.