Using topic-noise models to generate domain-specific topics across data sources

• Published in: Knowledge and information systems Vol. 65; no. 5; pp. 2159 - 2186
• Main Authors: Churchill, Rob, Singh, Lisa
• Format: Journal Article
• Language: English
• Published: London Springer London 01-05-2023; Springer Nature B.V
• Subjects: Computer Science; Data Mining and Knowledge Discovery; Data sources; Database Management; Datasets; Digital media; Dirichlet problem; Documents; Effectiveness; Graph theory; Information Storage and Retrieval; Information Systems and Communication Service; Information Systems Applications (incl.Internet); IT in Business; Noise generation; Noise levels; Regular Paper; Social networks; Topic blending; Generative topic modeling; Topic noise model
• ISSN: 0219-1377; 0219-3116
• DOI: 10.1007/s10115-022-01805-2

Domain-specific document collections, such as data sets about the COVID-19 pandemic, politics, and sports, have become more common as platforms grow and develop better ways to connect people whose interests align. These data sets come from many different sources, ranging from traditional sources like open-ended surveys and newspaper articles to one of the dozens of online social media platforms. Most topic models are equipped to generate topics from one or more of these data sources, but models rarely work well across all types of documents. The main problem that many models face is the varying noise levels inherent in different types of documents. We propose topic-noise models, a new type of topic model that jointly models topic and noise distributions to produce a more accurate, flexible representation of documents regardless of their origin and varying qualities. Our topic-noise model, Topic Noise Discriminator (TND) approximates topic and noise distributions side-by-side with the help of word embedding spaces. While topic-noise models are important for the types of short, noisy documents that often originate on social media platforms, TND can also be used with more traditional data sources like newspapers. TND itself generates a noise distribution that when ensembled with other generative topic models can produce more coherent and diverse topic sets. We show the effectiveness of this approach using Latent Dirichlet Allocation (LDA), and demonstrate the ability of TND to improve the quality of LDA topics in noisy document collections. Finally, researchers are beginning to generate topics using multiple sources and finding that they need a way to identify a core set based on text from different sources. We propose using cross-source topic blending (CSTB), an approach that maps topics sets to an
s
-partite graph and identifies core topics that blend topics from across
s
sources by identifying subgraphs with certain linkage properties. We demonstrate the effectiveness of topic-noise models and CSTB empirically on large real-world data sets from multiple domains and data sources.