Generating Cyber Threat Intelligence to Discover Potential Security Threats Using Classification and Topic Modeling

• Main Authors: Hossen, Md Imran, Islam, Ashraful, Anowar, Farzana, Ahmed, Eshtiak, Rahman, Mohammad Masudur, Xiali, Hei
• Format: Journal Article
• Language: English
• Published: 15-08-2021
• Subjects: Computer Science - Cryptography and Security; Computer Science - Learning
• DOI: 10.48550/arxiv.2108.06862

Due to the variety of cyber-attacks or threats, the cybersecurity community
enhances the traditional security control mechanisms to an advanced level so
that automated tools can encounter potential security threats. Very recently,
Cyber Threat Intelligence (CTI) has been presented as one of the proactive and
robust mechanisms because of its automated cybersecurity threat prediction.
Generally, CTI collects and analyses data from various sources e.g., online
security forums, social media where cyber enthusiasts, analysts, even
cybercriminals discuss cyber or computer security-related topics and discovers
potential threats based on the analysis. As the manual analysis of every such
discussion (posts on online platforms) is time-consuming, inefficient, and
susceptible to errors, CTI as an automated tool can perform uniquely to detect
cyber threats. In this paper, we identify and explore relevant CTI from hacker
forums utilizing different supervised (classification) and unsupervised
learning (topic modeling) techniques. To this end, we collect data from a real
hacker forum and constructed two datasets: a binary dataset and a multi-class
dataset. We then apply several classifiers along with deep neural network-based
classifiers and use them on the datasets to compare their performances. We also
employ the classifiers on a labeled leaked dataset as our ground truth. We
further explore the datasets using unsupervised techniques. For this purpose,
we leverage two topic modeling algorithms namely Latent Dirichlet Allocation
(LDA) and Non-negative Matrix Factorization (NMF).