Evaluation of the trends in jobs and skill-sets using data analytics: a case study

• Published in: Journal of big data Vol. 9; no. 1; pp. 1 - 28
• Main Authors: Alibasic, Armin, Upadhyay, Himanshu, Simsekler, Mecit Can Emre, Kurfess, Thomas, Woon, Wei Lee, Omar, Mohammed Atif
• Format: Journal Article
• Language: English
• Published: Cham Springer International Publishing 19-03-2022; Springer Nature B.V; SpringerOpen
• Subjects: Big Data; Case studies; Case Study; Communications Engineering; Computational Science and Engineering; Computer Science; Data analysis; Data analytics; Data mining; Data Mining and Knowledge Discovery; Database Management; Decision making; Dirichlet problem; Factor analysis; Future of work; Information Storage and Retrieval; Jobs and skills analysis; Labor market; Mathematical Applications in Computer Science; Natural language processing; Networks; New technology; Skills; Jobs and skills analysis; Data analytics; Natural language processing; Data mining; Future of work
• ISSN: 2196-1115; 2196-1115
• DOI: 10.1186/s40537-022-00576-5

Introduction
Fast-emerging technologies are making the job market dynamic, causing desirable skills to evolve continuously. It is therefore important to understand the transitions in the job market to proactively identify skill sets required.
Case description
A novel data-driven approach is developed to identify trending jobs through a case study in the oil and gas industry. The proposed approach leverages a range of data analytics tools, including Latent Semantic Indexing (LSI), Latent Dirichlet Allocation (LDA), Factor Analysis and Non-Negative Matrix Factorization (NMF), to study changes in the market. Further, our approach is capable of identifying disparities between skills that are covered by the educational system, and the skills that are required in the job market.
Discussion and evaluation
The results of the case study show that, while the jobs most likely to be replaced are generally low-skilled, some high-skilled jobs may also be at risk. In addition, mismatches are identified between skills that are imparted by the education system and the skills required in the job market.
Conclusions
This study presents how job market and skills required evolved over time, which can help decision-makers to prepare the workforce for highly demanding jobs and skills. Our findings are in line with the concerns that automation is decreasing the demand for certain skills. On the other hand, we also identify the new skills that are required to strengthen the need for collaboration between minds and machines.