Using deep learning to value free-form text data for predictive maintenance

• Published in: International journal of production research Vol. 60; no. 14; pp. 4548 - 4575
• Main Authors: Usuga-Cadavid, Juan Pablo, Lamouri, Samir, Grabot, Bernard, Fortin, Arnaud
• Format: Journal Article
• Language: English
• Published: London Taylor & Francis 18-07-2022; Taylor & Francis LLC
• Subjects: Algorithms; class imbalance; Deep learning; Engineering Sciences; Free form; Industry 4.0; interpretability; Machine learning; maintenance; Natural language processing; Oversampling; Predictive maintenance; Unstructured data; deep learning; interpretability; class imbalance; Industry 4.0; natural language processing; maintenance
• ISSN: 0020-7543; 1366-588X
• DOI: 10.1080/00207543.2021.1951868

Past maintenance logs may encapsulate meaningful data for predicting the duration of machine breakdowns, the potential causes of a problem, or the necessity to stop production to perform repair activities. These insights may be accessed using machine learning (ML). However, maintenance logs tend to have imbalanced distributions and rely on noisy unstructured text data provided by operators. Additionally, the limited interpretability of ML models results in human reluctance when accepting model predictions. Hence, this study explored the use of two recent deep learning models (CamemBERT and FlauBERT) for natural language processing (NLP) to harness unstructured data from maintenance logs. The class imbalance effect was mitigated using data-level and algorithm-level approaches. To improve interpretability, a technique called LIME was employed to interpret single predictions and to propose a method for insight extraction from several maintenance reports. Results suggest three key points: CamemBERT and FlauBERT can achieve excellent results with minimum text pre-processing and hyperparameter tuning. Second, random oversampling (ROS) generally mitigates the effect of class imbalance. However, ROS was observed to be unnecessary when performing pertinent data pre-processing. Finally, at the maintenance level, the proposed insight extraction method can provide valuable information from a set of poorly structured maintenance reports.