Non-linear shrinking of linear model errors

• Published in: Analytica chimica acta Vol. 1258; p. 341147
• Main Authors: Helin, Runar, Indahl, Ulf, Tomic, Oliver, Liland, Kristian Hovde
• Format: Journal Article
• Language: English
• Published: Netherlands Elsevier B.V 01-06-2023
• Subjects: Deep learning; Hybrid model; Interpretation; Neural network; PLSR; Residual modelling; Deep learning; Residual modelling; Hybrid model; Neural network; Interpretation; PLSR
• ISSN: 0003-2670; 1873-4324
• DOI: 10.1016/j.aca.2023.341147

Artificial neural networks (ANNs) can be a powerful tool for spectroscopic data analysis. Their ability to detect and model complex relations in the data may lead to outstanding predictive capabilities, but the predictions themselves are difficult to interpret due to the lack of understanding of the black box ANN models. ANNs and linear methods can be combined by first fitting a linear model to the data followed by a non-linear fitting of the linear model residuals using an ANN. This paper explores the use of residual modelling in high-dimensional data using modern neural network architectures.
By combining linear- and ANN modelling, we demonstrate that it is possible to achieve both good model performance while retaining interpretations from the linear part of the model. The proposed residual modelling approach is evaluated on four high-dimensional datasets, representing two regression and two classification problems. Additionally, a demonstration of possible interpretation techniques are included for all datasets. The study concludes that if the modelling problem contains sufficiently complex data (i.e., non-linearities), the residual modelling can in fact improve the performance of a linear model and achieve similar performance as pure ANN models while retaining valuable interpretations for a large proportion of the variance accounted for.
The paper presents a residual modelling scheme using modern neural network architectures. Furthermore, two novel extensions of residual modelling for classification tasks are proposed. The study is seen as a step towards explainable AI, with the aim of making data modelling using artificial neural networks more transparent.
[Display omitted]
•Residual shrinking is developed for both regression and classification problems.•The proposed strategy can improve predictions while retaining interpretability.•This contributes to explainable AI by shrinking the black box of ANNs.