Learning Properties of Holomorphic Neural Networks of Dual Variables

• Published in: ICASSP 2023 - 2023 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP) pp. 1 - 5
• Main Authors: Kozlov, Dmitry, Bakulin, Mikhail, Pavlov, Stanislav, Zuev, Aleksandr, Krylova, Mariya, Kharchikov, Igor
• Format: Conference Proceeding
• Language: English
• Published: IEEE 04-06-2023
• Subjects: Algebra; Artificial neural networks; deep learning; dual modulus; dual number; dual-valued batch normalization; dual-valued neural networks; Gravitational waves; Hardware; holomorphic dual-valued neural networks; holomorphic dual-valued operators; Mathematical models; Measurement; Signal processing
• ISSN: 2379-190X
• DOI: 10.1109/ICASSP49357.2023.10095457

Artificial neural networks have become an inseparable element of human life. Researches do not stop at the current progress and try to improve neural networks and expand fields of applications. The most widespread way to make models better consists in generalization of existing methods and approaches. In this paper, we make a step in an unusual direction: we propose to use neural networks based on dual numbers. We develop a special subclass of dual-valued operators, which satisfy the equivalent of the Cauchy-Riemann equations for the dual domain. We also propose a new type of preprocessing and batch normalization, relying on peculiarities of dual numbers. We test deep holomorphic dual-valued models on music transcription and gravitational wave detection tasks and show that our holomorphic dual-valued networks achieve better inference time compared to the dual-valued models and are better than their real-valued counter-parts in sense of metrics.