Complexity reduction over Bi-RNN-based nonlinearity mitigation in dual-pol fiber-optic communications via a CRNN-based approach

• Published in: Optical fiber technology Vol. 74; no. 103072; pp. 103072 - 12
• Main Authors: Shahkarami, Abtin, Yousefi, Mansoor, Jaouën, Yves
• Format: Journal Article
• Language: English
• Published: Elsevier Inc 01-12-2022; Elsevier
• Subjects: Complexity reduction; Convolutional recurrent neural networks; Engineering Sciences; Equalization; Kerr nonlinearity; Optical fiber communications; Optics; Photonic; Equalization; Complexity reduction; Optical fiber communications; Convolutional recurrent neural networks; Kerr nonlinearity
• ISSN: 1068-5200; 1095-9912
• DOI: 10.1016/j.yofte.2022.103072

Bidirectional recurrent neural networks (bi-RNNs), in particular bidirectional long short term memory (bi-LSTM), bidirectional gated recurrent unit, and convolutional bi-LSTM models, have recently attracted attention for nonlinearity mitigation in fiber-optic communication. The recently adopted approaches based on these models, however, incur a high computational complexity which may impede their real-time functioning. In this paper, by addressing the sources of complexity in these methods, we propose a more efficient network architecture, where a convolutional neural network encoder and a unidirectional many-to-one vanilla RNN operate in tandem, each best capturing one set of channel impairments while compensating for the shortcomings of the other. We deploy this model in two different receiver configurations. In one, the neural network is placed after a linear equalization chain and is merely responsible for nonlinearity mitigation; in the other, the neural network is directly placed after the chromatic dispersion compensation and is responsible for joint nonlinearity and polarization mode dispersion compensation. For a 16-QAM 64 GBd dual-polarization optical transmission over 14×80km standard single-mode fiber, we demonstrate that the proposed hybrid model achieves the bit error probability of the state-of-the-art bi-RNN-based methods with greater than 50% lower complexity, in both receiver configurations.
•Fiber nonlinearities can be mitigated in optical communication using low-complexity neural networks.•Neural networks outperform digital back-propagation for equalization in dual-polarization long haul optical fiber transmission.•Hybrid convolutional recurrent neural networks outperform CNN and pure RNN models for mitigation of PMD and Kerr nonlinearity.