Towards Efficient Deployment of Hybrid SNNs on Neuromorphic and Edge AI Hardware

• Main Authors: Seekings, James, Chandarana, Peyton, Ardakani, Mahsa, Mohammadi, MohammadReza, Zand, Ramtin
• Format: Journal Article
• Language: English
• Published: 11-07-2024
• Subjects: Computer Science - Artificial Intelligence; Computer Science - Computer Vision and Pattern Recognition; Computer Science - Hardware Architecture; Computer Science - Learning; Computer Science - Neural and Evolutionary Computing
• DOI: 10.48550/arxiv.2407.08704

This paper explores the synergistic potential of neuromorphic and edge
computing to create a versatile machine learning (ML) system tailored for
processing data captured by dynamic vision sensors. We construct and train
hybrid models, blending spiking neural networks (SNNs) and artificial neural
networks (ANNs) using PyTorch and Lava frameworks. Our hybrid architecture
integrates an SNN for temporal feature extraction and an ANN for
classification. We delve into the challenges of deploying such hybrid
structures on hardware. Specifically, we deploy individual components on
Intel's Neuromorphic Processor Loihi (for SNN) and Jetson Nano (for ANN). We
also propose an accumulator circuit to transfer data from the spiking to the
non-spiking domain. Furthermore, we conduct comprehensive performance analyses
of hybrid SNN-ANN models on a heterogeneous system of neuromorphic and edge AI
hardware, evaluating accuracy, latency, power, and energy consumption. Our
findings demonstrate that the hybrid spiking networks surpass the baseline ANN
model across all metrics and outperform the baseline SNN model in accuracy and
latency.