Spiking Autoencoders With Temporal Coding

• Published in: Frontiers in neuroscience Vol. 15; p. 712667
• Main Authors: Comşa, Iulia-Maria, Versari, Luca, Fischbacher, Thomas, Alakuijala, Jyrki
• Format: Journal Article
• Language: English
• Published: Switzerland Frontiers Research Foundation 13-08-2021; Frontiers Media S.A
• Subjects: autoencoders; backpropagation; Datasets; Embedding; Firing pattern; Human performance; Information processing; inhibition; Latency; latency coding; Neural coding; Neural networks; Neurons; Neuroscience; Neurosciences; Noise; spiking networks; temporal coding; Temporal variations; autoencoders; spiking networks; temporal coding; backpropagation; inhibition; latency coding; biologically-inspired artificial intelligence
• ISSN: 1662-4548; 1662-453X; 1662-453X
• DOI: 10.3389/fnins.2021.712667

Spiking neural networks with temporal coding schemes process information based on the relative timing of neuronal spikes. In supervised learning tasks, temporal coding allows learning through backpropagation with exact derivatives, and achieves accuracies on par with conventional artificial neural networks. Here we introduce spiking autoencoders with temporal coding and pulses, trained using backpropagation to store and reconstruct images with high fidelity from compact representations. We show that spiking autoencoders with a single layer are able to effectively represent and reconstruct images from the neuromorphically-encoded MNIST and FMNIST datasets. We explore the effect of different spike time target latencies, data noise levels and embedding sizes, as well as the classification performance from the embeddings. The spiking autoencoders achieve results similar to or better than conventional non-spiking autoencoders. We find that inhibition is essential in the functioning of the spiking autoencoders, particularly when the input needs to be memorised for a longer time before the expected output spike times. To reconstruct images with a high target latency, the network learns to accumulate negative evidence and to use the pulses as excitatory triggers for producing the output spikes at the required times. Our results highlight the potential of spiking autoencoders as building blocks for more complex biologically-inspired architectures. We also provide open-source code for the model.