Learning from Sparse and Delayed Rewards with a Multilayer Spiking Neural Network

Biological brains still far exceed artificial intelligence systems, both in terms of control capabilities and power consumption. Spiking neural networks (SNNs) are a promising model, inspired by neuroscience and functionally closer to the way neurons process information. While recent advancements in neuromorphic hardware allow energy efficient synthesis of spiking networks, the training of such networks remains an open problem. In this work we focus on reinforcement learning with sparse and delayed rewards. The proposed architecture has four distinct layers and addresses the limitation of previous models in terms of scalability with input dimensions. Our SNN is evaluated on classical reinforcement learning and control tasks and is compared to two common RL algorithms: Q-learning and deep Q-network (DQN). Experiments demonstrate that the proposed network outperforms Q-learning on a task with six-dimensional observation space and compares favorably to the evaluated DQN configurations in terms of stability and memory requirements.