Stochastic electronics for neuromorphic systems

Abstract

Advances in integrated circuit (IC) fabrication technology have reduced feature sizes to the order of nanometres, but have created several problems in IC design, such as lower noise immunity, increased process mismatch, and interconnect bottlenecks. Further, the failure of a few transistors may result in the failure of the entire chip, rendering it unusable. Similar to these problems of transistor failure and device mismatch in ICs, the brain is faced with the problems of heterogeneity of neuronal responses to stimuli and neuronal cell death. The biological nervous system functions well despite these problems, and this motivates us to apply its working principles in IC implementation. In this thesis, we draw inspiration from the brain and discuss how 'stochastic facilitation' can be used to perform useful and precise computation. We explore non-deterministic methodologies for computation in hardware and introduce the concept of stochastic electronics; a new way to design circuits and increase performance in noisy and mismatched fabrication environments. We illustrate this approach by presenting systems for both analogue and digital IC design. For the analogue system, we propose a generic and trainable architecture, which uses device mismatch and nonlinearities explicitly. In this way, the reduced device matching in newer technologies becomes an advantage, rather than something that needs to be engineered out of the design. We have developed a novel neuromorphic system called a Trainable Analogue Block (TAB), which uses device mismatch as a means for random projections of the input to a higher dimensional space. The TAB framework is inspired by the principles of neural population coding operating in the biological nervous system. Three neuronal layers, namely input, hidden, and output, constitute the TAB framework, with the number of hidden layer neurons far exceeding the number of input layer neurons. For the digital system, we use a stochastic computation (SC) framework to build massively parallel and low precision circuits to solve complex Bayesian inference problems. An advantage of the SC implementation is that it is robust to certain types of noise, which may become an issue in IC technology with feature sizes in the order of tens of nanometres due to their low noise margin, the effect of high-energy cosmic rays, and the low supply voltage. We present the implementation of two types of Bayesian inference problems to demonstrate the potential of building probabilistic algorithms in hardware. The first implementation, referred to as the BEAST (Bayesian Estimation and Stochastic Tracker), demonstrates a simple problem where an observer uses an underlying Hidden Markov Model to track a target in one dimension. In this implementation, sensors make noisy observations of the target position at discrete time steps. The tracker learns the transition model for target movement, and the observation model for the noisy sensors, and uses these to estimate the target position by solving the Bayesian recursive equation online. We show the tracking performance of the system and demonstrate how it can learn the observation model, the transition model, and the external distractor (noise) probability interfering with the observations. In the second implementation, we show how inference can be performed in a Directed Acyclic Graph (DAG) using stochastic circuits, and this implementation is referred to as the Bayesian INference in DAG (BIND). Our work describes canonical neural circuits, which are the basic building blocks of our models, and shows how these neural circuits can be easily implemented using digital logic gates. An advantage of our framework is that the flipping of random individual bits would not affect the system performance because information is encoded in a bit stream. Our work presents a novel approach for implementing probabilistic networks using simple logic gates, with the ability to perform the computation in real time. Overall, our work demonstrates the potential of building brain-inspired electronic systems by utilising intrinsic non-idealities of modern IC fabrication processes.

Date of Award	2016
Original language	English