A digital neuromorphic auditory pathway

Abstract

This thesis gives an overview of my work on the development of a digital binaural cochlear system, and its applications to a "where" pathway and a "what" pathway model. The binaural cochlear system models the basilar membrane, the outer hair cells, the inner hair cells and the spiral ganglion cells. It is stable, scalable, easy to use and can thus provide an excellent input hardware stage for more complex machine hearing tasks. The "where" pathway model uses a deep convolutional neural network to analyse correlograms from the binaural cochlear system to obtain sound source location. This approach is novel for binaural sound localisation and it shows excellent performance on experimental data in reverberant environments. The "what" pathway model uses a novel event-based unsupervised feature extraction approach to investigate the acoustic characteristics embedded in auditory spike streams and it shows best accuracy in an isolated spoken digits recognition task. In this thesis, the anatomy and physiology of the human ear are described. The key elements, including the basilar membrane 'band-pass' filters, the inner hair cell transduction and the mechanical feedback introduced by the outer hair cells are important for the creation of an electronic cochlear model. Digital models for these elements are presented, and measurement results are shown. The remaining part of the human auditory pathway consists of different types of spiking neurons, and the bulk of the signal processing in the auditory pathway is performed by these spiking neurons. Therefore, the electrophysiology and anatomy underlying the spiking behaviour essential for modelling neurons electronically are described. A digital stochastic neuron model is presented, and measurement results are shown. Combining the cochlear model and the stochastic neuron model, a fully digital, biologically inspired, binaural cochlear system is developed. With the ease of use of this digital system, we can start to model and engineer the auditory pathway for practical applications. Two examples of auditory pathway modelling based on the system are presented. The first "where pathway example uses the instantaneous activity of the binaural cochlear system's inner hair cell output to generate 2-D correlograms. The generated correlograms are then analysed using a deep convolutional neural network for regression to the angle of incidence of the sound. This system is evaluated using experimental data in reverberant environments, and the performance and comparisons with other biologically inspired sound localisation systems are shown. The second "what pathway explores an event-based unsupervised feature extraction with adaptive thresholds on spike streams generated from the binaural cochlear system. It extracts acoustic features from the spike streams, and these features are used in an isolated spoken digits recognition task. The hardware cochlear system on field programmable gate array provides an excellent input stage for the two pathway models, and its cochlear parameters are configurable to adapt to various tasks. A future direction for this work is to investigate optimal cochlear parameters for different applications. The "where pathway has shown superior performance comparing with other biologically inspired sound localisation systems. Future directions for this work include: implementing the system on hardware for real-time sound localisation; extending the 2-D localisation into 3-D by using multiple cochlea pairs; investigating sound segregation and tracking algorithms based on the model for more practical applications. The "what pathway has shown better performance than other event-based approaches in isolated spoken digits recognition. A future direction for this work is to investigate optimal configurations of the unsupervised feature extraction approach for event-based auditory signal applications.

Date of Award	2019
Original language	English