NSF_BITS Project Site

EIA-BITS-0130807/0130793:

“Collaborative Research: Processing of Temporally-Coded  Auditory  Representations for Sound Separation and Localization”

Institutions: University of Rhode Island and Mass. Eye & Ear Infirmary

Investigators: Peter Cariani and Ramdas Kumaresan

Current Work-in-progress (2003)

• Auditory nerve model for temporal processing (MATLAB)

We are developing a computationally-efficient auditory nerve model that replicates basic features of temporal discharge patterns in the auditory nerve. The behavior of the model is being compared with physiological data in order to replicate response patterns that we believe are critical for population-interval representations:
    Incorporation of band-pass filters that replicate the broad tuning of passive cochlear filters
    Incorporation of rate-level adaptive gain control (rate compression) that preserves shapes of period histograms with little distortion(e.g. Rose, 1971)
    Incorporation of uncorrelated spontaneous activity

• Extension of population-interval representations to other areas of auditory perception

Poster presentation at the International Workshop on Neural Coding of Pitch, August, 2002, Delmenhorst, Germany. Download (PDF, 8 Mb).

• Evaluation of recurrent timing nets for enhancing recognition of vowels in noise and for speaker separation

• Paper for Proceedings of IJCNN 2003, July, 2003. "Recurrent timing nets for auditory scene analysis." Download (PDF, 392k)

• Paper for Proceedings of Perspectives on Speech Separation, Montreal, October 30-November 2, 2003. "Recurrent timing nets for F0-based speaker separation." Download (PDF, 1Mb)

• Auditory nerve model, m-files
anf_model.m   computes PST and autocorrelation histograms for simulated auditory nerve fibers
pitch_sieve.m    computes pattern-salience of interval patterns associated with specified periodicities
plot_neurogram.m   plots neurogram (PST plots stacked by CF), rate-place profile, and population-interval distribution

These routines are experimental and haven't been publicly released before, so please email me if you have questions regarding the algorithms or difficulties implementing the script.

Project summary (June, 2002)

Project Goals:

To use insights from neural representation and processing of sounds in the auditory system to develop computer-based signal processing strategies for separation of speech signals. To bring together experts in auditory neuroscience and signal processing to develop new temporal (e.g. zero-crossing based) representations of speech signals and new temporal processing architectures (neural timing nets).
 

Outcomes:

Our project commenced in mid-January, 2002.

Ideas

The collaboration has provided insights into commonalities and differences between neurally-based signal representations  and computations and those currently used in artificial speech-processing systems.

Tools

We are currently working on a simple auditory front-end that captures the basic temporal structure of neural responses in real auditory systems that we hope will provide a useful tool for exploring temporal processing. We are also working on mathematically based (on the theory of zeros of entire functions), but biologically plausible signal representations in the auditory system.

People

The project has enabled electrical engineering graduate students to get first-hand perspectives of the nature of neural signal processing in the auditory system.

Impact:

The project is in its initial stages, so an evaluation of impact is probably premature. We have been working on the auditory simulation front-end to develop more realistic cochlear filters and on the temporal representation of signal envelopes. We are sorting out which aspects of cochlear and subsequent neuronal processing are likely to be most important for separating speech sounds. We have also explored modified buildup rules for recurrent timing nets, and are currently testing their enhancement of synthetic vowels in noise. Thus far  we are on track.
 

Barriers & Opportunities:

What are the primary barriers/needs of research or education in your area of research? There are very few people currently working on the nature of stimulus coding in the auditory system, and few  overviews of the auditory system from a basic signal processing perspective. It is very difficult for signal processing engineers to get a good working grasp of the nature of auditory processing (broad cochlear filters, precise and robust phase-locking, lack of highly tuned detector elements) from the auditory neuroscience literature. At the same time, there are few overviews of signal processing that usefully inform auditory neurophysiologists who are trying to understand the nature of signal processing in the auditory system. This presents rich opportunities for collaboration and cross-pollenation of fields, the possibility that fundamentally new functional principles for signal processing can be discovered from reverse-engineering the auditory system.