P.A. Cariani

Eaton-Peabody Laboratory, Massachusetts Eye and Ear Infirmary

243 Charles Street, Boston, MA 02114, USA

Tel. 617-573-4243, FAX 617-720-4408, Email: peter@epl.meei.harvard.edu

www.cariani.com

Poster, 1998 ARO Midwinter Meeting

Summary
 
 

Abstract

In the auditory nerve, there is an abundance of temporal information that precisely and robustly encodes many perceptually-relevant aspects of acoustic stimuli: periodicity, spectral shape, speech modulations, rhythms, and still longer time structures. Most central models of auditory processing that utilize this timing information have assumed that a time-to-place transformation must occur in the ascending auditory pathway, such that central representations of auditory forms are based on excitation profiles of frequency- or periodicity-tuned units. In these models, auditory discrimination and recognition is performed by comparing stored excitation profiles with incoming ones.

However, if timing information can be somehow be preserved and stored centrally, then purely temporal analyses of similarity and difference can be carried out by temporal correlation operations. Three very basic neural timing architectures that utilize coincidence detectors and tapped delay lines are proposed. A feed-forward network of tapped delay lines and coincidence detectors can compute cross-correlations and convolutions on pulse trains. Periodicities common to the input pulse trains appear in the time structure of the outputs, such that the array functions as a temporal sieve. To the extent that time structure of inputs reflect those of stimuli, such arrays can compute pitch similarity irrespective of timbre, and timbral similarity independent of pitch. If coincidence elements with local inhibitory connections are added to arrays of all-pass units, pulse coincidences are vetoed, and an anti-correlation, cancellation operation is performed.

A coincidence network with many recurrent delay paths can function as a reverberating memory. Incoming time patterns are cross-correlated with previously stored patterns circulating through each delay loop. The circulating patterns set up temporal expectations that are either reinforced or attenuated depending upon subsequent inputs (cf. Patterson & Allerhand, J. Acoust. Soc. Am. 98:1890-4). Time patterns of auditory objects with different fundamentals segregate into different reverberant loops each with its own corresponding recurrence time.

[Supported by Research Grant DC03054 from the National Institute on Deafness and Other Communicative Disorders, National Institutes of Health.]

Population-interval representations

Population interval representations are population-wide distributions of all-order interspike intervals (i.e. intervals between both consecutive and nonconsecutive spikes in the same spike train).

Population-interval distributions at the level of the auditory nerve and cochlear nucleus constitute autocorrelation-like representations capable of subserving the perception of complex auditory forms such as pitch, timbre, and rhythm. What kinds of neural computations could effectively use such global timing information?

Autocorrelation-like representations. We observed empirically that patterns of major and minor peaks in population-representations resemble those of their respective stimulus autocorrelation functions. This is a general consequence of phase-locking of discharges. To the extent that a receptor system produces neural discharges whose timings are highly correlated with stimulus time structure, distributions of all-order interspike intervals will resemble the stimulus autocorrelation function. In the auditory system, population-interval distributions can provide very general, precise, and robust vehicles for encoding stimulus periodicities below 5 kHz.

Representations of pitch. Features of population-interval distributions estimated from responses of 50-100 single auditory nerve fibers of Dial-anesthetized cats closely parallel human pitch perception (Cariani & Delgutte, J. Neurophysiol. 76(3):1698-1734, 1996). With very few exceptions, the most frequent all-order interval in the auditory nerve at any given time corresponds to the pitch that is heard. Periodic stimuli produce population-interval distributions whose major interval peaks are associated with the fundamental period and its multiples. Ratios of pitch-related intervals to all others qualitatively correspond to pitch salience. Many pitch-related phenomena are readily explained in terms of population-interval distributions: pitch of the missing fundamental, pitch equivalence, phase & level invariances, nonspectral pitch, dominance region, and inharmonic pitch shifts.

Representations of the timbres of stationary sounds. The timbre of stationary harmonic sounds is associated with shapes of spectral envelopes and patterns of minor peaks in autocorrelation functions. All stimulus components contribute their time structure to population interval distributions to the degree that they produce stimulus-locked discharges,. While patterns of major interval peaks in population interval distributions are associated with pitch, patterns of minor peaks are associated with timbre. In the auditory nerve, different vowels produce population-interval distributions with patterns of short intervals that are characteristic of formant structure.
 
 

Temporal processing in neural networks

Purely connectionist networks. Neural networks have always embodied assumptions about neural codes. Traditionally, neural networks have assumed that sensory information is encoded via "rate-place" codes (which neurons fire how much). Connectionist nets analyze spatial patterns of excitation in their inputs to produce spatial patterns of excitation in their outputs (place-to-place mappings).

Time-delay neural networks. In the auditory system the importance of time structure for encoding the form and location of sounds has long been appreciated. Consequently the first neural networks that were proposed for auditory computations (e.g. Jeffress, 1948; Licklider, 1951) were time-delay neural networks (TDNN’s). These networks used arrays of tapped-delay lines and coincidence counters to implement cross- and auto-correlation operations on their temporally-coded inputs. Other time-delay networks employ arrays of oscillators. Time-delay networks are generally used to convert timing patterns into spatial excitation patterns (time-to-place mappings), such that subsequent central processing is realized using purely connectionist networks. In both connectionist and time-delay approaches, auditory matching tasks are performed by comparing stored spatial excitation profiles with incoming ones. More recently time-to-place transformations that utilize the modulation-tuning properties of central auditory neurons rather than coincidence detectors have been proposed (Langner & Shreiner).

Timing networks. Neural networks can be envisioned that operate on time structure in their inputs to produce interpretable temporal patterns in their outputs (time-to-time mappings). Very few models of this type currently exist. If neural timing information can be somehow be preserved and stored centrally, then purely temporal analyses of similarity and difference can be carried out by relatively simple temporal correlation operations. Such models are related to computational theories of cortical structures that utilize cortical pyramidal neurons as temporal coincidence units (Braitenberg, Abeles) rather than rate integrators. Recurrent structure is ubiquitous in the the brain, and "nets with loops" have been analyzed as feedback shift registers (McCulloch). The bottom-up:top-down pattern matching aspects are in many ways similar to adaptive resonance architectures (Grossberg). The recurrent timing aspects are most closely related to the "neural loop model" of Thatcher & John (1977).Thus far the functional properties of medium- and large-scale (regular and randomly-connected) timing nets have yet to be explored.
 
 

Temporal memory traces in the brain

References

Abeles, Moshe. 1982. Role of the cortical neuron: integrator or coincidence detector. Israel Journal of Medical Sciences 18 : 83-92.

Abeles, M., H. Bergman, E. Margalit, and E. Vaadia. 1993. Spatiotemporal firing patterns in the frontal cortex of behaving monkeys. J. Neurophysiol. 70 (4) : 1629-1638.

Bharucha, Jamshed J. 1991. Pitch, harmony and neural nets: a psychological perspective. In Connectionism and Music. Edited by P. Todd and G. Loy. 84-99. Cambridge: MIT Press.

Braitenberg, V. 1961. Functional interpretation of cerebellar histology. Nature 190 : 539-540.

Bregman, Albert S. 1990. Auditory Scene Analysis: The Perceptual Organization of Sound. Cambridge, MA: MIT Press.

Cariani, P. 1995. As if time really mattered: temporal strategies for neural coding of sensory information. Communication and Cognition - Artificial Intelligence (CC-AI) 12 (1-2) : 161-229. Preprinted in Origins: Brain and Self-Organization. Edited by K. Pribram. 208-252. Hillsdale: LEA

Cariani, P. 1997a. Neural representation of pitch through autocorrelation. Proceedings, Audio Engineering Society Meeting (AES), New York, September, 1997, Preprint #4583 (L-3).

Cariani, P. 1997b. Temporal coding of sensory information. In Computational Neuroscience: Trends in Research, 1997. Edited by J. M. Bower. 591-598. New York: Plenum.

Cariani, Peter A., and Bertrand Delgutte. 1996. Neural correlates of the pitch of complex tones. I. Pitch and pitch salience. II. Pitch shift, pitch ambiguity, phase-invariance, pitch circularity, and the dominance region for pitch. J. Neurophysiology 76 (3) : 1698-1734.

Cherry, Colin. 1961. Two ears ­ but one world. In Sensory Communication. Edited by W. A. Rosenblith. 99-117. New York: MIT Press.

Cohen, Michael A., Stephen Grossberg, and Lonce L. Wyse. 1994. A spectral network model of pitch perception. J. Acoust. Soc. Am. 98 (2) : 862-879.

Grossberg, Stephen. 1988. The Adaptive Brain, Vols I. and II. New York: Elsevier.

Jeffress, L. A. 1948. A place theory of sound localization. J. Comp. Physiol. Psychol. 41 : 35-39.

John, E. R. 1967. Mechanisms of Memory. New York: Wiley.

John, E. R. 1972. Switchboard vs. statistical theories of learning and memory. Science 177 : 850-864.

Jones, Mari Riess. 1976. Time, our lost dimension: toward a new theory of perception, attention, and memory. Psych Revi 83 (5) : 323-255.

Langner, Gerald. 1992. Periodicity coding in the auditory system. Hearing Research 60 : 115-142.

Langner, G., C.E. Schreiner & U.W. Biebel. 1998. Functional implications of frequency and periodicity coding in the auditory midbrain. In Psyhophysical and Physiological Advances in Hearing. Edited by A. R. Palmer, A. Rees, A. Q. Summerfield & R. Meddis. 243-249. London: Whurr Publishers.

Large, E. W. 1994. Dynamic representation of musical structure. Ph.D., The Ohio State University.

Lestienne, R., and Henry C. Tuckwell. 1997. The significance of precisely replicating patterns in mammalian CNS spike trains. Neuroscience April 1997

Licklider, J.C.R. 1951. A duplex theory of pitch perception. Experientia VII (4) : 128-134.

Licklider, J.C.R. 1959. Three auditory theories. In Psychology: A Study of a Science. Study I. Conceptual and Systematic. Ed. S. Koch. .McGraw-Hill.

Longuet-Higgins, H.C. 1987. Mental Processes: Studies in Cognitive Science. Cambridge, Mass.: The MIT Press.

Longuet-Higgins, H.C. 1989. A mechanism for the storage of temporal correlations. In The Computing Neuron. Edited by R. Durbin, C. Miall and G. Mitchison. 99-104. Wokingham, England: Addison-Wesley.

MacKay, D.M. 1962. Self-organization in the time domain. In Self-Organizing Systems 1962. Edited by M. C. Yovitts, G. T. Jacobi and G. D. Goldstein. 37-48. Washington, D.C.: Spartan Books.

McCulloch, W.S. 1969. Of digital oscillators. In Information Processing in the Nervous System. Edited by K.N. Leibovic. 293-296. New York: Springer .

Miller, Ralph R., and Robert C. Barnet. 1993. The role of time in elementary associations. Current Directions in Psychological Science 2 (4) : 106-111.

Morrell, F. 1967. Electrical signs of sensory coding. In The Neurosciences: A Study Program. Edited by Quarton, Melnechuck &.Schmitt. 452-469. New York: Rockefeller University Press.

Patterson, Roy D., Mike H. Allerhand, and Christian Giguere. 1995. Time-domain modeling of peripheral auditory processing: A modular architecture and a software platform. J. Acoust. Soc. Am. 98 (4) : 1890-1894.

Pratt, Gill. 1990. Pulse Computation. Ph.D., M.I.T.

Reitboeck, H.J., M. Pabst, and R. Eckhorn. 1988. Texture description in the time domain. In Computer Simulation in Brain Science. Edited by R. M. J. Cotterill. Cambridge, England: Cambridge University Press.

Schreiner, Christof E., and Gerald Langner. 1988. Coding of temporal patterns in the central auditory system. In Auditory Function: Neurobiological Bases of Hearing. Edited by G. M. Edelman, W. E. Gall and W. M. Cowan. 337-362. Wiley: New York.

Tank, D. W., and J.J. Hopfield. 1987. Neural computation by concentrating information in time. Proc. Natl. Acad. Sci. USA 84 : 1896-1900.

Thatcher, Robert W., and E. Roy John. 1977. Functional Neuroscience, Vol. I. Foundations of Cognitive Processes. Hillsdale, NJ: Lawrence Erlbaum.