Peter Cariani
Massachusetts Eye & Ear Infirmary

Ninth Annual Meeting of the New England Sequencing and Timing (NEST) Workshop,
March 6, 1999 Hunter Laboratory, Brown University

Abstract

Last year we discussed temporal coding of pitch in the auditory system by means of population interspike interval statistics and timing networks for operating on these kinds of distributed temporal representations.  I will discuss some of the wider implications that common time structure on both the sensory coding and motor control sides might have for the integration of perception and action.

When we trace a path around a percept-coordination-action-environment loop, there are many places where common time patterns recur:  in the time structure of the neural signals that are sent to activate different sets of muscles at particular times, in the efference-copy signals that are associated with these commands, in the time patterns of neural signals produced by stretch receptors, in the movement of the muscles themselves, and in the concomitant time structure of the environmental perturbations that are caused by the movements of the muscles.  In the case of speech, the coarse time structure of the acoustic production mirrors that of the muscle movements, and this coarse time structure is then also re-presented to the brain through the time-structured discharges of auditory neurons.  In the case of bodily movements, the time structure of the muscle movements is replicated in the temporal discharge patterns in the vestibular and visual systems.  There are thus, plausibly, some cross-modal invariants of temporal structure that underlie both motor productions and perceptions that may facilitate sensorimotor integration.

For example, in a rhythm continuation task, there exist neural temporal discharge patterns that correspond to the perceived rhythm.  If the rhythmic discharge pattern itself is stored in a reverberating memory as a temporal pattern, then motor systems could potentially be triggered directly by this memory trace to follow / continue the rhythm.  This also yields a possible explanation for how cross-modal interactions such as the McGurk effect can occur -- the movements of lips carry time structure that would normally be associated with particular phonetic distinctions, and the time structure of these overt movements will be replicated in the phase-locked responses of visual neurons.  Thus, there may be a basic role for common time structure in multimodal integration, learning, and memory.

To the extent that neural substrates for the representation of sensory information and for the execution of actions are temporal and iconic in character, then correlational, relational alternatives to local feature-detectors, decision-trees, and hierarchical motor programs are possible.  In essence, by means of stimulus-locked time structure, sensory systems import the temporal correlational structure of environments into neural substrates.  Invariant, recurrent time patterns in those substrates can then be amplified within recurrent timing networks to build up stable objects, such that those invariances relevant for action can be detected and utilized.

Some slides