Reviewed by Donna McCabe for The Computer Music Journal
Worcester, Massachusetts, USA
Languages of the Mind is a product of the past five years of research by Ray Jackendoff on the nature of mental representation in a variety of cognitive domains. Jackendoff, a Professor of Linguistics at Brandeis University, has tackled difficult issues in theories of mind and cognitive processing for the past two decades. Languages of the Mind extends his formal theory of semantic representation and the overall levels of conceptual structure earlier presented in Semantics and Cognition (1983) and Consciousness and the Computational Mind (1987). Perhaps best know in the music field for his research with Fred Lerdahl on tonal music in A Generative Theory of Tonal Music (1983), Jackendoff extends his investigation of the mind in terms of formal symbolic descriptions of information structures. He also explores the modularity of the mind and suggests multiple central processing divisions.
Chapter 1 lays out the general theme for the book. Jackendoff discusses the relation between connectionist theories and formal symbolic theories, and the reasons that he does not find them mutually exclusive approaches. Frequently applying theories of philosophers Fodor and Searle, the chapter Languages of the Mind asks precisely what kinds of information are computations in the brain carried out with? What categories, distinctions, and relations must be encoded in mental information structures in order to account for behavior, experience and intelligence? Studies have shown that sheer size of the brain (or computer) is not enough to find the form of the information the brain processes, stores, and retrieves. Therefore, Jackendoff searches for the medium (or computer program) which has until now simply been referred to as a nondescript liquid which passes between so called containers of long and short term memory. Jackendoff deals with both input and output, as well as goupings and interaction of information in a central processor. He is particularly interested with the different forms of input (the senses, ie. touch, vision, auditory) and output and their interaction which provides a basis for his theory of a central processor within the mind. Based on this interaction, the central processor is therefor able to translate or transform information from the input forms into the central format (the so called malleable liquid), and later to the output form. This enables us to use our understanding of the world as a basis for action. His use of language and syntax are interesting both for their research and documentation value, as well as for purely musical analysis of language and syntax. Viewing the input information from language, vision, and music, Jackendoff uses a connectionist theory to argue for universal grammars.
Chapter 2 summarizes the basic organization of conceptual structure which he explains in detail in Semantic Structures (1990). Entitled What Is a Concept, That a Person May Grasp It,? Jackendoff attempts to better understand human nature. He characterizes the mental resources that make possible the articulation of humans' knowledge and experience of the world. Conceived as an extension of Chomsky's goals, Jackendoff believes a concept to be an entity within one's head, a private entity - a product of language, gesture, drawing, or some other imperfect means of communication. His Conceptual Semantics sketch a number of major elements of the internal structure of concepts, showing how his approach accounts for various basic phenomena in the semantics of natural language, and how it meets various well-know (within the field) objections to theories of lexical decomposition.
In Chapter 3, Word Meanings and What It Takes to Learn Them: Reflections on the Piaget-Chomsky Debate, Jackendoff discusses the similarities and differences of each approach to their theories of what it takes to learn. Chomsky's argument focuses almost exclusively on complex details of the learning of syntax, whereas Piaget's ground for argument is conceptual learning. Jackendoff basis his argument on his Conceptual Well-Formedness Rules, that is rules that characterize the space of possible conceptual states - the resources available in the brain for forming concepts. The exploration of word meanings, how they are formed and linked with concepts is extremely interesting and well stated in this chapter. Jackendoff investigates what can be learned with what is innate to our being; what is universal and what it culturally modulated.
The general theory that is emerging from contemporary cognitive science is that the mind can be factored into a considerable number of modules, each of which specializes in processing a particular form of information and therefor caries out particular kinds of perceptual or cognitive tasks. Chapter 4 is concerned with the possibility that the distinct modules of the mind include a module specifically devoted to social cognition, and that such a faculty can be studied congruously to those used for study of the language faculty. Again, Jackendoff deals in terms of input/output vs. central modules. There is a very brief discussion of a possible musical module where Jackendoff decides that music along with language, share a culture-dependent component. His social cognition module is a central module (a translator) whose task is to develop an integrated picture of the self within society. A linguistic battle ensues regarding notions like dominance, authority, privilege, right, obligation, value, and ownership which clearly demonstrate his theory that a social cognition module exists.
Chapter 5 speculates a theory of mental representation that Jackendoff applies to issues in psychodynamics (the contemporary form of Freudian personality theory). Entitled Unconscious Information in Language and Psychodynamics, Jackendoff discusses the ability of the mind to understand and utter an indefinitely large number of sentences that have never been heard or spoken. It is this seemingly unbounded mental capacity of the mind which generates his theory that contends that a person is governed by complex principles of which they are not, and cannot possibly be consciously aware of. Therefore, if we are unaware of these principles we cannot pass them down to our children by explicit instruction, yet they are carried on somehow. The examples used show parallels between language and psychodynamic phenomena. This leads Jackendoff to encourage further tandem research into the unconscious because it has been demonstrated by linguists that language exposes the rich structure possible in the unconscious and the precision with which it can be studied. It has been a major hypothesis of psychodynamic theory that unconscious motives and desires can be made conscious, and that the process of making them fully conscious can enable a person to come to interact differently with the world. Jackendoff pushes for psychodynamic phenomena to be studied in both their content and formal structure to conceive of a formal system underlying ordinary reasoning.
Chapter 6, Spatial Language and Spatial Cognition was co-authored by Barbara Landau Together Jackendoff and Landau explore important details of the way in which conceptual structure communicates with another of the central levels of the mind, spatial representation. Their research here builds on the preliminary suggestions about such a connection made initially in Consciousness and the Computational Mind. This chapter leads toward a clearer understanding of how it is possible for humans to talk about what they see, and furthermore why it is that we talk about what we see in the way we do. How is it that language encodes objects and spatial relationships? Since spatial information can be derived from vision, from auditory localization, and from the haptic (touch) module, it demands a format that is not modality-specific. Spatial representation involves translating modality-specific information into a common format via a central processor. Their first argument relates to viewing objects by a complex combinatorial system using the Marr and Biederman framework. Once Landau and Jackendoff define what an object is, they then begin to address where the object is in space. They tackle issues of trajectory, visibility and occlusion, distribution, axis, distance and direction. They conclude that the relevant words that describe what an object is are much more complex than those needed to describe where an object is through an exhaustive list of prepositions in English. This indicates that the translation from spatial representation to linguistic representation seems to involve a certain amount of filtering, such that significant aspects of spatial relations go unexpressed (ie. there are less significant digits involved) without an apparent loss of data. This chapter requires less specific research into a particular subject than most others.
The chapter dealing directly with musical parsing and musical affect comes late in the book. Jackendoff uses his previous research with Lerdahl on musical cognition to investigate the possibilities for a musical parser based on a musical grammar of strictly tonal music. What is going on is a listener's mind during the real-time processing of music, such that it is possible to account for the listener's understanding of the music? Based on the previous Generative Theory of Tonal Music, Jackendoff continues his search for a musical parser. GTTM states that the perception of music parallels the perception of language and Languages of the Mind contends that as there is successful linguistic parsing, a music parser is possible (for tonal music). With the use of a Schenker time-span reduction from Bach's St. Matthew Passion he describes musical parsing in relation to musical affect. Jackendoff proposes several different types of parsers which he claims may be similar to what the uses to determine such things as key and meter. (It hardly seems to me that what is of value to musical affect is necessarily either key or meter, and his decision to continue to study strictly tonal music was somewhat disappointing.) There is also a discussion of memory and the expectation theory of affect where L.B. Meyer is cited.
The final chapter explains the two sides of Cognitive Science; the philosophical and the psychological. The philosophical version asks what is the relationship of the mind to the world, such that we can have knowledge of reality, beliefs and desires about things in the world, and that our sentences can be true or false (ideas which were explored in Chapter 4). The psychological version asks how does the brain function as a physical device, such that the world seems to us the way it does, and that we can behave effectively in the world? Again, Jackendoff discusses mental representations through vision, language, music, social cognition, and our physical selves. This acts as a more concentrated comparison of the formal and philosophical tradition of symbolic theories of mind first entertained in Chapters 1 and 2.
In Languages of the Mind, Jackendoff cites over 200 articles and books as reference. This list includes 5 books, and 8 articles of his own. It is this list of references and the end notes that make Languages of the Mind a valuable resource if you intend to delve into any one of the many areas that Jackendoff covers. The book was originally written as a series of papers which were to stand alone, and in an effort to make the book as a whole cohere there is frequent cross referencing among chapters which sometimes becomes confusing. With so many references cited, including many of Jackendoff's own books, it is clear that Languages of the Mind is another slice of time in his ongoing research. One would need to have Jackendoff's own background to fully grasp all the theories, but his generous list of references makes this a wonderful springboard into any one of the many areas dealt with.
On the lighter side, there was only one reference to a picture being worth a thousand words!