What are the different theories about how the brain processes speech?

Question

When we heard speech the brain needs to go through a series of steps to go from the received sounds to a mainly conceptual meaning that follows from the words involved. This involves solving a series of problems along the way like moving from sounds into phonemes, then to words and to sentences, to a possibly holistic meaning of the whole speech. How does the brain solves this problem?

I was told there are many theories, but I don't have any names, except what some people call "the oscillatory theory of speech perception" of which I could not find much literature. Do anyone know books and papers that cover this topic, presenting different theories of speech perception, including this "oscillatory" theory?

I will greatly appreciate any linked books and papers.

Answer 1

This question is quite broad and since it seems to be basically a reference request, I will suffice by giving some prominent theories of speech perception with sources and references.

Speech perception theories are grouped into two:

1. Passive (or non-mediated) theories. These theories mostly focus on finding the identity of certain constant perceptual cues and on the ways the auditory system might extract them from the acoustic signal. These theories are basically filtering theories, and do not involve the mediation of higher cognitive processes in the extraction of these cues. It is assumed that acoustic cues are extracted in the periphery of the auditory system (source: Mannell).
2. Active (or mediated) theories. These theories suggest that there is no direct relationship between the acoustic signal and the perceived phoneme, but that higher level processes are involved instead that compare input patterns with internal patterns (source: Mannell).

Passive theories

• The distinctive feature theory defines a set of acoustic and articulatory features that is used to generate a binary code system of yes/no decisions to allow the identification of speech sounds at a phonemic level (source: Mannell).
• The acoustic theory is also based on a set of distinctive features that are largely production based. The distinctive features are assumed to be encoded directly, and passively in the periphery of the auditory system (source: Mannell).

Active theories

• The motor theory (or analysis-by-synthesis) assumes that perception and production are closely linked and that motor commands in the brain that control movements of the muscles used to speak, also help us to perceive speech. It claims that humans are equipped with an innately present speech processing 'module' (sources: UC Calgary, Mannell, Libermann & Mattingly (1985)).
• The trace theory assumes that there are 3 connected speech perception levels - features, phonemes and words. Facilitation of these levels is assumed to occur through connections between the levels (source: UC Calgary).

<sub>Reference
- Libermann & Mattingly Cognition (1985); 21: l-36</sub>

Answer 2

Factors to be considered in speech processing:

• Serial versus parallel processing (ie, whether processes are carried out sequentially or processes occur simultaneously).

• Ascending vs. descending processes (representations about basic or fundamental characteristics or parameters of the (Vs, representations of related characteristics).

• Automatic vs. controlled processes (not only do they differ in conscious control or speed, but other factors such as learning, flexibility, interference, etc.) need to be taken into account.

3 major problems that should explain theories of speech perception:

• Co-articulatory speech demands: When pronouncing a sequence of phonemes, each phoneme is not articulated separately, but the articulatory organs adjust their position to produce the anterior and posterior phoneme, that is, the phoneme production / n / depends on the other phonemes accompany it. The concept of co-articulatory speech demand is the source of segmentation problems, of absence of invariance; of the indissociable presence in the speech of acoustic characteristics as fundamental frequency that establishes the tone of vos and the intensity of the speech, and that the perception of the speech remains intact even when there is a loss of acoustic information by effect of the noise.

• Absence of invariance: Lack of correspondence between acoustic signal fragments and phonemes. The speech segments are continuous, influenced by the acoustic context. They lack invariant properties, yet the listeners capture the perceptual constants and identify the sounds. That is, the physical signal does not have the same characteristics for the same phonemes, depending on the context the signals are different, although the listener identifies the same phoneme.

• Absence of segmentation: The speech signal is continuous, while the speech sounds are perceived as discontinuous, for example, if in a syllable we separate the fragment of the consonant spectrum from the vocal spectrum and present the subject only the part of the Consonant, he is able to guess the vowel.

These problems lead to the need to recognize different levels of processing:

• Auditory level (represents the signal according to its attributes of frequency, intensity and time).

• Phonetic level (recognition of sounds by a combination of acoustic signals, such as formant transitions).

• Phonological level (the phonetic segment becomes a phoneme applying phonological rules to the sequence of sounds). These three levels can be interpreted as successive discriminations applied to the signal.

Modularity: In relation to a principle of functional specialization is studied if the processing of the language constitutes a modular system. See characteristics of the modular systems exposed by Fodor in 1983.

https://en.wikipedia.org/wiki/Modularity_of_mind

We also study the relations of the processes of the perception of the speech with the rest of processes carried out in the language.

Theory of categorical perception (Liberman, Harris, Hoffman and Griffith, 1957) It indicates that responses to verbal stimuli occur in absolute terms even when the stimuli themselves can vary continuously, and indeed vary, in some gradual way.

While this theory is not one of the most supportive of today, categorical perception has developed a lot and can be considered a research paradigm. A good article on categorical perception:

https://cocosci.berkeley.edu/tom/papers/magnet.pdf

Speech perception motor theory (Liberman et al., 1967) speech perception is produced by reference to production, that is, that listeners use their implicit articulatory knowledge, knowledge about how sounds are produced.

Interactive Activation Model (McClelland and Rumelhart, 1981) Based on previous efforts (Adams, 1979; Johnston and McClelland, 1980; Morton, 1969). Model mainly focused on word, applied to written or spoken. It is based on the fact that the processing takes place simultaneously and with interaction of the three levels: traits, letters and words. Inhibitory, inhibitory and excitatory and inhibitory levels. It is the beginning of interactive and connectionist models.

In general the connectionist models accept influence of the syntactic and semantic context.

Source: Carroll, David, W. (1999). Psychology of Language. Pacific Grove: Brooks/Cole Publishing.

There are many theories and studies on the perception of speech, it is a very developed and complex field, I hope to reflect basic questions to facilitate a field of study for any theory. I believe that the book on which I base provides a great basis for studying later any theory, study, etc.