Funded Grants

Coordinating mental, motor, and perceptual constraints in language

Scientists have long observed that human languages are highly organized systems. This is true of every spoken language that has been described and in multiple dimensions: the combination of sounds to create words (phonology), the combination of morphemes – single meaningful units – to create complex words and phrases (morphology), and the combination of words into sentences (syntax).

Linguists have taken these behavioral properties of human language as evidence for a set of abstract cognitive representations and principles present in the mind of every speaker. Representations include an inventory of phonemes (contrastive sounds), a set of lexical items, while principles include those governing the possible and impossible combinations of sounds, and the combination of words into sentences.

Languages differ from each other because these representations can differ: Each language has its own inventory of phonemes, its own lexical items, and its own organizing principles. Restrictions found in the phonology include: phoneme inventories tend to be spread out in articulatory and acoustic space, single lexical items tend to have upper and lower bounds on their size, and certain sequences of sounds are permissible while others are not.

Our interest lies in the representation of the phonological component: the sounds and the restrictions on their combination. Phonology is unique among analytical levels of language in that restrictions on sound combination are local: languages obey restrictions against neighbouring segments, and not against the co-occurrence of segments with lots of intervening material. For example, many languages forbid a sequence like np (preferring mp), but no language is known to have a rule forbidding n and p to occur in the same word when many other sounds separate them. This property of phonology is known as Adjacency.

The focus of our study is harmony, a phenomenon in which sound restrictions apparently hold over segments that are not quite adjacent. In a harmonic system, certain sounds within a word must share a certain feature. For example, some languages (like Hungarian) require all vowels within words to share a polar value of tongue backing. Intervening sounds may intervene, so consonants interspersed within the word need not agree with this polar value. Such intervening but unaffected sounds are called transparent.

Transparent segments are important because they illustrate a potential abstraction in the organization of sounds into words. The phenomenon of adjacency suggests that phonological restrictions have articulatory/motor reasons for their existence, yet transparent segments suggest otherwise. Phonological theory accounts for transparent segments by construing harmonic sounds as adjacent at an abstract level.

However, there is more to be known about transparent segments. Linguists infer the mental representation of components and rules from the behavior of spoken language. In fact, there is an additional step of inference that normally goes unquestioned: We infer articulatory behavior from acoustic signals, and in turn infer representations from those inferences. Harmony and Transparency are articulatory phenomena – yet the current evidence for them is only acoustic in nature. Our goal is to determine whether the true articulation of transparent segments matches what is inferred from the acoustic signal. It may be that transparent segments are actually not impervious to the harmonic feature, and rather that the presence of the feature goes undetected by speakers and by linguists.

The only way to detect such a mismatch is to plot the movement of articulators inside the mouth instead of simply inferring from the acoustic signal. Our proposal is to use ultrasound technology to accomplish this. Other alternatives for obtaining such imagery are less feasible: X-ray filming is dangerous, while magnetic resonance imaging (MRI) cannot capture moving images.

We will apply our experimental procedure to a number of languages with harmonic systems, using ultrasound to capture images inside the mouths of native speakers. This allows for a straightforward determination about the articulatory nature of transparent sounds. Knowing what the harmonic feature is, we can test to see if the gesture is present in transparent sounds.

There are two possible outcomes for any individual case:

  • The transparent segment in the harmonizing environment is no different from its equivalent in a non-harmonic environment.
  • The transparent segment carries a gestural reflex of the harmonizing gesture.

Both results are interesting. The first confirms the traditional view that transparent segments are evidence that Adjacency is an abstract property of the pre-motor cognitive representations of spoken words. The second provides evidence that transparency is a perceptual trick, in that apparent transparent segments are interpreted by the perceptual faculty of the listener as being identical in harmonic and non-harmonic environments, despite having distinct articulations.

In short, we answer the question, does perception reconstruct mental representation because of articulation or despite it? Our research questions can be answered with an innovative but rigid experimental design, using objective measurements that are readily compared within and across subjects. The procedure adds a new dimension of measurement to the domain of language description

We foresee impacts of our research on linguistics, cognitive science, and beyond. We already know that mental representation of language uses abstract constructs: no speaker represents a spoken utterance simply as a linear sequence of sounds. With the special case of transparency in harmony, we can locate an instance of such abstractness with confidence, either in the mind of speaker or the mind of the listener. This result will solidify our understanding of the connection between cognitive representations and their physical manifestation.

In addition, our procedure is an innovative application of existing technology; the use of ultrasound in linguistics is in its infancy yet shows great promise for answering long-standing questions.

Finally, our research is important for the languages we intend to study, some of which are underdocumented and endangered. Solid generalizations about language must be based on a diverse sample of languages. This diversity diminishes when the last speaker of a language dies. To the extent that speakers of dying languages will work with us, our project will contribute to their maintenance by documenting the precise articulations while speakers remain alive.