PhD Thesis — Multimodal Speech Perception & Perceptual Learning

Situation

Speech is hard to perceive when it is produced differently — for example by speakers with Down syndrome, whose oral-motor control and auditory/somatosensory systems are affected. Their speech is often poorly intelligible, yet familiar interlocutors understand them better than strangers do.

For my PhD, conducted at GIPSA-lab (CNRS) under Marion Dohen, Jean-Luc Schwartz, and Amélie Rochet-Capellan, I reframed intelligibility as a cooperative act between speaker and listener — and studied the listener side: how audiovisual cues and perceptual learning help people understand atypically-produced speech.

Task

Two research questions anchored the work:

• Does seeing the speaker's face improve perception of consonants produced by a speaker with Down syndrome, despite atypical anatomy?
• Can familiarization — and imitation during listening — help listeners better perceive this speech?

Action

Two experiments built on classic paradigms in the speech-perception literature, with controlled, reproducible protocols:

• Audiovisual speech perception in noise: measured consonant perception with and without the visual channel. Seeing the speaker's face improved intelligibility comparably to typical speakers — the visual channel is relatively preserved despite atypical production.
• Perceptual learning through familiarization ± imitation: adapted a familiarization paradigm to test whether imitating heard words boosts perception of a Down-syndrome speaker's speech. Results suggest it does — imitation recruits the listener's internal motor representations.

Statistical analysis with mixed-effects models over perceptual and acoustic data.

Result & contributions

• Original, peer-reviewed evidence that listeners adapt — that the listener, not just the speaker, carries part of the intelligibility burden.
• Clinical and theoretical perspectives: atypical production is a lens on the generality of speech-perception mechanisms established with typical speakers, and on the role of the listener's motor system in perception.

Publications

• Thesis: "Perceiving speech when it is produced differently: study of multimodal/multisensory familiarization mechanisms between ordinary speakers and speakers with an articulation disorder" (defended 26/06/2019) — theses.fr
• Journal article: "Does the visual channel improve the perception of consonants produced by speakers of French with Down syndrome?", Journal of Speech, Language, and Hearing Research, 61(4), 2018
• Full list of peer-reviewed articles on Google Scholar