Watch the Lips Say Ga, Hear the Voice Say Ba, and Report Da
Dub the sound of one syllable onto a film of a mouth shaping another, and people report hearing a syllable that's on neither track. The trick is old, simple and stubborn, and it shows sight reaching into hearing.
Here's the set-up. Take a short film of someone saying the same syllable over and over. Strip out the sound. In its place, lay in a recording of the same voice saying a different syllable, timed so the new sound lands where the old one was.
Now show it to people and ask what they hear.
Many of them name a syllable that was never recorded. Not the one on the soundtrack, not the one the mouth was making, but a third. This is the McGurk effect, first described in a 1976 study that grew out of an observation of people watching a dubbed film. Back then, hearing speech was generally treated as a job for the ears alone, even though most conversation happens with the speaker in plain view. The study showed a then-unrecognised way that vision feeds in, and the finding has held up: it's been repeated many times since and has sparked a large body of research.
The da nobody said
The film in that study showed a young woman's face. On the soundtrack she was saying ba, again and again. On screen, her lips moved as if she were saying ga.
Normal adults watching this reported hearing da.
Sit with how odd that is. Nobody on the recording said da. The sound was a clear ba and the mouth was a clear ga. What people reported was neither, and it wasn't a coin toss between the two. It was something new that seems to sit between them, which is why this version is called fusion. It's the best-known form of the effect, and for a lot of people it is the whole effect.
Not a muddy recording
A sceptic would ask the obvious question. Maybe the recording is just poor, ba and da sound alike on a cheap soundtrack, and people are guessing.
The 1976 study checked. When the same people heard the soundtrack with no picture, they reported what was there: repetitions of ba. When they watched the film without the dubbing, they reported what the mouth was really making: ga. Each track, on its own, was perfectly clear.
So the sound wasn't the problem, and neither was the picture. The mix-up appears only when the two arrive together and disagree.
Sight doesn't just help. It overrules
The usual story about lips and speech is a modest one. Seeing a face helps a bit, the way subtitles help, by giving you a second clue when the sound is hard to make out.
This is stronger than that. The sound in the demo was easy to make out; people got it right when they heard it alone. Adding the face didn't help them hear the voice better. It changed what they heard.
One proposed definition puts it plainly: mismatched visual speech causes a change in the category of what is heard, so the listener ends up with a single experience of hearing something other than what the voice said. The sight and the sound are merged, and the listener doesn't get to inspect the parts. Nobody hears ba, sees ga, and then works out a compromise. They simply hear the result.
That's why the fused version is so persuasive to researchers. Strictly, a study can only record what people say they heard; nobody can listen inside someone else's head. But if people were just choosing between what they saw and what they heard, they'd report one or the other. A syllable that matches neither is hard to explain unless the two were combined.
Three outcomes from one trick
The 1976 study also ran the dubbing backwards: the sound was ga and the lips were ba.
This time, a majority reported something like bagba or gaba. Not a single blended syllable, but both sounds, one after the other. Later descriptions treat this as its own variety, a combination rather than a fusion. The sound and the sight both get in, but this time they queue up instead of merging.
Swap in other pairings and you get other results. Put the sound of a b onto a face shaping a d, and people hear d: the picture simply wins.
Which outcome you get depends on how clear and reliable each channel is. When the sound is the more reliable of the two, people tend to hear what the voice said and the illusion is weak. When both carry useful information, you get the fusions and combinations. When the face is more reliable, people tend to hear the lips. That's why the effect is better thought of as a family than a single fixed result. Many people hear the fused da; some keep hearing ba. A clip that fools one viewer and not the next is behaving exactly as described.
The syllable you assemble
None of this is reserved for lab films. The same weighing goes on in ordinary conversation, where the face and the voice agree and there's nothing to notice; the face simply matters more when the sound gets harder to rely on.
The dubbed film just pulls the two apart and lets you catch the process at work. You're not hearing a voice and checking it against a mouth. You're building one syllable out of both, and handing yourself the finished product. Most of the time it's the right one. Sometimes, as with da, it's a syllable nobody said.
SOURCES
H. McGurk and J. MacDonald, "Hearing lips and seeing voices," Nature 264, 746–748 (23 Dec 1976). https://www.nature.com/articles/264746a0 (accessed 24 Oct 2026)
K. Tiippana, "What is the McGurk effect?," Frontiers in Psychology 5:725 (2014). https://www.frontiersin.org/journals/psychology/articles/10.3389/fpsyg.2014.00725/full (accessed 24 Oct 2026)
Comments