McGurk effect, how does it work?

November 5, 2011 4:56 PM

Please explain the McGurk effect as if you're talking to a 5th grader.

In particular, why would watching this make me feel nauseated? I don't have a fear of heights and rarely experience vertigo, but I hear the "sound" implied by the action in the clip and then feel sick to my stomach. Is that a common effect of auditory illusions?

posted by fuse theorem to science & nature (14 answers total)

Hmm, I don't believe this has to do with the McGurk effect. So far as I'm aware, that refers to a specific phenomenon wherein people hear a 'wrong' phoneme based on deceptive visual information.
posted by threeants at 5:11 PM on November 5, 2011 [1 favorite]

I don't have an answer for what's happening to you, but I do know that that's not the McGurk effect.
posted by brainmouse at 5:11 PM on November 5, 2011 [5 favorites]

McGurk Effect Video (it breaks my mind)...
posted by milqman at 5:32 PM on November 5, 2011 [12 favorites]

I have a hard time figuring out what implied sound you hear. Like electronic interference, like feedback from a microphone? Or like that long plastic tube thing that you swing around in a circle and it makes a weird noise?
posted by sarae at 5:34 PM on November 5, 2011

The only physiological connection I can come up with (and I have experienced the sound induced nausea) is that the sound resonates with your vestibular (balance) system in the inner ear, giving your brain messed up positional information about your body, which causes nausea. If the sound isn't actually present but you imagine it anyway, and you've experienced the real thing, your brain may just be making the association. Just a guess.
posted by sarae at 5:37 PM on November 5, 2011

The sound doesn't have to directly, physically affect your vestibular system to make you feel nauseated.

Many input mismatches that can make your brain nervous that something has gone awry. For instance, the mismatch between your eyes (we're moving fast!) and your inner ear (we're sitting still) when riding in a car famously makes many people ill. Or spinning around a lot and then stopping, where the roles are reversed.

If your eyes (that's a g!) and your ears (that's a k!) are giving you different signals and you are particularly sensitive to that kind of thing, you brain might say "we might be poisoned! hurl!".
posted by DU at 6:15 PM on November 5, 2011

I have a hard time figuring out what implied sound you hear.

I "hear" the sound of something very large hitting the ground repeatedly. Something large like a power line tower, as implausible as that is, of course. I don't think the experience is unique to me; I originally saw the gif on another forum not too long ago (where it was perhaps mistakenly referred to as an example of the McGurk effect) and others reported "hearing" the implied sound as well.
posted by fuse theorem at 6:38 PM on November 5, 2011

That's an amazing effect!

I hear a sound when the transmission tower hits, too, but I don't think that's the primordial source of your nausea.

I think what really does it is the jolt of the picture as a whole, which you'll notice blurs and vibrates the picture, and is also delayed by a fraction of a second from the instant when the tower hits the ground.

The blur and the delay are both consistent with what would happen if a big mass slammed into the ground and a huge shockwave came along the ground toward you.

When that shockwave arrived at the point where you were standing if the scene were real and you were looking at it from the distance it seems to be, the vibrations would make your vision jump and blur just that way, and the delay is consistent with the time it would take for the shockwave to reach you.

However, your inner ear says nothing like that happened, and your brain chooses to believe your eyes, concludes something is wrong with your inner ear and makes you nauseated, possibly because the consequences of treating it as a false alarm-- not throwing up poison you have ingested-- could be so much worse than acting as if it's not real when it is than acting the other way around.
posted by jamjam at 6:50 PM on November 5, 2011 [1 favorite]

.. could be so much worse than acting as if it's real when it isn't, that is.

Good thing getting all turned around writng something isn't nauseating.
posted by jamjam at 7:01 PM on November 5, 2011

Okay, this question was very confusing until I figured out that you're not referring to the McGurk effect at all. McGurk is when a guy is saying a "d" sound but you hear a "b" sound because of the way the lips move.

I looked at the power line gif and it gave me a vague icky feeling, like slight nausea, but to me it stems from the way everything in the image is made to slightly shake when the tower "hits" the ground. I don't hear anything. I think it wouldn't bother me at all if there wasn't that shaking effect.
posted by IndigoRain at 8:36 PM on November 5, 2011

Both my partner and I "hear" a thud when the image shudders. I'd describe it as very quiet, and very low—verging on infra-sound. I can see how one would liken it to the McGurk effect, in that the visual signals are causing my brain to fill-in a sound that isn't even remotely there. Wild. I don't know why you're feeling nauseated, but is your experience of the faux sound something that's just below the range of normal hearing? That doesn't nauseate me, but I can imagine it would get to some folks.
posted by mumkin at 11:19 PM on November 5, 2011

Like mumkin, it doesn't make me nauseous but I do hear a very faint thud. The McGurk effect seems to be specific to speech but the effect does seem related - your brain is perceiving a sound based on visual input.
posted by missmagenta at 6:45 AM on November 6, 2011

In particular, why would watching this make me feel nauseated?

The only thing missing from jamjam's wonderful explanation is the common, familiar term for what he describes. So in the interest of completeness: When you watch that animation, you experience motion sickness as a result of the shaky camera.

In other words, your nausea is visually induced; it's not caused by the sounds you imagine. And you can easily confirm this with a simple experiment: while looking at your keyboard or out a window or somewhere else entirely, imagine the exact same sounds that you "heard" when you watched the animation. Do you feel nauseous from those imagined sounds alone, without the corresponding visual input?
posted by arm's-length at 5:41 AM on November 7, 2011

while looking at your keyboard or out a window or somewhere else entirely, imagine the exact same sounds that you "heard" when you watched the animation. Do you feel nauseous from those imagined sounds alone, without the corresponding visual input?

Yeah, actually I do. It's definitely sensation of something really large creating vibrations when it hits the ground repeatedly that gets to me. It's a "feeling" reaction more than a "seeing" reaction. So perhaps it's not the gif's imagined sound so much as my association of the "boom...boom...boom" sound with feeling the ground shaking.
posted by fuse theorem at 3:00 PM on November 7, 2011