(Summary here.)
"We investigated whether the perception of the crispness and staleness of potato chips can be affected by modifying the sounds produced during the biting action. Participants in our study bit into potato chips with their front teeth while rating either their crispness or freshness using a computer-based visual analogue scale."
It was found that ,
"The potato chips were perceived as being both crisper and fresher when either the overall sound level was increased, or when just the high-frequency sounds (in the range of 2-20 kilohertz) were selectively amplified."
Ig Nobel Prize - BBC Coverage
In other words the sound associated with the crisps affected how they taste. (Charles has also done some work with the Chef Heston Blumenthal who has created a dish where the customers listen to the sounds of the sea whilst eating.)
This is not as bizarre as it sounds as our senses work together to help us perceive the world. There's some evidence that vision calibrates audio in terms of spacial information, sound calibrates vision in terms of temporal information as they are respectively more accurate in their different fields. Another classic example of the slightly bewildering outcomes of this is the McGurk effect.
Still not convinced ?Watch these two versions of these blobs moving across the screen. Without prompting most people see them as bouncing off eachother firstly, before a different version shows them flying past eachother. (Of course its just the same video with different sound)
It's clear that such multimodal perception permeates our culture when we think of statements such as 'That tastes very sharp', ' It's bitterly cold', 'That's a loud shirt'. There's also been the suggestion by some that the corss modal influence of shape and sound may form the original of spoken language - as shown by the Booba / Kiki effect.
If one of these shapes is called a booba, and the other a Kiki then which is which ? 95-98% of people agree - but why would they unless there is some sort of relationship between the visual shape and the sound of the word. (Great series of 'Reith Lectures' given by Vilayanur Ramachandran in which he discusses this here)
Why is any of this of more than passing interest to us as sound designers ? Well it's multimodal perception that allows us to do what we do, characterising a visual object through the bonding of sound and image (Synchresis ) or stretching the relationship between the audio and visual to create a sum that's greater than it's parts.
This has a long history in the arts (Summary of 'visual music' colours and synesthesia in the arts)
and continues to hold a fascination for many, including Tetsuya Mizuguchi, designer of the game REZ.
Of which more another time ....
Golan Levin has a comprehensive bibliography of synestheia and phonesthesia research.
See also John Waterworth's The case for synaesthetic media, do Get your synesthesia patching on : vvvv adds music features
Posts
