Faceache

Now that's what I call interactive music !

In Space no one can hear you drone on about game audio

I've heard the PC control scheme leaves something to be desired but I'm still looking forward to this game.

Original sound version interview

Music4Games

Another Music4Games - Interview with the composer Jason Graves

Interview about the technical implementation of the music with Don Veca.

Nobody likes to hear 'it's in the post'

In a recent article in gamasutra by Alexander Brandon 'Next Gen Audio Square-off', Gene Semel, audio director of SCEA says,

'I foresee more robust real-time logic systems that will allow sound designers and production directors to actually make decisions and mix the game at a "post-production" like stage of development. '

Looking at examples like Scarface (Rob Bridgett Gamasutra postmortem, + mix magazine article here) the opportunities for 'mixing' the game audio soundscape, as opposed to treating it in the fixed way that has traditionally been the approach up until now, are really exciting.....

Scarface GDC 2007


(Another interesting discussion on game mixing on gameaudiopro here)

However we need be be careful not to wish the 'post-production' role upon ourselves anytime soon. (I'm pretty sure this is not what Gene really meant but let's discuss it anyway...). As I discussed in my related blog post what is needed is better , earlier integration with the planning and development stages, not to ape the unhealthy practices of many films.

Over on 'Sound-article-list' Randy Thom suggested reading an article from Film Journal International,
'Setting the scene : Inside the world of the production designer'

'Below is a link to an article on Production Design. Trying to imagine what it would be like as a sound designer to be taken even ten percent as seriously as a production designer, I think it's useful to learn as much as possible about their process with the director.'

(Read the full thread here)

Bob Kessler sums up a typical scenario,

'I work primarily on lo/no/micro budget projects. Most of the time I am retained after the first edit is completed, and that happens when the client realizes that there is more to sound than having an unpaid first-time PA with nothing better to do hold a cheap mic on a broomstick somewhere in the vicinity of the talent. That being the case I spend the bulk of my time doing noise reduction or incredibly painful ADR sessions and then adding what touches I can before a quick mix. On those rare occasions when I am consulted before shooting begins I mostly spend my time lobbying for them to hire a competent production sound team.'

Dave Stone also describes how the structure of film-making and the concept of sound as a post-production process mitigates against creative involvement.

It was refreshing to read Sande Chen's excellent article on Gamasutra and the call for a multidisciplnary approach.

'By espousing this multidisciplinary approach to narrative design, developers can elevate the art of game development as well as increase the bottom line. Meaningful games require advance planning, but players benefit much from the integration of story, art, gameplay, sound, and music. Using themes, narrative designers ensure that each play experience is not only immersive, but also a meaningful one....

If we're going to build really powerful games, we need interdisciplinary teams," says Sheri Graner Ray. As a freelance game designer and production consultant, Ray knows firsthand the level of collaboration that can occur when artists, writers, designers, programmers, and composers work together....

As with the narrative designer, Ray recommends bringing in a sound designer and/or composer early as part of an interdisciplinary team. She understands fully the power of sound and music in games. Just recently, she heard the notes from a once-favorite game and experienced an emotional pull back to those times. "I almost got misty over it," she recalls. "It was like a family reunion." Truly, the emotional heartbeat of a game can be heard through its music and sound design. Narrative designers can work with composers and sound designers to strengthen the emotional connection so that players always have a powerful and meaningful experience.'

So at a time when the technology seems to finally be there to allow us to play an equal role in the storytelling within games let us not be relegated to the 'post' production model of film.

I'll leave the last words to John Broomhall from this Music4games interview.

'I believe game audio pros have always taken inspiration from all other media. Looking at movies in particular, and thinking about the craft and artistry of sound design, there’s 80 years of development that I’m sure all of us in audio have drawn upon. It’s very exciting that, as technology barriers and constraints fall away, we’re all on more of a level playing field technically - and it’s the power of ideas that really matters. Whether film or games, we are all creating stories, emotional moments, intense action and drama – and sound is an amazing tool to deploy. ........

A few years back, it was a huge blast simply to bring virtual worlds to life – and realism was the watchword. As time progressed, we became able to create a dense literal description of the world with carefully placed sound emitters and x, y, z co-ordinates in 3D geometry spaces, sporting scientifically accurate acoustic reflections and reverberation. ......But what about the subjectively chosen sound treatments that are used purely to enhance and underscore drama and narrative? How many games have fully explored the potent force of sound that tells a story, provides exposition and characterization, leads navigation and drives primal emotional responses – sound that works beyond the literal using metaphor and suggestion?........

It seems to me that we have a job of evangelization to do and that’s something that needs to happen in every developer all over the world, week-to-week.'

Game Sound Book

Karen Collins (of Gamesound.com) has written a rather good book !
I can highly recccommend it as probably the most academic (in a good way !) book on the subject so far. Particularly strong on dynamic and interactive music techniques it's a must read for anyone interested in game audio.

Multimodal Crisps

I'm always interested in examples of multimodal perception so the recent Ig Nobel Prize awards caught my attention. Charles Spence won an award for his 'improbable research' on THE ROLE OF AUDITORY CUES IN MODULATING THE PERCEIVED CRISPNESS AND STALENESS OF POTATO CHIPS.
(Summary here.)

"We investigated whether the perception of the crispness and staleness of potato chips can be affected by modifying the sounds produced during the biting action. Participants in our study bit into potato chips with their front teeth while rating either their crispness or freshness using a computer-based visual analogue scale."

It was found that ,

"The potato chips were perceived as being both crisper and fresher when either the overall sound level was increased, or when just the high-frequency sounds (in the range of 2-20 kilohertz) were selectively amplified."

Ig Nobel Prize - BBC Coverage

In other words the sound associated with the crisps affected how they taste. (Charles has also done some work with the Chef Heston Blumenthal who has created a dish where the customers listen to the sounds of the sea whilst eating.)

This is not as bizarre as it sounds as our senses work together to help us perceive the world. There's some evidence that vision calibrates audio in terms of spacial information, sound calibrates vision in terms of temporal information as they are respectively more accurate in their different fields. Another classic example of the slightly bewildering outcomes of this is the McGurk effect.




Still not convinced ?Watch these two versions of these blobs moving across the screen. Without prompting most people see them as bouncing off eachother firstly, before a different version shows them flying past eachother. (Of course its just the same video with different sound)



It's clear that such multimodal perception permeates our culture when we think of statements such as 'That tastes very sharp', ' It's bitterly cold', 'That's a loud shirt'. There's also been the suggestion by some that the corss modal influence of shape and sound may form the original of spoken language - as shown by the Booba / Kiki effect.




If one of these shapes is called a booba, and the other a Kiki then which is which ? 95-98% of people agree - but why would they unless there is some sort of relationship between the visual shape and the sound of the word. (Great series of 'Reith Lectures' given by Vilayanur Ramachandran in which he discusses this here)

Why is any of this of more than passing interest to us as sound designers ? Well it's multimodal perception that allows us to do what we do, characterising a visual object through the bonding of sound and image (Synchresis ) or stretching the relationship between the audio and visual to create a sum that's greater than it's parts.

These ideas concerning multimodal perception are closely linked with the concept / condition of Synesthesia (wiki) where sensory information of one mode of perception is involutarily experienced through another. For example seeing letters of the alphabet in consistent colours, or experiencing sounds as taste.

This has a long history in the arts (Summary of 'visual music' colours and synesthesia in the arts)
and continues to hold a fascination for many, including Tetsuya Mizuguchi, designer of the game REZ.

Of which more another time ....

Golan Levin has a comprehensive bibliography of synestheia and phonesthesia research.
See also John Waterworth's The case for synaesthetic media, do Get your synesthesia patching on : vvvv adds music features