The Games Audio Tutorial (Beta)

Ok so it's a it later than promised (a bit of feature creep and a lot of procrastination) but after a successful airing at the recent AES convention here is our game audio tutorial, built as a playable level in the Unreal 3 engine. By 'playing' the level and opening it up to see how it works you'll hopefully learn a bit about interactive sound and music for games and specifically how to implement it using the Unreal engine. (See the demo video here. )

You'll need a copy of Unreal 3 to run it. Follow the instructions in the 'Readme' file.
You can download the level from the IASIG website or here.

We hope you enjoy it,
Dave Raybould, Richard Stevens.

ps. Thanks again to the fantastic Rodrigo Rubilar for providing the music cues for the level !

pps. For those of you running Unreal via Steam ... In your steam 'My Games' tab select Unreal Tournament 3. Right click and select 'Properties'. Under the general tab you can either create a desktop shortcut and alter it as described in the README, or under 'Set Launch options' type -useunpublished.

Special edition of Audio-Media on Games Audio

This months Audio-Media magazine is a special edition on games audio.
Download the PDF for free here.

Infamous - Doc on the music

Two of my favourite things - interesting sound sources and interactive music.
Great little documentary from Wired.

Interactive music with WWise

Article on the interactive music system in WWise.
http://www.music4games.net/Features_Display.aspx?id=317

Audiokinetic's increasingly popular middleware is also now fully integrated with the Unreal 3 engine but unfortunately this capability seems only to be available to official developers - Boo !!

Unreal 3 Sound and Music Implementation

To coincide with the release of Unreal 3's 'Titan' patch this weekend I've decided to make public an old tutorial for those who are still getting to grips with what you can do r.e. sound within the engine. You can download it here :

UnrealAudioTutorial

I hope that some of you find it useful !

My colleague Dave Raybould and I have also built an interactive tutorial that covers sound and music in a great deal more depth. We recently presented it at the AES conference in London (http://www.aes.org/events/35/). A taster here :

Music, Silence, Timbre

Today's post is brought to you by a recent hard drive partitioning disaster and Music4games recent interview with Kevin Reipl.

M4G: What would you like to hear more of from game soundtracks?
Kevin Riepl: More silence, I think a lot of games use music too too much, playing and looping it constantly, under action sequences, under ambient settings, all over. As a gamer, I personally get more engulfed into a game when the sound design takes center stage to convey the environment. Of course I think music is an integral part of gameplay and story, it just seems that its used all too often as a blanket. When voice acting is top notch as is sound design, music can then be brought in to emphasize key story plots or important gameplay sequences.

Kevin's views about wanting more silence in games (or at least less music) is a surprisingly common comment from composers (both for games and films). In a thought provoking discussion (Mike Figgis - Walter Salles - Composer and director : Exploring the collaboration) Mike Figgis describes the over use of music in film like a kind of addiction.

Mike Figgis: ...a really important question for filmmakers is: When do you introduce your first piece of music on the film? And my experience is this, I try now not to put any music on for a long long, long time. Because I know that the minute I use fifteen seconds of music the film has lost its virginity, and has become a kind of sex maniac. In a way it just can’t have enough. And it has worked really well as a virgin up until that point, and then the minute you put something on you kind of go, wow, now it really sounds like a film, let’s get some more of that stuff. And then the temptation is just to wallpaper the entire film with delicious music that you have stolen from somewhere else.

One of music's most important roles in film is realised simply by being there. Stephen Deutsch, (Soundscapes : The school of sound lectures)

‘In most films , the appearance of any music at all is a signifier of emotional content. When the music comes on , we are told that the scene is invested with emotional significance and is different from those scenes which contain no music.’

So by having music there all the time in a game we are actually undermining it's effectiveness. (The increasingly blurred boundaries between 'musical' atmospheres and 'sound design' will have to wait until another post !)

Kevin goes on,

Also, I think video games need to stray away from trying to have the Hollywood sound. That certain sound that has become sort of a ‘brand’ in Hollywood is now heavily leaking into games and is causing many upon many of the games out there to sound very similar. Now don’t get me wrong, I am not saying there is anything wrong with that style of music and by all means I am all for and love cinematic epic style writing and some games indeed call for that, but I honestly think if the industry as a whole had more confidence in being who they are and stopped trying to emulate film, it would allow composers more freedom and a wider palette to provide a signature sound for a game. Also, I think as an industry we have already met the standard of film and have passed it. Video games and film need to now head off in their own respective evolving direction.

I'm not one of those people who think that the only real 'game music' was from the glory days of 8-bit chip tunes but I think the point is well made. I was struck by how widespread the homogenisation of the 'Hollywood' score towards the orchestral template is when I tried to think of a single 'blockbuster' hollywood film in recent years that didn't have a full orchestral score (+ the obligatory pounding percussion) - I couldn't. If we look at the top grossing movies 2007 you can see what I mean (I can't speak for 'Wild hogs' as I haven't seen it and, unless trapped at 30,000 feet in some sort of in-flight-movie-hell, probably never will). (See also the USA top 100 box office films). In aspiring to create the 'blockbuster' game (particularly 1st / 3rd person shooters) it seems like game directors and producers are adopting a similarly myopic stance.

In the music history class during my first degree we were once given an assignment - 'Write the history of western music on the back of a postcard'. (I'm a great believer that if you really, truly understand something that you should be able to explain it simply and concisely - otherwise you don't really understand it, so this is in my view a great assignment to do on any subject !). Being immersed in the detail of music history at the time of course I thought it impossible, and failed miserably. Now however I think I may have at least a stab at fitting it onto the back of a postage stamp instead. The history of western music perhaps goes as follows :

Rhythm, Melody, Harmony, Timbre.

Discuss.....

I think that in the age of timbre it is time that game producers had the confidence in their composers to allow them to use a wider palatte of instrumentation and sound. It's interesting to note how the film soundtracks that have explored this have become so influential (I'm talking about 'scored' movies here - not ones that use a lot of soundtrack music).

Morricone's 'westerns', Vangelis 'Blade Runner', Mancini's 'Pink Panther', Bernard Hermann's 'The day the earth stood still', and 'Taxi driver' (not to mention the unusual use of strings in 'Psycho') and other iconic scores are characterised not purely on their melodic / harmonic / rhythmic approach but by their instrumentation.

So a call for less music - more timbre ?

How does all this relate to my personal hard drive tragedy ? Well the drive that went down was the drive called 'Patches' that, in addition to having various commercial sample banks (which given endless patience I can reinstall), had numerous sample patches that I'd put together over the last few years. So given this discussion maybe it's a good impetus to get out the mic and start creating some new ones instead of reaching for the nearest 'Orchestral sample patch 101' ?

Faceache

Now that's what I call interactive music !

In Space no one can hear you drone on about game audio

I've heard the PC control scheme leaves something to be desired but I'm still looking forward to this game.

Original sound version interview

Music4Games

Another Music4Games - Interview with the composer Jason Graves

Interview about the technical implementation of the music with Don Veca.

Nobody likes to hear 'it's in the post'

In a recent article in gamasutra by Alexander Brandon 'Next Gen Audio Square-off', Gene Semel, audio director of SCEA says,

'I foresee more robust real-time logic systems that will allow sound designers and production directors to actually make decisions and mix the game at a "post-production" like stage of development. '

Looking at examples like Scarface (Rob Bridgett Gamasutra postmortem, + mix magazine article here) the opportunities for 'mixing' the game audio soundscape, as opposed to treating it in the fixed way that has traditionally been the approach up until now, are really exciting.....

Scarface GDC 2007


(Another interesting discussion on game mixing on gameaudiopro here)

However we need be be careful not to wish the 'post-production' role upon ourselves anytime soon. (I'm pretty sure this is not what Gene really meant but let's discuss it anyway...). As I discussed in my related blog post what is needed is better , earlier integration with the planning and development stages, not to ape the unhealthy practices of many films.

Over on 'Sound-article-list' Randy Thom suggested reading an article from Film Journal International,
'Setting the scene : Inside the world of the production designer'

'Below is a link to an article on Production Design. Trying to imagine what it would be like as a sound designer to be taken even ten percent as seriously as a production designer, I think it's useful to learn as much as possible about their process with the director.'

(Read the full thread here)

Bob Kessler sums up a typical scenario,

'I work primarily on lo/no/micro budget projects. Most of the time I am retained after the first edit is completed, and that happens when the client realizes that there is more to sound than having an unpaid first-time PA with nothing better to do hold a cheap mic on a broomstick somewhere in the vicinity of the talent. That being the case I spend the bulk of my time doing noise reduction or incredibly painful ADR sessions and then adding what touches I can before a quick mix. On those rare occasions when I am consulted before shooting begins I mostly spend my time lobbying for them to hire a competent production sound team.'

Dave Stone also describes how the structure of film-making and the concept of sound as a post-production process mitigates against creative involvement.

It was refreshing to read Sande Chen's excellent article on Gamasutra and the call for a multidisciplnary approach.

'By espousing this multidisciplinary approach to narrative design, developers can elevate the art of game development as well as increase the bottom line. Meaningful games require advance planning, but players benefit much from the integration of story, art, gameplay, sound, and music. Using themes, narrative designers ensure that each play experience is not only immersive, but also a meaningful one....

If we're going to build really powerful games, we need interdisciplinary teams," says Sheri Graner Ray. As a freelance game designer and production consultant, Ray knows firsthand the level of collaboration that can occur when artists, writers, designers, programmers, and composers work together....

As with the narrative designer, Ray recommends bringing in a sound designer and/or composer early as part of an interdisciplinary team. She understands fully the power of sound and music in games. Just recently, she heard the notes from a once-favorite game and experienced an emotional pull back to those times. "I almost got misty over it," she recalls. "It was like a family reunion." Truly, the emotional heartbeat of a game can be heard through its music and sound design. Narrative designers can work with composers and sound designers to strengthen the emotional connection so that players always have a powerful and meaningful experience.'

So at a time when the technology seems to finally be there to allow us to play an equal role in the storytelling within games let us not be relegated to the 'post' production model of film.

I'll leave the last words to John Broomhall from this Music4games interview.

'I believe game audio pros have always taken inspiration from all other media. Looking at movies in particular, and thinking about the craft and artistry of sound design, there’s 80 years of development that I’m sure all of us in audio have drawn upon. It’s very exciting that, as technology barriers and constraints fall away, we’re all on more of a level playing field technically - and it’s the power of ideas that really matters. Whether film or games, we are all creating stories, emotional moments, intense action and drama – and sound is an amazing tool to deploy. ........

A few years back, it was a huge blast simply to bring virtual worlds to life – and realism was the watchword. As time progressed, we became able to create a dense literal description of the world with carefully placed sound emitters and x, y, z co-ordinates in 3D geometry spaces, sporting scientifically accurate acoustic reflections and reverberation. ......But what about the subjectively chosen sound treatments that are used purely to enhance and underscore drama and narrative? How many games have fully explored the potent force of sound that tells a story, provides exposition and characterization, leads navigation and drives primal emotional responses – sound that works beyond the literal using metaphor and suggestion?........

It seems to me that we have a job of evangelization to do and that’s something that needs to happen in every developer all over the world, week-to-week.'

Game Sound Book

Karen Collins (of Gamesound.com) has written a rather good book !
I can highly recccommend it as probably the most academic (in a good way !) book on the subject so far. Particularly strong on dynamic and interactive music techniques it's a must read for anyone interested in game audio.

Multimodal Crisps

I'm always interested in examples of multimodal perception so the recent Ig Nobel Prize awards caught my attention. Charles Spence won an award for his 'improbable research' on THE ROLE OF AUDITORY CUES IN MODULATING THE PERCEIVED CRISPNESS AND STALENESS OF POTATO CHIPS.
(Summary here.)

"We investigated whether the perception of the crispness and staleness of potato chips can be affected by modifying the sounds produced during the biting action. Participants in our study bit into potato chips with their front teeth while rating either their crispness or freshness using a computer-based visual analogue scale."

It was found that ,

"The potato chips were perceived as being both crisper and fresher when either the overall sound level was increased, or when just the high-frequency sounds (in the range of 2-20 kilohertz) were selectively amplified."

Ig Nobel Prize - BBC Coverage

In other words the sound associated with the crisps affected how they taste. (Charles has also done some work with the Chef Heston Blumenthal who has created a dish where the customers listen to the sounds of the sea whilst eating.)

This is not as bizarre as it sounds as our senses work together to help us perceive the world. There's some evidence that vision calibrates audio in terms of spacial information, sound calibrates vision in terms of temporal information as they are respectively more accurate in their different fields. Another classic example of the slightly bewildering outcomes of this is the McGurk effect.




Still not convinced ?Watch these two versions of these blobs moving across the screen. Without prompting most people see them as bouncing off eachother firstly, before a different version shows them flying past eachother. (Of course its just the same video with different sound)



It's clear that such multimodal perception permeates our culture when we think of statements such as 'That tastes very sharp', ' It's bitterly cold', 'That's a loud shirt'. There's also been the suggestion by some that the corss modal influence of shape and sound may form the original of spoken language - as shown by the Booba / Kiki effect.




If one of these shapes is called a booba, and the other a Kiki then which is which ? 95-98% of people agree - but why would they unless there is some sort of relationship between the visual shape and the sound of the word. (Great series of 'Reith Lectures' given by Vilayanur Ramachandran in which he discusses this here)

Why is any of this of more than passing interest to us as sound designers ? Well it's multimodal perception that allows us to do what we do, characterising a visual object through the bonding of sound and image (Synchresis ) or stretching the relationship between the audio and visual to create a sum that's greater than it's parts.

These ideas concerning multimodal perception are closely linked with the concept / condition of Synesthesia (wiki) where sensory information of one mode of perception is involutarily experienced through another. For example seeing letters of the alphabet in consistent colours, or experiencing sounds as taste.

This has a long history in the arts (Summary of 'visual music' colours and synesthesia in the arts)
and continues to hold a fascination for many, including Tetsuya Mizuguchi, designer of the game REZ.

Of which more another time ....

Golan Levin has a comprehensive bibliography of synestheia and phonesthesia research.
See also John Waterworth's The case for synaesthetic media, do Get your synesthesia patching on : vvvv adds music features

Gears of war 2

Gears of war 2 - big crunchy sounds - yeah !
Developer diary - Touch the sound.

Audio Engineering Society - Audio for games !

The Audio Engineering Society 35th International conference is on 'Audio for Games' !
Should be good. 11-13th Feb in London UK.

Earth's Sonic Mysteries from Noise Addicts

 'If you listen closely, you’ll find that the earth is full of sounds.  Some are things that you hear every day, some are truly remarkable and some sounds hail from origins completely unknown.  What follows here is a list of “sonic mysteries” for your pleasure - many of them include audio.'

Strange an mysterious Sounds from the Earth - Mistpouffers,hum,bloop,Hell Hole,slow down,wow | Noise Addicts music and audio blog

'Found sound'

Taking Luigi Russolo's advice from 'The Art of Noises' Paul Hartnoll (Orbital) has worked on the latest VW Golf ad 'Rhythms of everyday'.

“We must break out of this narrow circle of pure musical sounds and conquer the infinite variety of noise sounds.” (Russolo)

Making of here.

Reminscent of Coldcuts 'Timber' it mixes the use of found sounds with a close audio-visual integration.



Coldcut were / are (?) involved in the development of the VJamm software - here's an illustration of how they use it in their sets.



As well as being open to all kinds of sound sources the use of 'Found sound' can also raise political points such as in Matthew Herberts 'Plat du jour' which examines problems in our food chain through the use of related sounds to create the music itself.

'I am tired of having to tolerate the international language of cheap convenience food - convenient mainly to those that make and serve it. The bright pinky orange of farmed salmon in aeroplane trays, the branded waters 1000 times more expensive than tap water, the dismal spread of the hotel breakfast buffet, with its pre-formed meat slices, pasteurised juices, mechanically produced bread and nestle yoghurts full of sugar and potassium sorbate…

This record then, aims to tell some of the hidden stories behind the overly-elaborate and wasteful packets. It looks at what's on the menu and asks you to makes decisions based on criteria other than taste. The album will include tracks made from a grain of sugar, 30,000 chickens, a salmon farm, the sewers below London and water.'

Similarly Matmos' album 'A chance to cut is a chance to cure' uses sounds recorded during plastic surgery procedures (not for the squeamish). (Good interview with Matmos here.)

On a games related note below is a track made by using the weapon sounds from 'Call of Duty' by Serpento.

Portal and Vocal Characterisations

Coming late to the Portal party I have to say I really enjoyed the game although all the flying through the air and flipping around did make me a little queasy at times. There's been a lot of comment on the great voice characterisation of Glados and of course the now infamous Portal Song - don't watch this unless you've completed the game as it will spoil it !

There was some discussion of how the process was done at Engine Audio but it seems to have disappeared and so below is a youtube version instead :



There are a couple of ideas about how to recreate the effects here and CosmicD has done a tutorial on how to recreate the vocal effect using Melodyne (while your there look at the video doc on their new 'direct access' technology - impressive and certain to result in a flurry, more like slurry, of crazy remixes featuring Girls Aloud singing Bach Cantatas etc).


Whilst GlaDOS is great she still doesn't beat my favorite menacing-female-sci-fi-type-character Shodan from System Shock.

If you enjoyed the game then don't forget to check out the flash version and 'Still Alive' for the NDS. Also the highly amusing 'Day in the life of a turret'.

Don't try this at home ! Warhammer music.

Came across this video via originalsoundversion blog. Probably not a good idea !

Wall-E Sponsored by Mac ?

Unusually Ben Burtt as sound designer has been doing the promotional rounds for the film 'Wall-E'. (See my previous post). A great film with as you would expect some really nice vocal type sound characterisations.



A smirk was raised by Wall-E's 'reboot' sound which is uncannily similar to the Mac startup sound - a bit of an in-joke I guess. Reminded me of this related prank.





More on the mac startup sound here. I'm obviously not the first to notice this ! More on Mac referenced in Wall-E here.

Name That Game - Game sounds quiz

The Video Game Revolution: Name That Game | PBS

I did particularly badly in this - not enough ZX Spectrum classics !

Game editors - Fun and Art ?

This looks fun, but I don't know if anybody is still playing Quake III. Probably a few dedicated people left ?

Simpsons map for Quake III Arena - Boing Boing

As these game editors are so easy to use I'm surprised not to see a lot more experimental or artistic work taking place using them. I'm working on an installation using the Unreal 3 engine right now but haven't seen a great deal of other stuff.

I love this work by Alison Mealey (interview here) that uses Unreal Tournament processes to generate visual art.


Kurt Hentschläger has produced this work 'KARMA/cell' using the Unreal 3 engine.

There's a large array of digital art taking place in Second Life but if anyone knows of other game engine based work - let me know !