DSP plugins and real-time mixing for games

Guy Whitmore talks to John Broomhall in develop this month (here) about using the greater processing power of current gen consoles for real time effects. Guy has a permenant place on my list of games audio genius's for his score to 'No one lives forever', probably still one of the most convincing adaptive games scores out there, and certainly the one that first convinced me it was possible. He discusses it on the IASIG site.

It's great to hear them discussing the reasons why the adoptiong of real time DSP and mixing is vital and that is to move away from the very fixed way in which games audio is usually treated.

John Broomhall: As for dynamic mixing – presumably we also want to get funky with fader moves and DSP behaviours (in response to game variables we’re polling) for purely creative, moment-enhancing, subjective-effect reasons?

Guy Whitmore: Yes - I think most dynamic mix decisions would be based on emotional subjective things rather than realism. For me, mixing for games is way too stuck in a literal distance-based approach – everything getting quieter the further away it is, regardless of how important it is. We need a re-think and again, to me, DSP is a very important component, particularly when you’re getting into ‘I want this section to feel dreamy the second time’ or ‘I want the player to suddenly feel the sadness of the character’s feeling’. It’s the same type of emotional choices a sound designer for a movie might make but working in a non-linear context…

If you listen to a film soundtrack there is often a deliberate attempt to highlight the story through a manipulation of the soundtrack, for example by hearing the filmworld through one of the characters' subjective point of view. (Read Randy Thom's 'Designing a movie for sound' for more thoughts on this subject + all of filmsound.org while you're there !). When audio in games is treated as a fixed system we miss out on all of these creative opportunities.

Whilst working on a game I was once pleasantly surprised to come across a game designer who'd actually thought about sound.. At a certain point (when the player was crawling through an air duct above a room full of hostiles - classic !) he wanted all the sound to fade out and just to highlight the players movements and breathing in this confined space. Great idea (he'd obviously recently watched Bressons 'A man escaped') but impossible to do with our then 'state of the art' audio engine without significant amount programming time (and of course that wasn't going to happen was it !).

Over the past few years I've noticed a few specific instances where the audio system has started to become more flexible (in the now ubiquitous 'Shell Shock' effect - volume down + filtered sounds after proximity to an explosion - first noticed by me in the original 'Call of duty' but probably used somewhere before then) so lets hope this is a continuing trend.