Audio For Mobile Games
Issue: March 1, 2015

Audio For Mobile Games

On the train platform on your commute home from work, on the plane before take-off, at your nephew’s middle school winter recital, and even in the bathroom — yeah, I went there and you know you have too — everyone seems to be occupied with mobile games. And not simply as a means of killing time. These games offer stolen moments of self-indulgence, of escape, of risk-free interaction with strangers. Today, as users demand higher quality entertainment crammed into those 100MB download packs, these AAA games in the palm of your hand are delivering 3D-graphics, better game interfaces, and richly-designed sound (that doesn’t annoy, and doesn’t distort). 

FIFA 15 Ultimate Team for Android 

EA Sports' FIFA franchise is known for recreating the exciting game-day atmosphere of the soccer world’s most popular clubs. And FIFA 15 builds on past success by including ambience and cheers recorded in various stadiums at over 20 Barclays Premier League matches. Live crowd reactions for goals and misses, whistles, and club chants during those games were captured on 24 mics placed around the stadium. Those real, authentic team sounds elevate the game experience to another level. 

In addition to the FIFA 15 console game, senior audio artists Amos Hertzman and Tyler Berrie at [Audioworks] EA Canada in Vancouver, British Columbia, Canada (www.ea.com/ca) designed the crowds, sound effects and music systems for the mobile version, FIFA 15 Ultimate Team for Android, using EA's proprietary audio tools. They worked in conjunction with Emanuel Balaceanu, EA Romania’s audio artist responsible for integrating into the mobile version many of the audio assets created by Hertzman and Berrie for the console version. 

It’s no easy task boiling down 5GBs worth of game audio content to fit within the mobile version’s 250MB initial download limit. One key to making it work is to offer additional content in separate downloads. For instance, Berrie notes that after the initial download, users can choose a commentary language: English, French, Spanish or German. “That extra commentary chunk is 400MBs,” says Berrie. 

Ultimately, to shrink the amount of audio data to mobile-friendly size, choices have to be made. So what stays and what goes? Hertzman says, “We have so many licensed teams in our game, over 250, and we can’t include all those authentic crowd reactions and chants for all those teams in the mobile version. We see what teams are getting played the most and focus on those, making sure they sound the most authentic.” 
 


To save on data, the stadium ambience tracks are downmixed from 7.1 to stereo, and general ambience elements are shared between different teams that wouldn’t be shared on the console version. “On the gen-4 consoles, we have the space to get very specific,” says Berrie. “When we look at putting those on mobile, we hand-pick chants, so maybe five chants from Liverpool and three chants from Manchester United. Generally, we can deliver only one or two chants instead of seven or eight,” adds Hertzman. 

The team used iZotope RX 4 Advanced to clean up all the recordings captured on-location at the matches. “If there are individual shouts we wanted to remove from the crowd, we used the Spectral Repair feature,” says Hertzman. 

The music tracks are also cherry-picked. From the 50 tracks included in FIFA 15, roughly 10 are chosen for FIFA 15 Ultimate Team for Android. “We have a stellar soundtrack we can choose music from. We were able to capitalize on our amazing FIFA console soundtrack to use in the mobile version,” says Hertzman. 

FIFA 15 Ultimate Team for Android features unique sound design to support interface functions, such as swiping and tapping. There are also unique reward sounds to reinforce players’ purchases and achievements in the ultimate game mode. “That mode is all about collecting and gaining special cards and building your absolute ultimate team. The sounds are a bit synthy, with card shuffling elements as well, to make it feel like the player just opened a pack of collector cards,” says Hertzman. The team creates their sound design in Pro Tools, choosing elements that match the visuals, as well as characteristics of front-end elements from other EA Sports games. 

For the most part, the audio for FIFA 15 Ultimate Team for Android closely matches the FIFA 15 console game. Thanks to proprietary in-house audio authoring tools, Hertzman and Berrie are able to essentially use the same sound engine and share assets between gen-3, gen-4, and mobile products. “We can really leverage as much of the behavior and work that we are doing with our console version with the mobile products. Our integrator at EA RO, Emanuel Balaceanu, is helping to pick and test what assets are making it into the mobile version,” explains Berrie. He claims Balaceanu has a wall full of mobile devices on which he can test the game. Their goal is to make one mix that will playback well on different speakers, in different listening environments. “Most importantly, the commentary [designed by audio artist Kevin Maeng at EA Canada] must cut through in all playback situations. That’s always our goal,” concludes Hertzman.

Eisenhorn — Xenos

Chris Sweetman and Samuel Justice are the sound designers behind the upcoming Pixel Hero Games release Eisenhorn — Xenos, based on the Warhammer series Inquisitor of the Ordo Xenos, Gregor Eisenhorn. Sweetman and Justice started their cleverly-named, game-audio company, Sweet Justice, eight months ago, bringing together their many years of experience in the game audio biz. Sweet Justice (www.sweetjusticesound.com) is located in Worthing, England, but they work remotely with clients all across the UK and here in the States. Their recent game audio work includes the titles Call of Duty, Evolve, Star Citizen and Fortnite. 

According to the Pixel Hero Games site, Eisenhorn — Xenos is a 3D action/adventure game set in the grim, dark future of Warhammer 40K, where inquisitor Eisenhorn (who’s job description includes uprooting heretics and witches) is sent out into the galaxy to fight and protect the imperium of man from threats. 

Eisenhorn — Xenos will be available later this year on mobile devices as well as on Steam (an Internet-based game distribution and social networking platform compatible with PC, Mac and Linux systems). While the soundtrack for the mobile and Steam versions will be relatively similar, Sweetman notes one big consideration is the capacity to play back the audio. “You’re thinking about the amount of memory you have to store the sounds, but also you’re thinking about the ways the sound is transmitted to the player. For console or computer, players are either using TV speakers or a surround sound system. But with mobile games, you’re using either ear buds, or a little plug-in system, or the mobile phone speaker itself. You have to think about sound a bit differently because there are certain frequencies that will not transmit properly across the mobile phone, particularly low-end frequencies,” he explains. Therefore the mobile version’s soundtrack will have less low-end information. Taking out explosion sweeteners will help the team stay within their audio data budget.



While data size matters, it’s not nearly as strict as it used to be for mobile devices. Sweetman cut his teeth on game audio for the original PlayStation system, which he says allowed only 512k of data. Mobile games today can have a whopping 20MB allocated for audio data. “Compared to 512k, that’s a massive amount. But a lovely thing is we can go back in time and use the old tricks we had back then to save space and meet our data limitations,” Sweetman says, who explains there are different schools of thought on how much variation there should be on key sounds. “Take Japanese games, for instance, they believe in repetition, whereas in the West, we like variation. For example, a sword in a Japanese game may have two or three sounds and they train the player to recognize those sounds. In the West, we tend to have 150 variations for that one sword,” says Sweetman. The more variation, the more data required. While ultimately the decision on variation lies with the client, Sweetman feels that repetition in mobile games can be a good thing. “I’m more concerned about having gameplay feedback for the player. When a certain sound is played, the player knows a certain action happens. You know you completed a parry perfectly because there is only a few sword sounds.”

The processing capabilities and storage capacity of mobile devices is increasing exponentially. That, coupled with the proliferation for middleware programs, like Audiokinetic’s Wwise, have boosted the possibilities for mobile game audio says Sweetman. “Wwise creates your audio engine depending on what graphics engine the game uses — in the case of Eisenhorn — Xenos, it’s Unreal Engine, and bolts straight into it. Wwise enables you to integrate audio in ways you realistically couldn’t do otherwise. Often you don’t have the budget or the workforce capacity to designate someone to create the audio tools you need on a particular production, especially when you’re working on a remote freelance basis, which is how we operate at Sweet Justice.” Sweetman and Justice design, edit, and mix in Pro Tools and then use Wwise to implement sounds, including Eisenhorn — Xenos’ score by composer Joshua Crispin, very quickly into the game. In working remotely with clients, Sweetman explains, “A key thing for us is making sure we have a very strong workflow.” 

Sweetman and Justice focused most of their attention on creating the signature sound for Eisenhorn, voiced by actor Mark Strong (recently appearing in The Imitation Game). His weapons, clothing and accessories — namely his ornate belt, footsteps, and special telepathic and psychic abilities — are all covered in over 150 discrete files. That’s not a lot considering the size of the game and all the different things Eisenhorn can do, adds Sweetman. 

Eisenhorn’s weapons, Sweetman notes, are all built in accordance with the Warhammer universe (for those Warhammer super-fans closely scrutinizing the game). “Most of these weapons never had a sound before. We got concept art and built the weapons’ sounds based on what we saw. It was up to us to make the sounds for them. It was a lot of fun,” he says. 

Wave Champions 

Richard Aitken, a veteran of the audio biz, has worked on a wide range of game titles, most notably the 2010 Ivor Novello Award-winning Killzone 2 [composed by Joris de Man] — the first time an Ivor has ever been awarded to a video game soundtrack. Aitken brings computer programming and professional songwriting experience to all the game soundtracks he crafts. From AAA to mobile games, Aitken seeks to raise the bar for game audio. “Where I want to go with mobile game audio is to have it treated like another headline format,” says Aitken. And he’s in good company as far as game developers are concerned. He’s been working with Kwalee — a developer and publisher of mobile games based in Leamington Spa, England. According to Aitken, Kwalee is a company “full of big leaguers, massively experienced with AAA titles. What they’re aiming at is slightly ahead of the technology curve. They want to have AAA games on mobile devices, with a quality and interaction on par with console games.” 



Aitken’s first experience with Kwalee was on Farm Fighters — a turn-based shooting game with farm animals. Aitken came in at the end of the project and only tweaked the sounds already designed for the game. “It was interesting to see what a non-audio person did with sound,” notes Aitken. “There was little adherence to loudness quality, no real balancing.” According to Aitken, the audio implementation for Farm Fighters was very straightforward and didn’t involve middleware. When a specific action happened, a specific sound would play. In contrast, Aitken’s second project with Kwalee, the upcoming Wave Champions, is much more sophisticated.

Wave Champions, to be released later this year, is a boat racing game where players can draw their own race courses and challenge random opponents online. Unlike the simple sound effect approach for Farm Fighters, Wave Champions is designed to give players an audio experience. “I really pay attention to balancing game audio because it is a dynamic medium and so you have to use dynamic changes,” explains Aitken, who used middleware by Tazman-Audio called Fabric that works with the Unity game engine and allows independent audio designers to easily integrate their audio assets into a game. “Middleware engines are becoming ubiquitous,” adds Aitken. With Fabric, he was able to create side chain loops, giving certain sounds precedence over others. For example, in Wave Champions, collision sounds take precedence over every other sound in the mix. All side chained sound elements are turned down, and the collision sound dominates the mix. 

In addition to a dynamic mix, Wave Champions’ highly-detailed soundtrack features custom recordings of boat engines. Aitken captured sounds from small hover crafts, Sea Doos, jet boats and swamp boats. “There is a whole load of engine detail in there. I recorded various boat engines, cut all the sounds, and re-integrated them so they work in a fairly-interactive manner,” says Aitken. To create a wide variety of collision sounds from a small amount of discrete sounds, Aitken added meta-tags on all the sounds, then built a collision engine to trigger one water sound, one plastic sound, and one concrete sound (since the boat has a plastic hull, it’s hitting something made of concrete, and it’s on the water) to make a unique collision. “If you have a pool of 10 sounds for each, then what you essentially have is 10x10x10. Now you have 1,000 variations for the collisions. I also have metal objects, wooden objects, big wooden objects, small wooden objects, grass — I’ve only stored about 100 sounds but there are a couple hundred thousand possibilities that could be generated,” says Aitken. He notes that he’s done that process for nearly all the sounds in Wave Champions, except for the engines. 

Engine sounds require long sound samples, and therefore, are big files. If Wave Champions were a console game, Aitken would consider using a particle-based, granular synthesis system like REV from Crankcase to reproduce the engine sounds instead of relying on sound samples. “Phones just aren’t powerful enough. Also, we don’t have the budget like we would on an AAA game title at this moment,” states Aitken.  

For loudness metering on mobile games, Aitken uses Nugen Audio’s VisLM. He follows the Sony non-mandatory guideline of -16dB, which he notes, is the mastering level used for iTunes. Loudness is a big issue for mobile games, says Aitken, “If you want to start creating quality games for mobile devices than the sound mandate has to sit within tolerable, professional levels. If the sound is distorted then it sounds amateur.” He explains that every element of a game adds two percent to the overall picture. 

“Not having your graphics slightly anti-aliasing, making sure that when you collide with something you stop, making sure when you hit something the sound isn’t late, making sure the buttons feel like they’re pressing when you press them. Those things on their own won’t make or break a game, but your whole team has to add that two percent to go from an average game, 50 percent, to 80 percent at least. If I can add, with the total audio budget, an additional six percent to the score, then that’s great. That’s fantastic. We can go from a B to an A-, on just the audio alone.”  

Out There: Omega Edition

Award-winning composer Siddhartha Barnhoorn, located in Katwijk aan Zee, Netherlands (www.sidbarnhoorn.com), scored over 70 films, and has a growing list of game credits, including the score for the highly-acclaimed, first-person puzzler Antichamber, developed by Alexander Bruce, and the well-received strategy game Out There by Mi-Clos Studio. Out There was Barnhoorn’s first experience creating sound design for a mobile game, since the developers desired him to handle both sound design and music. Creatively, Barnhoorn wanted the sound design integrated into the music, “so the sound design wasn’t just random sounds,” he explains. “I wanted to create a flow between the soundtrack and the sound design elements.”



Out There is a non-combat space strategy game that requires players to explore the far reaches of the universe, find and manage resources, maintain their ship, and survive the harsh unknown. It’s available for both iOS and Android devices. Mi-Clos Studio is currently perfecting Out There: Omega Edition, a larger, multi-platform version of the mobile game that will be available on PC, Mac, Linux, and Android and iOS. Out There: Omega Edition, currently in beta, will feature new alien breeds, new space ships, new interactive stories, uncharted planets and environments, as well as new endings. 

The Omega Edition soundtrack will feature 20 new, shorter music tracks. While the original Out There game used five music tracks, with the main gameplay track being close to nine minutes long, the shorter music tracks in Omega Edition will act as introductory music for each different type of star a player discovers. Barnhoorn explains, “The main focus for Omega Edition was the music changes with every new star you visit. We have yellow dwarfs, white dwarfs, red dwarfs, supernova, neutron stars — which are very dangerous and have a lot of power, red giants, blue giants, and also black holes, but those are dangerous places as well.” Barnhoorn adds that once a star’s introductory music is complete, it will transition into an ambient background track, and for garden planets that sustain life, there will be a few special ambient tracks that feature ethereal vocals. “It sounds like a siren singing, a subtle, strange, yet beautiful ambience with vocal elements,” he says. The vocals are performed by Barnhoorn’s girlfriend, vocalist Lara Ausensi.

To create the music tracks, Barnhoorn used a combination of analog synth sounds from the Korg Minitron and software synths by Native Instruments, including Absynth, Massive and FM8. He also used library samples from The Unfinished by British composer/sound designer Matt Bowdler. “The Unfinished samples sound very amazing for the style that I was going for — that ambient spacey sound,” says Barnhoorn, who also recorded guitar parts he played using an EBow by Heet Sound Products. The EBow continuously vibrates a guitar string, and creates a long stream of sound. Barnhoorn uses Steinberg’s Cubase for recording, editing, sequencing, mixing and mastering. After mastering, he sends full mixed tracks (not stems) to Mi-Clos Studio for implementation. 

Barnhoorn wrote the original Out There score in D minor and created the sound design in D minor so the two elements would blend together. “When you press a button, for example, you’ll hear the sound design for it, but it’s in D minor, so it will match very nicely into the music,” explains Barnhoorn. He created button sound variations in D minor for different features on the galaxy map and solar system map. Other tonal effects include a transform sound for when players terraform a planet. “You shoot a beam at a planet and it changes the planet from a garden planet into a rocky planet with no life, or from a rocky planet into a garden planet,” explains Barnhoorn. “That sound had to be short, about six seconds, but it had to be interesting too. I experimented quite a lot with sound design elements I recorded myself, making many layers of many sounds to eventually create that one sound.”

Non-tonal effects, such as drilling or shooting out a probe, Barnhoorn created using a combination of library effects and sounds he recorded in the studio. Additionally, players can communicate with alien life, whose language is inspired by the Philippines national language Tagalog. Barnhoorn created a communication sound by recording the eeeps and urnts generated by a '90’s style modem and manipulating those using the plug-in Paul Stretch for pitch and time processing, and the Glitch plug-in by Illformed to generate a randomized, mutated sound from which he pulled interesting snippets. “Paul Stretch keeps the important elements of the sound intact. You can stretch from two minutes to 20 minutes and it will still sound good,” claims Barnhoorn.