Iain Blair
Issue: November 1, 2007


CULVER CITY - Based on the English epic poem of the 8th Century, and with a script by graphic novelist Neil Gaiman (Stardust) and Roger Avary (Pulp Fiction), the new Robert Zemeckis film version of Beowulf doesn’t pull any punches in terms of gore and violence. It also brings the ancient tale of the mighty warrior Beowulf (Ray Winstone), who slays the demon Grendel and incurs the wrath of its monstrous-yet-seductive mother (Angelina Jolie), firmly into the modern world of digital cinema.

Here, visual effects supervisor and Sony Pictures Imageworks staffer Jerome Chen, who previously collaborated with Zemeckis on The Polar Express and who earned his first Academy Award nomination for the groundbreaking visual effects in Stuart Little, talks about the challenges of making Beowulf and pushing the digital envelope even further than they did on The Polar Express.

POST: What were the biggest challenges of making the film?

JEROME CHEN: “The film’s sheer scope in terms of the 6th Century fantasy-oriented environments that we needed to make, including castles, caves, the ocean. Beyond that, we also had to create very realistic humans and creatures, including a dragon — which in itself is tough as there have been so many in film and audience expectations are so high — and the Grendel creature. That had to be ferocious but also emotionally vulnerable and able to elicit sympathy from the audience, so we had a lot to contend with.”

POST:: Where did you do all the visual effects, and what tools did you use?

CHEN: “It was all done at Imageworks with a team of between 400 and 500, and I’ve been on it for three years now, so it’s been huge — the biggest job I’ve ever done. We used [Autodesk] Maya as the backbone of our animation pipeline, and [Side Effects] Houdini as our effects pipeline. We rendered in Pixar RenderMan and composited in our in-house software Katana and Bonzai. Katana is also our RenderMan interface: it’s a lighting/compositing interface.

“It’s odd talking about visual effects because the film is totally CG. It’s really a hybrid animation/visual effects film with a lot of live-action components because we deal with actors in terms of capturing their performance. The whole front end of the production process in terms of designing sets, costumes and the motion capture all feels very live action, and that’s why I love these projects because they’re these hybrids that give you the best of both worlds. You still interact with the live action, then in post the whole thing is a visual effects film. The only difference is you’re doing everything in the film as opposed to a select group of shots that enhance the live-action photography. So here, you’re not only lighting the humans and creatures in CG, you’re also compositing them onto a CG-created environment.”

POST: How big a step forward was this from Polar Express, or was it more of a refinement of those techniques?

CHEN: “I’d definitely call it a leap forward, in both technology and creativity, which work hand in hand. The more advanced the technology we developed, the more time we had to work on the creative aspects of creating the imagery. What I’ve found in most visual effects work is, if you spend, say, 75 percent of your time trying to work out how to do it, you only get 25 percent of the time to make it look good. But here, it was more a 50/50 split, and it shows.”

POST: So what new technology enabled you to also raise the creative bar?

CHEN: “We used Imageworks’ patented Imagemotion performance capture system to bring it all to life. The advancement of the motion capture volume, in terms of having more motion capture cameras and higher quality ones, so we could get more of the facial and body movements, was crucial. That capturing part is the equivalent of shooting negative, except it’s negative that can only pick up human motion and nothing else. And pre-production is the scanning of the actors into the computer and the whole modeling of costumes and so on.”

POST:: How did the pipeline work once that part was complete?

CHEN: “The post process involves taking all that movement data, cleaning it up and applying it to the virtual skeletons of the characters. We had to create a library of CG characters that were affordable in terms of being able to make enough high-quality humans to fill the movie, as in some scenes there are over 100 characters in a hall or a crowd. We didn’t have enough time to make 100 different people, so we made about 24 male and female bodies, and from those created variations of costumes and hairdos, which were extremely complex and time-consuming to do.

“The next step was creating all the CGI environments and sets, and there were numerous rooms in the castle, a forest, a beach, countryside, a cave and so on. All those had to be built and then texture-painted. Then we had all the creatures in addition to Grendel and the dragon, including horses and dogs, which was kind of like building miniatures. Next, you get into the process of dealing with the motion capture elements, and while you’re capturing actors’ movements, you’re not actually creating a camera POV yet. All you do is record human data moving — you don’t have the cinematic POV. So there’s an editorial process that happens, and Bob Zemeckis will select the performance he likes and then we’d process that onto the CG characters. And for any given scene, we’d take the characters and place them into the set they’re meant to be in. So you need very careful records when you do motion capture. Once that’s done, we turn it over to layout — the equivalent of the camera department, and they start creating the cinematic POVs Bob wants. The interesting thing is, performance is locked and now you shoot your coverage.”

POST: What technology was used for this?

CHEN: “We used the realtime capabilities of game technology to help us visualize this part, specifically the realtime rendering engine inside [Autodesk] MotionBuilder.  MotionBuilder has better realtime capabilities than Maya, and realtime is important since we have an in-house camera layout system that works in realtime, so that way we can use the skills of a real camera operator to shoot the characters.

“Rather than having an animator keyframe these cameras, we actually have the motion characters play back in realtime, the camera operator looks at them on a screen and basically pans and tilts to the characters walking across the screen, and we record that live. So you get all the nuances of a cameraman’s style, and it doesn’t feel too keyframed. And it worked well for giving the film fluid human movements.

“That’s the first pass of performance integration onto the CG character and camera layout; it’s what the editor edits with. Now he has a whole bunch of shots to make his cut with, and after these camera layout shots are created and the editor cuts together a sequence, then we start, in some sense, a more traditional filmmaking process. The big difference is if we decide, for instance, to add a tighter reaction shot, the editor can then do it immediately. There’s no compromise, or ‘we missed that on the day.’ The performance is exactly the same, as you’ve locked off on it, and that’s part of the flexibility Bob likes about the performance capture process, because he gets the best of both worlds. He gets the actors’ performances, and then has the strength of the virtual technology to make the best film he can make.”

POST: Once you have a cut of the film, how do you deal with the more traditional aspects of post production?

CHEN: “We take the characters and start doing digital cloth and hair simulations on them, and since that’s the time-consuming part, we do all that after we have a cut put together. So you know exactly how long a shot will be and how many people will be in it. For cloth and hair simulation, we used the Maya cloth simulators as the backbone of our system, with a customized toolset that we built here. Then we have our own in-house hair styling system that’s then rendered in RenderMan. And as all the hair and cloth are being done, in parallel you have all the visual effects shots.

“So if a scene calls for snow or fire or debris, then we use the Houdini effects pipeline to create the elements that are also then rendered in RenderMan. On top of that, we spent almost 18 months writing customized RenderMan shaders to create the look of the human skin, eyes and clothes of the characters. Then digital lighting starts on a shot-by-shot basis once cloth and hair are done, and the lighting layers are rendered separately and then composited together. And then all the final color balancing and final lighting tweaks are done in the composite.”

POST: Beowulf seems far less stylized than The Polar Express?

CHEN: “Exactly. It had to have a component of realism and very dynamic performances from the actors. One of the biggest challenges was making sure that those performances translated. It’s not yet a perfect system. You don’t just turn the cameras on and their soul comes through automatically. Creating human characters that aren’t distracting is very, very hard. Even after doing this film I’d still consider that to be the hardest thing to do. We also had to tackle all the hard CG elements, like cloth, water, hair and so on.”

POST: After three years on this film, just how groundbreaking is Beowulf for you?

CHEN: “It definitely feels like we’re in the middle of some big kind of change that’s happening in the industry. It’s not a sweeping change, but it’s this new avenue that’s opening up, because as we’re working on the movie, we start speaking to other directors and writers and creatives who are becoming very interested in this whole process of performance capture and CG.

“Guys like Jim Cameron are using something similar on his new film Avatar, and other directors are exploring this kind of filmmaking process. The big appeal is that it has a live-action component to the process; it’s not just making an animated feature where the performances are storyboarded and created by animators. This is a different genre. My background is animation, but this won’t replace animation and it’s not intended to. It seems to be becoming its own medium, another avenue for creatives to tell a story in, and which requires all this new technology.”