What is the best kind of storage? The kind you don’t have to worry about. That is, across the board, the consensus from visual effects pros.
With 2K, 4K and stereo projects coming into studios, storage is more important than ever, and having a system that is fast, flexible and able to grow with artists’ needs hits home. Also important? Not needing a small electrical plant to power the system or having to break the bank to keep things cool.
THE CREATIVE-CARTEL
The Creative-Cartel (www.thecreative-cartel.com) in Culver City manages projects from camera to DI, providing everything from engineering digital pipelines to near-set lab and dailies services on location to VFX and stereo production management. Their resume includes Priest, Ted and most recently, After Earth.
Storage is incredibly important to every aspect of what they do, and how they do it, and they are all about efficiency and streamlining the workflow. So much in fact that they created their own management system called Joust, which acts as a repository for all digital media and metadata during principal photography, including data wrangling, script and camera notes, as well as pertinent color information for each shot. After the data is collected, Joust becomes a dailies and vendor review system, with the ability to create bid packages, watermark images and automate vendor submissions. Plus, it allows editorial to manage plate pulls and transcoding so that VFX plates are delivered the same day.
The Creative-Cartel's Craig Mumma and Jenny Fulle on the After Earth set.
The Creative-Cartel has been using JMR tools for about three years. “Before we started using Joust we used their tools for our own infrastructure needs,” reports CEO Jenny Fulle. “We use the Bluestor and their servers. We will call on JMR for pretty much all of our hardware needs at this point.”
The relationship began when the studio was looking for technology to do stereo playback at 4K for reviews. CTO Craig Mumma approached a couple of companies to see if they could help, but “they all said they didn’t have the architecture and hardware to do it.” Then he tried JMR. A week after the request, The Creative-Cartel had a box over at The Amazing Spider-Man for testing. “They pushed the limit on the playback speed of dual 4K 3D, which was amazing from a single box.” In addition to the technology, Mumma appreciates their customer service: “You get directly to the heads of the company at any time. That’s important in our industry because the speed of the production is ridiculous and you need answers right away.”
The Creative-Cartel’s most recent job was the Sony F65-shot After Earth, which brought its own set of workflow concerns. “When we started, we were pioneering the workflow for F65. The last thing we wanted to worry about was our storage solutions because we had to worry about cameras more than anything,” explains Mumma. “We went to JMR and said we need to have enough storage for all the camera files, and we are going to be traveling and need robust equipment that will last for all the different areas.” The production brought them to Costa Rica, Pennsylvania, Eureka, CA, and Maob, UT.
After Earth was the first film in which The Creative-Cartel used Joust almost as a complete package. “That meant keeping all those original Raw files live and online, which was a big deal,” says Fulle. They had 150TBs live rolling with them from town to town.
The Creative-Cartel provided production management for After Earth and acted as the hub for eight or nine visual effects companies, which produced about 700 shots. “We started at camera and did the mobile lab,” explains Fulle. “So we processed all the dailies and kept all the files online, and once we started engaging with the vendors, Joust did all of the transcoding from Raw files to Open EXR, which was the format we worked in. We then managed all the digital images — moving them between the vendors and bringing them back in, showing them to the director, getting them to the DI house. After we finished the dailies it was all about managing the visual effects workflow. We were able to do light grading on the Baselight Transfer Station for visual effects stuff too. It was a really robust pipeline we worked out between Joust and the equipment we had on hand.”
Mumma points out how important power and cooling are to a drive’s efficiency, especially considering how much traveling they had to do on After Earth. “That is important to consider when you are moving around with these drives. The JMR [drives] have a low power requirement, so you don’t have to build a power plant to get these things up and going. We don’t have to have big, special rooms. Now we can set them up in a hotel room with basic cooling and power.”
Fulle gives an example of when production took them to a remote location in Costa Rica. “They set us up in a hotel, and by hotel I mean in the middle of the jungle with bugs and lights flickering. We had all of our equipment set up and we had ran out of outlets for all the gear. Craig was able to take a sconce off a wall, pull the wires out and wire up a plug. Five years ago you couldn’t have done something like that because you would have set the place on fire.”
There has never been a better time to take advantage of storage. “It’s not as cost prohibitive anymore,” concludes Fulle (@JennyFulle). “One hundred terabytes is not going to break the bank, so you can keep it live and online and save all of the days that were wasted before and put them back into the hands of the artists.”
SAVAGE VISUAL EFFECTS
Six-year-old Savage Visual Effects (www.savagevisualeffects.com) focuses on film, television and spot work, with studios in Los Angeles and Pittsburgh. While they might not be a huge company, the do work with some big names, such as directors David Fincher (The Social Network, Girl With the Dragon Tattoo, House of Cards), Louis Leterrier (Now You See Me) and Bryan Singer (Valkyrie).
Currently Savage has a core staff and builds up as needed, but they are currently moving toward staffing a bigger office in Pittsburgh with a larger, full-time staff.
The studio’s ties to Pittsburgh begin with co-owner James Pastorius, who grew up and went to school in the city; he has many contacts and artists to call on.
Savage typically calls on Shake and Nuke for their compositing needs, and relies heavily on various 3D applications and RenderMan, but they will call on other software packages as needed depending on what freelancers they bring on for certain jobs.
In terms of hardware, co-owner Brice Liesveld (@Brice) recognizes that having the right kind of storage is a hugely important. “It’s all about efficiency,” he says. “We are constantly needing more storage as things evolve, and if you can’t access data in realtime your efficiency just drops through the floor. It’s really important for everyone to have access to what they need without the hiccup of having to go pull something online.”
The need for storage never seems to end, especially with new technologies and the growing prevalence of 4K and beyond. “We aren’t Weta by any means, but we chew through a lot of data, even at our size,” explains Liesveld. “It was 2K, now it’s 4K and 5K, and next it will be 6K and 8K. You have multi-layered files like EXR, deep compositing, which is file-size intensive, and newer cameras generating more and more metadata. Everyday there is another chunk of data you need to store and access. Without having reliable, consistent and large enough storage, you can’t do your job efficiently.”
In order to get a system that worked for them, Savage contacted Venice, CA’s Open Drives, which offers a scalable and easy-to-manage data storage platform built specifically for the media and entertainment industry.
“Jeff Brue at Open Drives based the system on hardware from SuperMicro and he uses OpenIndiana, which is an open source operating system, which he has fine tuned for film and media,” reports Liesveld. “Our current Open Drives system gives us 50TBs of live storage with the capacity to expand to approximately 150TBs with the purchase of additional disks.”
Savage’s production set-up offers 50TBs of SAS disks that sit behind 960GBs of L2Arc cache, which Liesveld describes as an SSD RAID, offering very fast read/write capabilities. “Elements and plates that are used frequently are automatically pushed to the L2Arc cache, and are then served off of those faster disks instead of relying on the slower SAS pool.”
The Open Drives platform uses RAID-Z, which Liesveld likens to RAID-5 but it manages file space better and is self-healing. “ZFS also gives us hot-swap capabilities for back-ups so I could create a data pool of disks, push project data to it and then pull those drives out for archive instead of having to go to LTO-3 or a FireWire-type back-up solution.”
The studio has 10GigE Fibre connections to all its workstations. “That allows us to get realtime 2K stereo and 4K files directly from shared storage to the artist. It also gives us the luxury of working with full-resolution plates rather than introducing proxies into the workflow.”
While working on the first season of the Netflix’s series, House of Cards, Savage was able to keep the entire show, along with all related elements and reference footage, online and available to artists from start to finish. “Prior to teaming up with Open Drives we had to do a bit of digital juggling, archiving and restoring shots and assets to manage space.”
The VFX studio provided over 300 shots for Season One, including a CG library, greenscreen car shots, monitors, sky replacements and a variety of other invisible effects. “House of Cards was shot at 5K with the Red Epic camera, so considering the volume of work we had coming in, the ability to put together affordable and scalable storage was essential.”
Now that Season One is completed, Savage has moved the critical data to nearline storage, which is essentially the same as their 50TB set-up without the SSD cache in front of it. “We push recently-wrapped data to our nearline storage and let it sit for a while before it gets fully archived. That way it’s easy to access and if we need to get data back on the production server quickly, we can.”
Savage will be starting up on House of Cards’ second season this summer.
Sums up Liesveld, “The last thing you want to worry about is ‘do we have enough disk space,’ because then you can’t focus on the actual work.”
ELEMENT X
Dallas-based Element X Creative (www.elementxcreative.com) is a 25-person full-service visual effects, motion design and animation studio targeting commercial, television and film work. Recently they began creating their own animated properties.
“We focus on design and storytelling while investing heavily in our staff,” explains Element X (@xcreative) CEO/partner Chad Briggs, who believes in not getting too big and instead relying on a manageable core staff.
Briggs feels that not being tied to only one aspect of the work has given Element X Creative (EXC) what he calls a leg up. “I’ve never been a snob to one medium; I love visual storytelling of all kinds. Our guys tend to cross pollinate between visual effects, animation and graphics, and it’s that cross pollination that drives the design because we are not limited in how we think about a project — we can approach it from the best angle to tell the story.”
All of that couldn’t be accomplished without having a deep, fast and expandable storage platform. “Storage is the life blood,” says Briggs. “You have to have enough, and you always need more, especially these days with 2K, 4K and 4K stereoscopic. The demands of production on the visual effects side of things continues to get more extreme.”
EXC has had a long-standing relationship with EMC. “They have been rock solid, and the support is great,” he says.
The ability to add storage as needed, as well as having the ability to have a group of users connected to that storage, is hugely important to Briggs. In addition, the latest version of Isilon offers more Gigabit networking on each node, “so we have the ability to connect a lot more users in a centralized place.”
EXC has three EMC Isilons F200s that offer 6TB nodes — each one has four 1GB Ethernet ports and 6GBs of RAM for a total of 18TBs. Even with that amount they often hit those limits, so Briggs says the studio is currently looking into buying another node or two.
All that storage comes with the need for robust power and cooling. When EXC moved into its present location about four years ago they had to add a new electrical sub-system to accommodate. “It gets really hot,” he says. “Once you start throwing in Isilons nodes and back-up systems and renderfarms — we have about 35 to 40 machines dedicated to rendering, each running different softwares/engines — that gets power intensive. Then you have the cooling aspect, keeping it under 70 degrees.”
In terms of projects, Element X just completed a national spot for 7/11 out of the Integer Group promoting the convenience store’s July 11 free Slurpee giveaway. It features comedian Nathan Barnatt doing the “slurpee dance” in front of a greenscreen with retro ‘80s graphics behind him. The visuals were composited and animated at EXC. Charlieunformtango handled the edit.
Another job was for Occam Marketing client Leapfrog. EXC created a flurry of CG products and graphics for the piece.
Autodesk’s Softimage XSI is the studio’s primary 3D package. They also have Pixologic ZBrush and The Foundry’s Nuke, Nuke X and Nuke renderfarm nodes; a seat of Mari will soon be added.
For editing work they call on FCP 7 from time to time, but for the most part have switched to Adobe Premiere.