Ross Shain has been in the industry for more than 20 years, both on the creative post side and in product development. He once served as chief marketing officer at Boris FX, and is currently working with Topaz Labs, whose products are AI optimized.
“In some ways, the core IP of Topaz’s is training models,” he shares. “They have a team of PhD engineers. That’s what they do. They’re specialists in AI research, so they do their own training and create their own models.”
Topaz Labs offers a range of tools for video and photography that allow users to restore, enhance, upscale and perfect their imagery. Gigapixel, for example, is an AI-powered image upscaler that maximizes both the level of detail and the accuracy of an original image, increasing it in detail by as much as 6x. Documentarians can use the tool, for example, to improve the look of footage shot years ago, or taken from an archive. It runs locally or in the cloud, giving users the choice of whether or not they want to pay for rendering or handle it with their own hardware. The desktop option might appeal to those working on high-security projects that are restricted from using cloud services.
“There’s not going to be a licensing issue because Topaz, it’s their original models,” shares Shain. “One thing to understand is the difference between a GAN (Generative Adversarial Networks) model, which is a traditional AI upscaling (model), and this new diffusion kind of upscales. Topaz is really known for is quality and fidelity…If you’re a photographer and you have an image, all you want to do is basically upscale it — make it look better, improve the quality, denoise it — but you don’t want someone’s features to change. There’s a lot of different models that the company has that are tuned for different types of use cases, like, ‘This is shot with DV footage, or this is shot on film, or this has interlacing.’ That said, in the last year, the company has come out with some very interesting models that are diffusion-based that are very advanced and are literally taking the pixels, turning them into noise and then turning them back into pixels. These models understand that an eye is an eye and a nose is a nose. They’re called creative models, essentially, and there’s a precise setting which won’t really change the content.”
Shain says there is a new generation of content creators — Gen AI filmmakers — who are embracing AI tools to replace traditional camera-acquired footage or computer-generated footage. Topaz Labs’ tools can augment the work they are creating.
“Most of the models are running at 720p at the highest resolution,” he says of the Gen AI releases. “So then they take that into Topaz using one of these creative models to upscale it to 4K, and it just adds a lot more detail and actually makes it look like something you could watch on television.”
Shain credits visual effects studios as leading the charge in the use of AI tools, particularly for assistive AI in which repetitive, time-consuming tasks are becoming more automated, including paint, retouching, removing objects and cleaning up backgrounds.
“These are sort of traditional VFX processes that are not glamorous,” he explains. “They’re kind of like the invisible effects that are in almost every single feature film, not just the sci-fi and the fantasy, but literally everything. Many of those companies and artists are embracing these kinds of tools.”
In the past year, Shain has witnessed many professional workflows incorporate AI tools, pointing to Coca-Cola’s new holiday spot in which the beverage company’s trucks travel through a winter landscape, past an assortment of familiar animals, including their well-known polar bears. Secret Level built the entire spot using generative AI tools.
“(In the past) that was done with computer graphics,” he says of the Coke project. “That was done (with) animation, probably Maya. It was probably a very expensive ad that would have lots of special artists. This is original artwork created by artists, but they’re just using the tools to enhance it and bring it to life — animate it instead of using the more traditional CGI tools.”
Another recent spot for Jeep shows the evolution of the carmaker’s Cherokee model. 1986 Studios gave the Cherokee a makeover using numerous Generative AI imaging and video creation tools, including Midjourney, Veo3, Kling and Topaz Labs’ GigaPixel, Astra and Topaz Video.