The studio wrangles facial mocap performances into its proprietary suite of AI tools to tackle one of Marvel Studios’ most unique villains, a 5-foot-tall flying face, in ‘Ant-Man and The Wasp: Quantumania.’
With the arrival of Marvel Studios’ Ant-Man and The Wasp: Quantumania and the introduction of Phase Five, the MCU unveiled one of its most unusual villain yet with M.O.D.O.K., a five-foot-tall flying face, brought to life by award-winning VFX studio Digital Domain.
“Over our 30-year history, Digital Domain has created some of the most iconic CG characters ever seen on film, including Thanos, Benjamin Button, the Ready Player One cast and more, but this really is a first,” said Dave Hodgins, Digital Domain VFX supervisor. “Part of the fun of working with Marvel Studios is that they keep coming up with some of the most imaginative characters and environments, which allows us to create something that fans will remember.”
The character of M.O.D.O.K. – an acronym for “Mechanized Organism Designed Only for Killing” – debuted 50 years ago in Marvel Comics. Originally created in a lab by contemporary scientists, M.O.D.O.K. has been reimagined for the MCU and continues the story of the villainous Darren Cross (Corey Stoll) from 2015’s Ant-Man. The film explores what happened after he entered the mysterious Quantum Realm and was reborn in a new, monstrous form, leading the Digital Domain team to create a fully CG version of the character — driven by the actor’s performance.
Digital Domain began by capturing footage of Stoll and testing different looks, starting with various proportions. Multiple iterations were considered as the team sought a balance between a bizarre creature and a photoreal humanoid. The team even tested a version that included the character’s comic-accurate bowl haircut, along with other styles. Ultimately, Digital Domain and Marvel Studios decided on a version of the character that resembled Stoll’s face and hairstyle but at 5.72x the actual size, measuring 173 centimeters from head-to-toe and 124.77 centimeters from front to back.
To drive the performance, Stoll appeared in a capture volume wearing an HMC (head-mounted camera) and performed his scenes while watching the live-action footage from the set - the production used a large, gray 3D-printed cutout of his face as a stand-in. Stoll’s performance was then processed through Digital Domain’s proprietary Masquerade 2.0 facial capture system, which uses neural networks to map out the actor’s face and create working, photorealistic animation. Animation supervisor Frankie Stellato adds that regarding the use of data libraries for the facial AI work, “Only internal data created from the capture with the actor was used, along with some shapes created and enhanced by our facial team.”
According to Digital Domain CG supervisor Ron Miller, “For the last few years, artists at Digital Domain have utilized our proprietary tool Charlatan within our pipeline, which features an AI element. We leverage this technology either as a guide or as final output. Our Masquerade 2.0 system also contains a ML [machine learning] component that involves training a machine or computer to learn from data. This was developed and used on Thanos for Avengers: Infinity War.
Although Masquerade was able to generate a model that accurately conveyed Stoll’s performance, Digital Domain team still needed to add a few finishing touches by hand, including – and perhaps most importantly – the eyes. The VFX team studied the human eye extensively to determine the best way to expand it and retain a realistic look. Using Maya for design and Mari to texture, artists built the character’s eyes by hand, down to the tear ducts and folds within the eye. The team then used V-Ray 5's Random walk SSS skin shader, which maintains the base forms of the face and lets the fine pore details be more readable.
Asked whether or not the Charlatan and Masquerade system work is even doable without AI, Miller explains, “For face replacement, it is possible to recreate some of the work we did without using AI and ML tools, but it would take a much bigger team. We’d also have to bring in artists from every department to help match the likeness of the actor and keep it as accurate as possible for each shot. Doing it this way also creates more variables as well, which means the asset can fall apart. So even putting aside the amount of resources, it would be quite challenging to manage.”
But like any process, AI still needs human support to be successful, and there are still some limitations. Those walls, however, seem to be coming down faster and faster.
From there, the team still had to address the unique movement limitations of the character. During Stoll’s performance, the actor frequently used full body mannerisms to convey emphasis. The character of M.O.D.O.K., however, is a face held in place by a metallic chassis that constrains his jaw and limits any gesticulations. Using a custom deformer in Maya, animators removed any additional movements from Stoll’s footage while also altering his jaw to reflect the constraints of the character.
Finally, Digital Domain developed a new blood flow pipeline that leverages machine learning and creates a library of data from scanned data of Stoll. Once the team began working on the final version of M.O.D.O.K., the library of data was triggered by Digital Domain’s proprietary Weighted Pose Space Deformation (WPSD) system, which calculates edge strain differences to drive the maps so no animator input is necessary. The team capped it all off by adding a heartbeat monitor (that changes in the film as needed), a metal battle mask inspired by the character’s comic book roots, and a handful of futuristic weapons created to fit the specifications of the film.
Digital Domain’s work on Ant-Man and The Wasp: Quantumania marks the latest in a line of collaborations between the VFX studio and Marvel Studios that stretches back more than a decade. It includes blockbuster features such as Black Panther: Wakanda Forever, Doctor Strange in the Multiverse of Madness and Spider-Man: No Way Home, along with recent episodics including She-Hulk, Ms. Marvel and the upcoming Secret Invasion.
Dan Sarto is Publisher and Editor-in-Chief of Animation World Network.
