THE THIRD FLOOR founder continues to champion virtual production and lead the charge into virtual reality.
Chris Edwards has been pushing the envelope in film production techniques for more than a decade. Since co-founding THE THIRD FLOOR in 2004, he’s helped the company become a primary force in propelling previs from little-known technique to essential component in major visual effects movie production. All services considered, from pitch- through to postvis, TTF now estimates it is involved in 75% of all Hollywood feature productions. But Edwards is only just getting started. He is currently beating the drum for virtual production - prebuilt computer graphics taken on set to interact with the live shoot in real-time - aiming to elevate it to the same level of ubiquity as previs. Last year, he co-founded the Virtual Reality Company, a new entity harnessing art, storytelling and technology to explore the possibilities of VR.
In early December, I sat down with Edwards at the Advanced Imaging Centre For Future Visual Entertainment (AICFVE) conference in Beijing. In Part 1 of our conversation, Edwards outlines his thoughts on market and artist development in China. Here in Part 2, he talks about his vision for virtual production and virtual reality in global content creation.
Chris Colman: You’ve described the complete production workflow that THE THIRD FLOOR offers as 'the holodeck.’ What’s the biggest problem you solve for directors and filmmakers?
Chris Edwards: We are the medium by which directors can manipulate their vision. A lot of techniques have been used by directors to set their minds on a particular idea. Alfred Hitchcock and other Hollywood filmmakers used extensive storyboard artists and concept art that were all static. It didn't take them long to realize that once you could use a camera to play with the position of the staging or the shot, even if it was a modeled miniature mock-up of the set, that was a very useful process because you could kind of discover it instead of just drawing one iteration after another. What we do now is just the super-advanced version of that, involving all of the other disciplines in this world, too. There’s a real revolution that computer graphics and the workflow of previs allows - the production design can mesh with the staging of the action, to identify that a set needs to be bigger, a color needs to be changed, a shot could be better if you just start here instead of here. All of these things empower a director to walk on the set like a total rock star, knowing the future. They know they've tried it this way, they've tried it that way, but this is the sweet spot.
CC: Previs has become widely adopted now, at least in Hollywood. What about virtual production? Are DPs and directors familiar it? Do they understand how to use it?
CE: Virtual production is the next frontier. It's my new challenge, to get the same level of acceptance that we have for previs techniques, but expand it on set. This is one of the most misunderstood areas. It really is the collision point of all of the traditional disciplines. It's bringing computer graphics on set. Does that mean the visual effects company controls that? Is that a job for pre-production personnel? Honestly, the answer, I believe, is it's not really the job for anyone; it's the job for all of us. That requires a lot of communication, a lot of standardization of the process so that we can create, like you say, 'the holodeck' experience, but do it in a way where it's not for technology purposes, but for enhancing the creativity of the moment. Filmmakers and DPs don't know exactly what the mood is going to be until they strike the first light, until they place the camera and see how it’s responding through the lens that they've chosen. It can be difficult to think so far in advance in previs. The solution is to do some work in previs, but then bring that data onto the set, make it work with the live-action performers, with the actual set that have been built and be able to direct both components simultaneously. This saves you so much time in post-production, and makes the film better. But you have to do it so fast that it keeps pace with the speed of traditional filmmaking.
CC: How are you spreading the word and helping people start understanding virtual production?
CE: The best way to promote virtual production is to have people try it. There are a lot of bad examples of virtual production that turn off world-class DPs who say it's too clunky, it doesn't make sense, it doesn't look like real lighting. We're working on an initiative now where we're building a much more streamlined workflow, using game engines and the latest technology to create this ease of data flow. You may be able to manipulate the set as fast, if not faster, than what someone could do physically, by actually moving those objects into place. That's something that applies not just to film and television, but to all forms of media creation. It’s also the steppingstone to making great VR.
CC: What are the challenges you face in trying to reach the speed and efficiency you need?
CE: There's a big disconnect between the technologists who think that they've solved this equation and are enamored with their technology, and the traditional craftspeople that are not understood, except by their peers. The challenge really is a matter of communication, of understanding not only what's happening in the new digital world, but also how the digital folks understand the traditional artists. We're working towards bridging that. A lot of what a previs team does is to help connect everybody on the team and really seek to be brought in to the inner circle of their process. I think as this becomes more widely adopted people will ask how they could have ever done it any other way.
CC: We're still using 2D interfaces and old school techniques in virtual production when most agree we need to start creating inside virtual reality. Where are we in that process?
CE: It’s a moving target but we're excited about making the game engine the future of all content creation. The Epic Games downloads using the Oculus Touch controllers are a harbinger of the future - manipulating the contents of a game engine in real time so you actually feel like you're there. You can pick up an object, move it, duplicate it, using the interface inside of VR. The technology of VR should be used as the content creation tools for the professionals whether or not they're creating a virtual reality experience. Use VR to create a movie by previsualizing the visual effects and the set in a way where you feel like it actually exists for real, before you have to commit to it. That's the brilliance of previs and it's the brilliance of what virtual reality enhanced previs can be.
CC: When did virtual reality start to enter your thinking?
CE: Well you know VR was a bit of a surprise for all of us, right? We knew that VR technology was out there, but it wasn't really widely used because we didn't have the level of computing power to really make it convincing, real enough or real-time. For me, the example set forth by Facebook's purchase of Oculus for around two billion dollars was such a shot across the bow, a harbinger of how much value there could be in VR and AR. It made everyone wake up and look at the potential. We were quick to put the pieces together, in that, “Hey, we've been doing real-time content creation for over a decade”, while the rest of Hollywood has gone a lot less real time and worked on massive cloud rendering that takes a long time to process images, not 90 frames per second, created in real time per eye. I felt well positioned to help traditional content creators be the best that they can be in VR, to understand the limitations and the advantages. Our role and our goal are effectively to be this place where people can explore and prototype their VR experiences, if not finish them.
CC: What excites you about VR?
CE: I’m excited about the transmedia future of it all. How we can take one IP and simultaneously build it as a film and virtual reality experience and a theme park attraction at the same time, using the same crew. Think about how efficiently that can be done. The director actually working with all of these different moving parts, but each product is supportive of the other. People have been talking about transmedia for a long time, but they haven't really been able to achieve it in most cases because the technology wasn't supportive of transferring data from one format to the next. Now I think there are more universal standards, databases, and systems that allow us to do this. Now that everything is converging around real-time creation and the ability to be immersed in a real-time world, filmmakers and video game makers alike can unite around this VR revolution.
CC: What is the makeup of the Virtual Reality Company and the VR department at THE THIRD FLOOR?
CE: I’m a founder of VRC, along with Robert Stromberg, and Steven Spielberg has come on as a collaborator on a new project being announced soon. At THE THIRD FLOOR, we have our own purpose-built VR department made up of game engineers and VR designers with years of VR/AR experience and multi-platform game engine expertise, including talent behind many AAA+ games and VR titles. We're excited to have worked on projects like The Martian VR Experience and VR experiences upcoming for some major filmmakers.
CC: VR content is still limited in quality. How are you approaching content creation?
CE: The best way to stand out in the crowd is to do something new that people haven't seen before. Certainly that's the approach of VRC, to be the first of a new genre. For example, we worked with Lytro, a light field camera company that's innovating ahead of the curve in creating a volumetric capture, a way to capture all the data so that you can later manipulate it from any angle. We found a kinship with them and helped them shoot their first test of that unbelievable technology. We're innovating with the folks at Epic Games, the creators of the Unreal Engine. They are very supportive of our projects and we're super encouraging of them to push the boundaries of what's possible in the game engine. We believe that the game engine is the future sound stage. In some cases it's going to replace the need to build large physical sets and allow you to manipulate not only the CG components, but also the live-action elements, all in real-time. This future is not science fiction, this is happening, but it's gonna take a team effort to realize it.
CC: How did The Martian VR Experience come together?
CE: We worked with Fox Innovation Lab, a unit within 20th Century Fox in Hollywood. They are out in the lead basically saying, “We're committed to VR, we're going to take our biggest IPs and make a completely custom VR experience that brings fans beyond the screen and into the world they love.” That’s what we did for The Martian VR Experience, which was directed by Robert Stromberg. We’re very proud of the detail that we added to the project, and publishing that experience on all three of the major tethered platforms - Oculus, Vive and PlayStation VR. We've learned so much that now we're able to take that technology and re-scan the avatar so you could be Mickey Mouse, Shrek or any other character and you'd have the ability to use your hands to manipulate objects and interact with the world. That capability is being added to every day by our R&D unit. Essentially, we bring in new creators, we show them what we've done with The Martian and then they understand, “Oh that's what's possible, but can I do this?” We'll maybe do a technical test and find a way to prototype that tech. Marrying that with amazing artistry and a good game plan are the components you need to create great VR. The jury's still out on whether or not this is going to be a thing, but we're certainly believers.
Chris is a writer & producer based in Shanghai. He’s the founder of the China Animation & Game Network, encouraging communication in the industry via live creative networking events.
