The Lion King uses a game engine platform to emulate live action film production in a VR space, a real learning process that allowed the team to create a suite of tools now available for all filmmakers to try.

Following the Big Screen’s ASC 100th session with the production’s cinematographer Caleb Deschanel and VFX supervisor Robert Legato earlier on September 14^th, attendees of IBC 2019 have an opportunity to see the technologically ground-breaking movie in IBC’s state-of-the-art Big Screen cinema, at 18:35.

The original Disney film, The Lion King, from 1994, left its mark, and this new version tries to recreate the original. Ratings at Rotten Tomatoes are low, and according to the website, the critics consensus is that “While it can take pride in its visual achievements,The Lion King is a by-the-numbers retelling that lacks the energy and heart that made the original so beloved–though for some fans that may just be enough.”

I have not seen the film, which aims to bring the beloved characters from a Disney classic back to the screen, but browsing the web, reviews praise the visual effects while being critical of the lack of originality and facial emotion of the characters. Some of the harsh critics may be the result of a generation and a time that tends to be very critical of everything. It’s not a discussion I want to have here, though, as I am more interested in the production process, or Virtual Production process used for The Lion King.

The Lion King comes after the success, both among the critics and commercial, of The Jungle Book, which used virtual reality tools representing the early stages of the tools now used in The Lion King. Much of the production process used in The Jungle Book is present in the new film, and the team involved have learned a whole lot of new tricks, that led to the choices made in The Lion King. Critics consensus at Rotten Tomatoes considered that “As lovely to behold as it is engrossing to watch, The Jungle Book is the rare remake that actually improves upon its predecessors — all while setting a new standard for CGI.” From where I see it, The Lion King takes the standard further.

Making a film with a game engine, and in VR space

Technicolor’s Moving Picture Company (MPC), that created, with Weta Digital, the visual effects for The Jungle Book, and developed new software for animating muscular structure in the animals, helped build the tools for virtual production in The Lion King, paving the way for the production of a film that represents a revolution in filming and opens new horizons.

Virtual Production is the keyword here, and Jon Favreau, the cinematographer behind both The Jungle Book and The Lion King, said that he worked in partnership with MPC and their visual effects supervisor Adam Valdez, to “build the tools for virtual production, which is a technique that we innovated for this particular project, along with a company called Magnopus. And these new consumer-facing VR hardware products led us to the opportunity of using a gaming platform to emulate live action film production in a VR space – even though the film is completely digitally rendered, every environment is made digitally by the artists at MPC, and every character is keyframe animated.”

Put the camera where you want

“The time was ripe to do this, but no one was doing it,” says three-time Oscar-winner Robert Legato, Production VFX Supervisor. “This was a push to make a film that broke the conventions …and if the result happens to be good, you can see how you can apply it to other things, and that’s not as common as you’d think. Everybody does VFX movies, everybody does animated movies, everybody does live-action movies—but to mix all of them together to make something that belies how it was done is, I think, the game-changing portion of all this.”

That’s how The Lion King was created. Rob Legato, who reteamed with director Jon Favreau, MPC, and other filmmakers from The Jungle Book to bring the beloved characters from another Disney classic back to the screen – in a whole new way, said this about the process: “the VR component we introduced after The Jungle Book allows you to walk in like you would on a real set, put the camera where you want, and the physicality of it helps you to psychologically root yourself. You start doing very natural camera work because you’re in an environment that you’re familiar with. You know where the light goes, you know where the camera goes, you know where the actors are – and all of a sudden on day one you’re off to the races.”

Films are following the path opened by games

The team is exploring new ground, no doubt, but this is something that game creators have done since the first 3D games made it to the market. Game creators have the advantage of being able to position their cameras anywhere, and when games started to build narrative sequences that looked like scenes from a Hollywood  blockbuster, film directors became jealous. Some even associated their name with games, to be able to experience the other side: from Steven Spielberg, with The Dig, published by LucasArts in 1995, to Peter Jackson with Ubisoft’s King Kong adaptation, not forgetting Clive Barker (with Clive Barker’s Undying) or George Lucas (The Force Unleashed), there is a long list, that also includes directors who have made trailers for games: George A. Romero, John McTiernan, Joseph Kosinski, Michael Bay, Rupert Sanders Brett Ratner or Tim Miller, to mention  some.

Cameras in video games are a film director’s dream. As Paolo Burelli, Assistant Professor at the Center for Computer Games Research, IT University of Copenhagen, states in “Game Cinematography: from Camera Control to Player Emotions”,  one of the most clear distinctions between a virtual and a real-world camera is that “a virtual camera is an abstract construct that defines the way the virtual world is presented to the user; it is designed to simulate the behaviour of a real-world camera and, at the current state-of-the-art in computer graphics, images produced through a virtual camera can be indistinguishable from real footage. However, contrary to a real-world camera, a virtual camera has no physical presence in the virtual world and its properties can change over time and adapt to the events filmed (Haigh-Hutchinson, 2009).

Virtual cameras and game cinematography

These advantages, continues Paolo Burelli, “allow virtual cameras to film scenes with much higher freedom and expressiveness; however, this can potentially contrast with the filming conventions developed in traditional cinematography. Such conventions describe aspects like the way the camera should be placed or the way it should be moved to make a transition between two different scenes, and adherence to these conventions is often important to generate a cinematographic experience and not to disorient the viewer – e.g. by crossing the line of action (Arijon, 1991).

The author also points to the fact that “the relationship between game cinematography and its traditional counterpart is extremely tight as, in both cases, the aim of cinematography is to control the viewer’s perspective and affect his or her perception of the events represented” stating that “however, game events are not necessarily pre-scripted and player interaction has a major role on the quality of a game experience; therefore, the role of the camera and the challenges connected to it are different in game cinematography as the virtual camera has to both dynamically react to unexpected events to correctly convey the game story and take into consideration player actions and desires to support their interaction with the virtual world.”

VR: from games and experiences to cinema’s Virtual Production

Virtual Reality, first associated with “experiences” and video games, appears here as the glue that gets everything in The Lion King to work. For some time now, we’ve seen VR associated with film festivals as Sundance , Tribeca or SXSW, and the creation of projects as Oculus Story Studio (now closed), that introduced the concept of VR film-making. As technology evolved, we’ve also seen new titles appear, as Age of Sail, Wolves in the Walls and others that ProVideo Coalition has mentioned in recent months. Virtual Reality is exploding, with the profusion of headsets, and the release of more new titles, from documentaries to animation, that mix every tool available from game engines, to cinematography techniques and the evolving technology associated with Virtual Reality, to confirm that this is not the trend that kills everything else, but another way to watch games, “experiences”, movies and, now, also to enhance the production of movies as The Lion King.

In that sense, The Lion King represents a milestone. As Rob Legato said, “when we’re in VR, which is different from what we did on The Jungle Book, the VR experience gives you the visceral feeling of being there, of being up high or at a faraway distance, all those various things, you feel it. So Jon, Caleb, James Chinlund, Andy Jones and I could all walk in together as a group of filmmakers, a group of collaborators, and see things for the first time, look at things from different angles. Caleb might have a better idea to shoot something to take advantage of a sunset that’s happening, and again it’s like music, where you’re bouncing off of each other. And when you’re done it’s a very successful location scout that otherwise you could never have done together in a CG world. But in the real world, or in our case a virtual world, we all can share in it and we all can respond to what we’re looking at – at the same time.”

The knowledge acquired through the experiences of The Jungle Book and The Lion King allowed MPC to refine constantly the tools used for Virtual Production and now, elaborates Favreau, “MPC has a suite of tools that are available to any filmmaker based on the innovations that we made on The Lion King.”

The words from Rob Legato, Jon Favreau,  Adam Valdez and other members of the team give us a glimpse of the whole process used for The Lion King. To read the whole story, with much more information, visit the webpage at Technicolor dedicated to The Lion King.