This is the LED set used for The Mandalorian. ILM The interior of the set, as performers might see it. ILM Here's the set with a large prop in it—sometimes only the LED set was used, and sometimes it was used in tandem with practical elements. ILM The camera captures both physical objects and the virtual background directly. ILM This visualization shows virtual set elements alongside real actors and how they appear in a shot. ILM Scene assets can be removed or moved easily. ILM As seen in this screen capture from a promotional video by Epic Games, directors and DPs can do location scouting in VR. Epic Games Also from that video: you can see that the frustum for the camera is displayed on the panel in real-time in sync with camera orientation. Epic Games For some shots, the scenes and assets used in shooting may not be final. In those cases, a green screen can be displayed for use in post-production. Epic Games Lighting can be changed in real-time using an iPad. In many cases, the lights from the LEDs as configured here act as the final scene lighting. Epic Games Presets can also be set and near-instantly switched between for efficient shoots. Epic Games A stage director handles many of these changes on-set. Epic Games
Industrial Light and Magic has published a behind-the-scenes video on the production of Disney+'s The Mandalorian that gives an illuminating look at two of the biggest, high-tech trends in film and TV production: LED sets, and using game engines to create scenes. The video explains a major shift in virtual filmmaking that is unknown to most viewers.
It has historically been impractical to achieve the production values seen in The Mandalorian in TV series, because the kind of visual effects work necessary simply takes more time than a TV production schedule allows. Generally, special effects-driven productions shoot scenes with actors and props in front of a green screen, and then teams add in the background environments and any computer-generated objects in a lengthy post-production period.
That's not how things worked on The Mandalorian. Executive Producer Jon Favreau, Industrial Light and Magic, and game engine-maker Epic Games collaborated to use the Unreal Engine to pre-render scenes then display them as parallax images on giant LED walls and an LED ceiling in a 21-by-75-feet digital set. It's part of a lineage of production techniques and tools developed by Favreau's teams called StageCraft. This approach offered numerous benefits.
First off, actors could see virtual objects and environments around them in real-time, including horizon lines. This solves a long-standing problem with VFX-heavy productions and actors' difficulties getting in the scene or responding realistically to objects or sights in it. Assets could be changed on the fly as requested by the director or director of photography (DP). So, if the director decides that a certain building in the background is messing with the framing or otherwise detracting from their vision, they could request that the stage operator move the building in just a few seconds.
Additionally to that point, real-time lighting is provided by the LED panels, so lighting can be changed with a simple iPad interface without requiring long periods of manually moving physical lights. (This task takes an enormous amount of time out of each day during traditional shoots.) Entire sets can be unloaded and replaced with totally new ones in a matter of minutes, provided there is also not a heavy use of practical effects in tandem—something completely impossible in traditional filmmaking.
All of this allows the director, crew, and creatives to be more flexible in production, try different approaches, and ultimately, avoid hundreds or even thousands of hours of revisions in post-production. It even extends to pre-production: the toolkit allows scoping out shots and location scouting in virtual reality before shooting begins. And it lets producers avoid the enormous cost of shipping entire productions, crews, and casts to far-flung deserts, forests, tundras, or what have you for location shooting.
Favreau feRead More – Source
[contf] [contfnew] 
arstechnica
[contfnewc] [contfnewc]
