Our parent company, Jon Peddie Research, recently released a report, Digital Content Creation Report 2023, which has a significant amount of information on an overlooked driver of the display industry: the coming together of 3D and video production capabilities giving rise to incredible content creation tools that are accessible to a broad range of creators and can fire up amazing productions for gaming, TV, film, even DOOH, and retail. Virtual production lies at the heart of the opportunities in this space.
Virtual production is a rapidly evolving technique in the field of content creation, revolutionizing traditional workflows that used to rely on green screens and chromakeying. LED walls play a crucial role in virtual production, serving as the backdrop for the virtual sets. These walls display high-resolution imagery and video, creating realistic environments for the actors to interact with during filming. The quality of the LEDs used in the walls is essential to ensure a seamless integration between the real and virtual elements.
Unlike traditional workflows, where actors are filmed against green screens and backgrounds are added later in post-production, virtual production requires the development of backgrounds and effects beforehand. This means that writers and production teams must collaborate to design and create virtual environments, enabling script trials and faster development processes. Multiple camera positions can be planned and rehearsed in advance, resulting in more shots captured in a shorter time. Lighting setups can also be pre-planned, enhancing the overall efficiency of the production process.
Virtual production must consider the integration of visual effects (VFX) seamlessly. Care must be taken not to frame shots too tightly, allowing room for later addition of VFX elements. Collaboration and coordination between the virtual production and VFX teams are essential for successful implementation. Lighting design must also be well-integrated to maintain consistency and enhance the visual quality.
In this brave new world of digital content creation, access to virtual locations that may be dangerous or challenging to reach in reality is a cinch. For example, virtual production allows the creation of war zones or inaccessible areas like the rose garden in the White House. This flexibility expands the possibilities for storytelling and extends the range of environments that can be realistically depicted.
Lighting technicians, or gaffers, still play a vital role in virtual production. However, their responsibilities have evolved to accommodate the virtual environment. For instance, they can manipulate lighting conditions by adjusting the position or intensity of virtual light sources, providing greater control over visual aesthetics.
Virtual production offers significant advantages in terms of time and cost efficiency. While the setup of virtual sets may involve upfront costs, once established, they can be rented out for multiple productions. This often proves more economical compared to physical sets and the associated expenses of travel. Additionally, virtual production can de-risk projects and reduce costs by approximately 30% compared to traditional content creation methods.
Although virtual production allows for more efficient workflows, it still faces challenges when integrating with traditional postproduction tools such as Nuke. But, for our purposes, we are going to focus on the displays in virtual production where there are still a number of opportunities for growth and new technologies.
LED Panels for Virtual Sets
These LED wall panels have come a long way from their early versions used in conferences and presentations, with manufacturers now producing panels that offer finer pitch and greater pixel density, allowing for more detailed and immersive backgrounds. Several companies, such as Barco, Christie, and others, manufacture these LED panels and related equipment.
We are witnessing the increasing size and technological sophistication of virtual production stages, reflecting a growing trend in the film industry as more and more studios embrace virtual production technology for its cost-saving and creative benefits. One of the major innovations in virtual production has been the use of curved screens, which can provide a field of view of up to 90 degrees, adding to the immersive experience.
Amazon claims to possess the largest LED wall stage in the U.S., which is drum-shaped, 80 feet in diameter and 26 feet high. Sony has a stage in Kiyosumi Shirakawa Base studio that features a curved high-definition Crystal LED screen measuring 5 meters tall and 15 meters long. In California, they’re building a similar set at the School of Cinematic Arts at the University of Southern California.
LED Video Wall Technology
LED video walls have become an integral part of virtual production, displaying computer-generated graphics in real time during filming. This technology has been widely adopted due to its ability to replace traditional green screens, making film production more accessible during pandemic lockdowns.
Virtual production is not without its challenges. For instance, the LED backdrops cannot “rack focus” between characters and the background due to the Moiré effect. LED volumes also limit the kind of scenes and movements that can be filmed. Additionally, game engines used in virtual production may suffer from motion blur issues due to pixel response.
| Requirements | Detailed description |
|---|---|
| High-resolution LED panels | LED panels form the building blocks of the LED wall. Higher resolution panels offer detailed backdrops. The pixel pitch (center-to-center distance between pixels) is important. A lower pixel pitch (1-2.6mm) is preferred for close-up shots, while a higher pixel pitch may be suitable for wider shots. |
| High refresh rate | To avoid on-screen flickering, the refresh rate (times the screen updates with new images per second) should be at least 120 Hz. This is particularly important in film production where camera sensors are highly sensitive. |
| Brightness and color accuracy | The LED wall should be able to deliver high brightness (measured in nits) to simulate natural light conditions. Additionally, the LED wall should support wide color gamuts ensuring more accurate color reproduction. |
| Processing power and software | High-resolution graphics in real-time require substantial processing power and specialized software. Unreal Engine is a commonly used solution. The system should be capable of managing real-time tracking data, rendering, and syncing with other equipment. |
| Camera tracking system | To adjust the visuals on the LED wall based on the camera’s position and perspective, a camera tracking system is necessary. It provides depth illusion and realistic responses to camera movements in the virtual environment. |
| Lighting consistency | LED walls also provide interactive lighting for scenes. To maintain lighting consistency, the color temperature and brightness need to be synchronized with the physical set lighting. |
| Content management | A media server or content management system is required for visual display on the LED wall. It should manage high-resolution content, offer real-time controls, and support various input and output interfaces. |
| Physical setup | The physical setup of the LED wall, either curved or flat, depends on the production requirements. The mounting structure must be robust enough to securely hold the LED panels. |
| On-set calibration and alignment | To ensure uniformity across all panels, the LED wall requires careful calibration for color and brightness. Moreover, the images displayed on the panels must be seamlessly aligned to create a convincing, unified image. |
Understanding the Challenges of LED Video Walls and Their Impact on Performance
One of the defining features of LED video walls is the large number of LEDs they contain. Even a small 2.6 mm pitch LED tile, measuring 192 × 192 pixels, consists of over 36,000 individual pixels. This means that controlling each pixel requires a considerable amount of input/output (IO) and control lines, considering that each pixel is RGB and requires separate control, and even with minimal power usage per tile, the sheer number of tiles adds up to a lot of electronics.
Most manufacturers use field programmable gate arrays (FPGAs) as programmable hardware to enable parallel control of sections of the LED video wall to reduce overheads, leading us to the scan rate, also known as the multiplexing rate, which determines the ratio of LED drivers to LEDs within the panel. Reducing the number of circuits driving the display results in higher scan rates and can lead to reduced brightness, flickering, and slower refresh rates.
Apart from scan rates, LED video walls present other challenges related to color accuracy. RGB LED sources emit light in narrow wavelength bands, leading to color inaccuracies. LED panel manufacturers are aware of this issue and are working on improving color accuracy in future iterations but production teams have to rely on a suite of software to support their work. Color use also has an impact on the wear and tear of LED walls. If, for instance, you are shooting a scene with a lot of reds, like sunsets, the red LEDs are going to get the most use and are going to wear out before the green and blue LEDs.
Another issue that arises with LED video walls is the occurrence of Moiré patterns. Moiré patterns result from the interaction between the pixel pitch and fill factor of the panels. While finer pixel pitch screens offer sharper images and allow shooting from closer distances, they can also introduce Moiré patterns. Moiré patterns are interference patterns that can manifest as unwanted visual artifacts, affecting the overall image quality. Brompton Technology is a company that is most often cited by production professionals as providing software solutions that are widely used in virtual productions using LED walls. The company has a product called Shuttersync which, you guessed it, tunes your LED display refresh to your camera, or the other way around.
None of the issues with LED walls in virtual production are being left unaddressed or are insurmountable. It might bear mentioning that while the use of LED volumes were not unknown in television production, they are relatively new in the film business. Studios are being built by Sony, Samsung, and LG, among many others, to facilitate greater adoption, as well as to provide showcase studios of the future. The one thing that we have found in Digital Content Creation Report 2023 is that content creators are expanding their reach across all devices and have an insatiable need for display technologies. From gaming, DOOH, retail, television, film, and moving into MR environments, content creators have a broad range of target displays, and with virtual production, they are starting with the first point of contact, the LED wall.
