Virtual Production is a term that is quickly becoming more and more popular and there is a lot of discussion around its current use and potential, but what exactly is Virtual Production? This page will explain it in detail, look at the different types of virtual production and run-through what you need to incorporate it in a workflow.
What is Virtual Production?
Virtual Production is a method of production that combines the physical world with virtual elements using real-time software. This can be achieved through different methods such as visualisation, performance capture or using an LED wall. One of the key benefits to virtual production is that creative decisions, such as moving elements within a scene, can be seen in real-time and this makes collaboration a much more stream-lined process.
The Problem with Traditional Production
The majority of productions today will feature visual effects in one form or another and there are entire genres that will rely on it heavily. The issue with a traditional style of production is that utilizing visual effects often requires techniques such as filming on a green screen or compositing an element in later, without knowing exactly what the final image will look like when these elements are replaced. This causes several problems with framing, lighting, actors staging and so on. This linear style of production also separates the filming and visual effects stages for the most part and that separation can lead to iteration, which can be a lengthy process.
Virtual production however, takes this process and makes it a more non-linear and collaborative workflow, all in real-time. This can manifest in pre-production using visualisation to bring all team members a closer representation of what the final image will look like before production has begun. Alternatively, it can be implemented in production with the visual effects department engaged to provide real-time imagery on what the final image is intended to look like.
Who Can Use Virtual Production?
We are currently seeing big-budget productions pioneer the use of virtual production techniques and benefiting the likes of cinematographers who can view camera visual effects in real-time, visual effects departments who can work more collaboratively across all stages of production and producers who can get the best results out of production and demonstrate their intentions with a project at earlier stages. As the required technology becomes more readily available, we are seeing this being used across all levels of production to gain benefits.
Virtual Production Types
Virtual production can encompass so many different techniques and terminology, when it’s broken down though, virtual production simply consists of four types of production that share the common feature of a real-time engine. Knowing what these are can help give an understanding of the benefits and advantages to a virtual production workflow.
Visualization
Visualisation is often part of the pre-production stage and is the creation of prototype imagery to plan and represent certain shots or sequences. This can take several forms such as:
• Pitchvis - Used in the development process to pitch key sequences or even trailers to a studio or potential investors
• Previs - You might recognise this from behind-the-scenes or before/after footage showing a concept for a visual effect element. This is the next step to storyboarding, allowing for more creative detail.
• Virtual Scouting - Location scouting occurs in traditional production to choose a location or plan the shoot. Virtual Scouting is the process of interacting with a completely digital version of a location, down to using virtual cameras and lenses to plan out specific shots.
• Techvis - The process of bringing virtual elements to the real world and assessing the physics of camera moves, placement and lens choices to plan certain shots, sequences or entire scenes.
• Stuntvis - Planning how stunt moves will integrate with the camera placement, set and props down to specific beats which can be used to improve safety or creativity of a stunt.
• Postvis - In a similar way to previs, this is demonstrating the combination of live-action elements with temporary visual effects to help guide earlier stages of post-production.
Performance Capture
Motion capture is the process of creating animated models by recording the movements of objects or actors, this is usually accomplished by an actor wearing a suit covered in markers and using depth-sensor cameras for facial capture. Performance capture is used to have virtual characters interact with real-world environments in real-time, which is designed to help aid in framing and timing on location.
Hybrid Virtual Production
In traditional production, when filming in a green screen studio the footage is sent to post-production who composite the image after the shoot and this can often be the first time the combination of the elements are seen. Hybrid virtual production is used either to provide a crew a live preview of composite shots which are then to be finalised later in post-production or this has already been utilised to key in real-time for live broadcast, especially in sports.
Full Live LED Wall
This is becoming the most popular and discussed form of virtual production at the moment, and is the process of combining camera-tracking with real-time images displayed on a live LED wall. When compared to traditional green screen filming some of the benefits of this include it allowing all departments to see the final-pixel imagery, the actors are lit by the wall while also being able to react to the environment and it removes any uncertainty.
Virtual Production Project Types
Fully Animated Virtual Productions
Utilising virtual production in the form of performance capture has already had a massive impact on the creation of animated films and we have seen this being utilised for a while now. As technology develops we are only seeing the accuracy and quality of this grow, coupled with the inclusion of virtual cameras, fully animated virtual productions are only going to benefit as they become more realistic and lifelike.
Live-Action Virtual Productions
It can be quite easy to imagine how live-action productions can use virtual production in the form of an LED-wall to bring a new dimension to visual effects-driven productions that would have previously been based in a green-screen. However we are also seeing how visualization techniques can enhance productions that are set entirely on location with minimal effects.
Key Features and Benefits of Virtual Production
Real-time Workflow
One of the greatest benefits of virtual production is that everything happens in real-time, down to the compositing. This means everything can be assessed as it happens and instead of requiring the formerly linear process of post-production to determine any issues that arrive or creative decisions that need to be made, they can now be spotted or made at the time of production.
Remote and Multi-User Collaboration
As technology and computing get faster, communication and collaboration in production is only getting stronger and in a virtual production workflow this has benefits across all stages of production such as remote virtual scouting sessions to collaborative performance capture sessions.
Malleable Tools
Using a real-time engine as part of a production gives various different departments presets and tools they will use as part of their virtual production workflow. These have the advantage that they can be revisited and re-used across different scenes, locations and even projects.
Real-time Physics
The physics in a real-time engine are a key part to certain areas of virtual production and are what makes tools such as simulating camera movement possible. This brings benefits to areas such as visualisation down to the visual effects seen in the final image.
The importance of a powerful Virtual Production Systems
Whether it’s compositing in real-time or displaying a high-resolution image across several LED screens this can require a lot of computing power and one of the most important tools you need to power a virtual production setup is going to be the workstation. Making sure you have the right workstation for your needs is crucial for achieving the best results and Scan Pro Video offer solutions for all scales of production. Below are some examples of specifications for some of the different virtual production uses.
Virtual Production - LED Wall
Virtual Production - Green Screen / Content Creation
CPU
AMD Threadripper Pro 5965W
AMD Ryzen 9 7950X 16C/321
RAM
128GB DDR4
64GB DDR5
GPU
RTX Ada A6000 48GB
RTX 4090 24GB
Drive - Primary
1TB Samsung 980 Pro
1TB Samsung 980 Pro
Drive - Projects
2TB Samsung 980 Pro NVMe
2TB Samsung 980 Pro NVMe
Add-In Card
Quadro Sync II
Capture Card
3XS Virtual Production Workstations
Scan Pro Video have a range of virtual production workstations that are designed and optimised for driving LED walls in a virtual production. These systems leverage the power of NVIDIA's A5000 and A6000 GPUs alongside a SyncII card, allowing you to sync your output across multiple high-resolution LED Walls. Check out our workstations below for more information.
The industry had a massive shift recently as our way of working was made to be more remote. The response from the latest technology made it that working collaboratively and remotely is the new normal. Real-time technologies make all this possible including tools such as NVIDIA Omniverse.
NVIDIA Omniverse is the platform that provides content creators the ultimate real-time collaboration experience.
Depending on the type of virtual production workflow, there are some hardware requirements that will be involved. Part of utilising real-world data for virtual productions requires translating camera operation to create more realistic movements. Companies such as MoSys provide camera tracking solutions such as StarTracker, an optical solution built to address in-studio camera tracking.
LED Walls
LED Walls are created by combining multiple LED panels to create a virtual screen using powerful processing hardware and real-time software. Unlike using a green screen backdrop, the image displayed on the LED Wall is recorded to the camera and delivered in the final project. To keep up with the quality of modern-day cinema cameras, it’s important the LED panels are high-resolution, have a wide colour space and are of the highest quality.
As part of offering solutions for all stages of the virtual production workflow, Scan Pro Video have solutions for LED walls, including all the necessary connectivity.
Scan has recently supplied Recode XR studios in Manchester with a custom, ultra-high-end 3XS VP workstation to run both their new LED wall, and Aximmetry (Unreal) VP software.
The system has been used exclusively on a number of high-end ITV, BBC & Sky Dramas since its installation, and continues to provide a high-end solution and un-wavered support for this developing technology for filmmaking.
This special 4U rack mounted workstation consisted of the latest AMD Threadripper Pro 5995WX along with a Nvidia RTX A6000 Ada GPU, 128GB RAM along with a Blackmagic Decklink 8k Pro.
We explore how Recode XR are utilising the latest NVIDA RTX Ada Generation powered 3XS Systems, delivering real-time results to stay at the forefront of new and emerging technologies.
Nightsky Studios are one of the very few independent stages in the UK, and the first in Scotland.
During the research and design phase of their project, they sought powerful and customisable workstations to power their VP stage. Our configurator proved to be just the tool they needed to be able to spec their workstation to exactly as they wanted.
Our team were available to help progress the project and were able to design and quote everything they needed to make their VP stage not only functional, but impressively powerful.
Scan supplied 5x 3|XS VP Pro 32C workstations with PNY A6000 GPUs, along with a Sony FX9 4K camera, Zeiss CP3 cine lenses HTC VIVE Pro camera tracking hardware and Arri Skypanel LED lights.
