Filmmaking is a highly creative but often time-intensive and complex process that must be executed in a linear fashion. More recently, filmmakers have been embracing the flexibility and artistic freedom that comes from virtual production. Some of the most talked about films and TV shows from recent years (like The Mandalorian and Thor: Love and Thunder) used virtual production techniques, achieving spectacular results!
In this article, we’ll introduce you to virtual production and how it can benefit filmmakers and creatives like you.
What is virtual production?
Virtual production is a collaborative process that marries the physical world of filmmaking (actors, sets, props, etc.) with the digital world (lighting, VFX, etc.) in one process. It brings digitally created content, which was traditionally done after filming, to the film set in real-time. Many popular shows (including HBO's House of the Dragon) have employed this innovative process.
Virtual Production techniques have been recently popularized by big productions, including ILM's successful series The Mandalorian
Filmmakers and real-time 3D artists use game engines like Unreal Engine to create digital scenes. These scenes are projected onto large LED screens, creating a virtual backdrop that imitates a set or real-world location, yet you have far more control over the scene. These curved screens, known as The Volume, wrap around the actors, props, and practical sets and seamlessly dome over the ceiling.
As the actors perform in a scene, camera tracking technology adjusts the 3D world behind them to match the live-action camera. The synchrony between camera and digital imagery creates parallax, helping to create the illusion that the real and virtual elements reside in the same space.
Virtual production and LED volumes allow actors, creatives, and filmmakers to see how their visual effects look within a scene in real-time. There are currently 100 LED stage volumes worldwide, with another 150 coming soon.
Learn more about the method and process of virtual production in this video from Unreal Engine. CG Spectrum is a proud Unreal Engine Academic Partner and Authorized Training Center.
Top 5 reasons to choose virtual production for filmmaking
Not only is virtual production optimizing production processes at every stage, but there are also many other reasons filmmakers are using technology like Unreal Engine in the filmmaking process, including the following top five reasons:
1. Enjoy creative freedom and develop virtual worlds
Virtual production allows creatives and filmmakers to create 3D scenes that often can’t be replicated in real life, a process commonly known as world building. This may include alien landscapes, romantic castles and kingdoms, or never before seen worlds and dimensions.
These digital worlds are created by real-time 3D artists and then displayed on LED stage volumes.
With LED volumes on set, filmmakers can:
- Adjust creative on the fly, testing different lighting, camera angles, etc.
- Extend the “set” as the actors move through it
- Duplicate crowds
- Use more realistic lighting and integration than green (and blue) screens allow
- Switch to new scenes/locations more quickly than using practical sets (once The Volume is set up)
Also, with the ability to create almost any virtual world you can imagine, production scouts spend far less time and money on securing appropriate locations. A production team can now shoot any environment they wish—real or imaginary—from the comfort of the studio (or, for smaller productions: your home computer!).
Still from a virtual world created by CG Spectrum alum, Marc Carratala, using Unreal Engine. "World-building needs a lot of referencing, and any small or big cinematic always needs a nice storyboard." [Marc]
2. Low barrier to entry
With the democratization of virtual production, a project that used to take 100 people to create can now be created by you on your home computer. Game engines like Unreal Engine and Unity are free to download and learn, and with the rising demand for real-time skills, now is the time to learn!
Creators can benefit from a trove of free educational resources (Unreal Engine has over 100 hours of free tutorials), or if you're considering a career in this field and learn better through a more structured curriculum and mentorship from a professional, consider CG Spectrum’s virtual production courses.
3. Be part of a cost-effective production
Virtual production greatly assists in a project's pre-production phases, making planning out shots and scenes—camera angles, actions, sets, etc.—faster and more accurate, which can save a lot of time and money during filming (production) as well as during post-production.
Virtual production also allows a director to have more creative input earlier on and have a more realized vision for the rest of the production.
4. See edits in real-time
With the help of "The Brain Bar," including live VFX compositors, a director and their production team can make adjustments such as color grades, lighting, and even smaller changes to digital assets in real time. This results in less need for reshoots (and if a reshoot was required, you would only need to bring the set back up on a LED screen as opposed to rebuilding a set or flying back to a location) and less work down the line in post-production.
The "Brain Bar" consists of real-time artists, camera tracking experts, video display I/O (incoming/outgoing) experts, content loaders, live compositors, IT experts, network specialists, and LED screen technicians. (Image: PRG)
Using Nuke, a live compositing artist can also quickly combine CG backgrounds with their corresponding live-action shots, called plates, which are used in the cut (until a more polished version can replace it) as a form of postvis. This assists the editorial and post-production teams and helps the director make important decisions about the cut.
5. Save post-production time
Another benefit of virtual production is that it saves post-production time and costs. Creating visual effects in post-production is time-intensive and, in some cases, it can actually be cheaper to go back and reshoot a live-action scene than to address something digitally.
When using virtual production, the LED screen gives off more realistic lighting than a green screen. As a result, VFX artists spend less time fixing “green screen color spill” and other lighting issues, which makes integrating post-production visual effects with the live-action shots more straightforward and achieves more realistic outcomes.
Student work by CGS alum Reed Yerien to exemplify compositing on top of a green screen (right). Reid needed to spend some time blending the actor into a CG scene and removing the green spill during the post-production stage (left).
Even some of the 3D models (assets) that are created for production can be packaged up using specific software and passed on to the post-production team to use as reference, a base for their 3D model or set, or even as the final asset.
Post-production artists will still need to hide wires, build and animate VFX creatures, and add additional FX and color correction, but a lot of the “world” and lighting will be refined during filming creating a more seamless integration of CG and live-action (and a clearer understanding of their client brief).
The future of filmmaking
The future of filmmaking is bright, and with technology such as virtual production, it will become much more accessible for all types of creatives working in the industry, including:
- Production teams: Virtual production will save production teams money and time on building practical sets or shooting on location. Virtual production's real-time editing capabilities also mean filmmakers can get the shot right on day one and not need to reassemble sets, actors, and production crews for reshoots. And the detailed and highly visual planning that can now take place during pre-production stages streamlines all proceeding production processes.
- Actors: LED stage volumes turn green screens into realistic environments surrounding the actor. Virtual production allows actors to become far more immersed in the scene, helping them to perform more authentically.
- VFX artists: Being able to render in real-time is a huge advantage for VFX artists, speeding up processes and allowing for more experimentation and creativity.
Virtual production is rapidly changing and evolving. Filmmaking in the future is likely to adopt more virtual production techniques, enabling them to create more realistic, larger-scale productions faster and at a fraction of the cost. This is sure to open up many exciting opportunities for creators!
How to start a career in virtual production
To be able to work in virtual production, you’ll want to learn the relevant theory, skills, and software.
CG Spectrum virtual production course alum Marc Carratala's biggest piece of advice for artists considering a career in virtual production for films is:
“Have an open-minded approach when starting the project and to properly do some research based on what you want to create. Spending time on the creative process during pre-production is always better than fixing things up late in the production stage.”
Depending on your current skill level, you could enroll in one of CG Spectrum's virtual production courses to get ahead in the field:
- Real-Time 3D Essentials: Take the first step towards becoming a 3D virtual artist with training and mentorship from real-time experts at an Unreal Authorized Training Center. In this introductory course, learn basic rigging and motion capture to create fully animated, playable characters ready to use in a scene.
- Real-time 3D Foundations Course: Learn to create film-quality environments using world building, digital humans, lighting, and camera techniques, and see how to apply these skills in the real world for job roles on a stage volume.
- Unreal Connectors: Virtual Production: Created in partnership with Epic Games, this 12-week intensive course, with 5 x live expert-led sessions per week, has a cutting-edge curriculum that helps studios and media professionals with production experience learn virtual production and industry workflows using Unreal Engine faster.
Ready to choose virtual production for filmmaking?
Do you want to break into the innovative and diverse industry of virtual production? With virtual production courses from CG Spectrum, get expert mentorship from virtual production and real-time professionals who've worked on incredible projects! You, too, can learn to use tools like Unreal Engine to create new worlds and enhance the future of filmmaking as a real-time 3D artist.
Related Links
