360 Video Production
It seems that nowadays, everyone is talking about VR, Immersive, or 360-degree video. The new medium, and the technology that surrounds it, is everywhere. From high-end, Hollywood-level blockbusters to smaller, grassroots efforts like the New York Times on-the-ground VR news stories.
However, with so many questions surrounding the proper protocols and practices of conducting a viable 360 video shoot and post, the questions linger large. Is there a correct rig? What does a viable crew look like? Should we do a stereoscopic or 2D shoot? How much creativity can be injected into the post-production phase and how similar is it to a traditional video job? And, most importantly, how much does all of this cost?
At Venture have spent the better part of the last year immersing ourselves in this new medium. We have studied the various types of camera rigs and their pros and cons to deliver the best 360 experience. And, out of all this testing we have truly grasped an understanding of the medium itself: its shortcomings and its potentials
The Concept
When it comes to 360 vide production, there are two different categories that they can fall into: live action and full-CG renderings. Live action is exactly what it sounds like. A series of cameras record action on a location that is live for eventual playback later. You can certainly composite in CG or motion graphic assets later, as you would be able to in any other traditional pipeline. However, the core plates are static, recorded live and played back later. Movement is restricted to rotation as the cameras were locked off. No information exists, for example, of the image if the focal point was tilted or moved slightly to the left.
The way to add some depth and movement to the live-action VR experience is to incorporate ‘stereoscopic’ or 3D camera rigs into the equation in order to give paralax (depth distance between different objects). However, this adds a tremendous amount of complexity to both the production and post-production phase and cost as you will be doubling the amount of cameras needed for the production.
CG VR is similar to a video game in that a rendering engine (most likely a video game engine like Unreal Engine or Unity) renders the frames in real-time for playback. This type of VR gives the viewer the best experience because the immersive world can be constantly updated to their movement and actions, unlike a previously recorded video, which is a set and concrete medium.
However, CG is a very expensive and difficult medium to pull off, both from a cost, time, and resource standpoint.
The Rigs
Assuming that live-action is the correct route for your project, the next step is to decide upon the correct camera rig for your project. Since 360 video production is such a new technique and medium, there is truly no correct or industry standard way of shooting any particular scene. Every day, it seems, new mounts, rigs, and setups are finding their ways onto sets across the worlds, whether they be manufactured by reputable companies or jury-rigged via 3d printers.
Therefore, it is important to choose the best setup for your project with a practical understanding of what your end deliverable needs to be. If you are simply looking to provide a 360-video experience that will be shown exclusively through Google Cardboard, then you shouldn’t shoot on a 12-RED camera stereoscopic rig. A 6 GoPro rig will do exceedingly well in most situations.
No matter what, you will be looking at a minimum of 6 camera rigs in order to capture a seamless, 360-degree view of the scene. Obviously, you will need to double those cameras if you wish to capture those images in a stereoscopic fashion as you will need to account for a left-eye and a right-eye camera.
The Set
The oddest part of 360 video, for me, is the fact that you cannot have ANY visible crew or lighting in the scene. In this way, 360 video is similar to that of stage production where audio equipment and lighting needs to be hidden, lest it be recorded.
When planning a 360 shoot, an alpha-producer can easily feel that he or she is not doing their job correctly. The equipment list is much smaller than a traditional shoot, as is the crew. However, one must keep in mind that a VR shoot needs to be scaled back because, literally, there is no place to hide.
Therefore, it is very difficult to have a “moving camera” shot as Dollies and Steadicams are easily recorded. Some have tried to mount the camera rigs to drones and have the camera fly through the scenes. However, this technique is faulty in two ways:
1. By having the horizon line shift and tilt, it will make the viewer feel incredibly uneasy and bring on nausea
2. The drone’s blades and body will still be visible in the outer reaches of the frame. Granted, human nature deems that people look “up” less often, but VR still elicits a complete 360 look-around response. Thus, a tremendous amount of VFX painting and cleanup will still be required.
Ultimately, a static camera, placed on a minimally invasive mount such as a mono-pod is the best-case-scenario. If movement is necessary for the story, then expect to do a fair amount of VFX fixing and paint/roto work.
Audio
Audio can greatly add to the immersive quality of VR. Sound recording mixing has been one of the earliest adopters of the medium and, as such, one can greatly add a tremendous amount of value to their production simply by incorporating some simple audio tricks. ProTools, for instance now allows one to mix in what is called VR Space. Going beyond Surround Sound, VR Space, essentially, allows you to choose what areas of a 360-space to place your sounds. For instance, if you want the viewers attention to be drawn to a door, then a creak can be “placed” at that exact spot, thus drawing the attention to that point in space.
Additionally, manufacturers have begun to produce microphones with the ability to record sounds within vast tonal and audible ranges. Since microphones need to be hidden in scenes, they often need to be able to pick up the subtlest whispers at the same time as they pick up the loudest crashes. These latest in VR microphones allow for the same single microphone to record all audio signals without any loss of quality to either sounds.
Post Production
So, you have your video acquired. Sure, it exists spread across a dozen-or-so cards that aren’t synced, but its there, dammit! Now what? Well, now you stitch. You see, you have all of these video clips that are facing different directions and are in different orientations. Some are facing the sky, so the exposure is brighter, some are facing the ground, so the exposure is darker. And the ones that were located next to each other all overlap their neighbors to some degree. It is now your job to “stitch” these images together at these overlapping parts so that when completely combined, you have a total 360-degree video.
Once that is done you can isolate each of the individual camera’s video and, essentially, color-grade that video so that it matches all the others. The end product being a pretty, professional-looking video.
If this is what you want? Great! Output an H264 and you’re good. If you want to add graphics or VFX, then your workflow is very similar to that of a traditional commercial post pipeline. The only difference is that you aren’t going to be working with a traditional 1920×1080 or 2k or 4k workspace. Instead, you will be working with a 360 workspace which can be tricky to judge spacing, size, and accuracy.
That's a Wrap
Obviously, this is just a top-level, fundamental skim of what one can expect from their first VR project. The thing that I love most about this medium is that people are pushing the boundaries of what was previously possible on a daily basis. Motion graphics are being introduced in new and unique ways, as are simple VFX techniques like matte painting and projection maps. Anything is possible because, frankly, the rules are being rewritten every hour.
The only steadfast rule is that you need a production company that understands the space, the technology, and roles of this unique and ever-changing medium. We at Venture would love to collaborate with you on your first (or next!) VR project.
If you have any questions or would like us to bounce around ideas with you, feel free to reach out to [email protected]
Venture is a Denver based production company that specializes in content production, creative editorial, and visual effects / finishing. Oh, yeah, and 360 Video!