HoloBox: A Virtual Green Screen Studio For Mixed Reality Videos

Dave Price, aka Weird Wizard Dave, a software developer and YouTuber, revealed HoloBox, which is an application that enables you to create “virtual mixed reality” videos for YouTube "let’s play" videos.

Mixed reality video is one of the best ways to demonstrate virtual reality to an audience. It provides a third-person perspective of the gameplay with a live video of the player enjoying the content. This technique has been around since before Oculus and HTC released their VR headsets. However, not many people create mixed reality content because recording it is cumbersome and requires expensive equipment.

To capture a mixed reality video, you need a powerful computer, a video camera and capture card or a good web cam, an extra motion controller or a Vive Tracker, and a green screen with studio lights. Not many people are willing to invest in that much equipment, and few people have the free space to create a dedicated studio. HoloBox offers a more economical option by replacing all the expensive equipment with a software solution.

Instead of embedding a video recording of you playing the game into the game footage, HoloBox uses animated avatars to represent you. The application runs alongside your VR game and renders your avatar over a solid backdrop to make it easy to isolate your avatar for compositing. The default backdrop is chromakey green, but Price said you could use any color as a backdrop.

HoloBox uses the position information from your VR headset and your motion controllers to animate your 3D avatar with your body movement. The software also includes a basic “chin-wobble” lip-sync system that animates your avatar’s mouth while you speak into the microphone. Price said that he plans to update the lip-sync system soon, though.

To record a traditional mixed reality video, you must dedicate a tracked object, like a third controller or a Vive Tracker, as an in-VR camera and match that up with a real-world camera. With HoloBox, you don’t need to worry about lining up the virtual camera with anything else. The software offers an interactive virtual camera overlay that you can see in-game. The overlay camera works like the spectator camera in Job Simulator. You can reach out and grab it at any time to reposition it within the virtual world. Price said that you could also automate the camera movement with scripts or an AI algorithm.

Price said that HoloBox and applications run completely independent of each other. HoloBox uses the tracking information from SteamVR, but it doesn’t interact with the games that you play. For instance, your HoloBox avatar is not subject to the lighting changes in the game. Furthermore, the HoloBox camera can’t detect the walls of a game, so you must be cognizant of its placement as you wander through VR environments.

Price said that HoloBox not yet ready for public distribution, but he wants to “put together a small group of testers” that would create content and provide feedback. He's also interested in help from other developers, 3D modelers, and animators who are interested in lending a hand. When the time is right, Price intends to release HoloBox free of charge.

You can find Price's email address on the About page of his YouTube channel.