Augmented Reality on Live Stages

Our main goal when we developed Ventuz was to allow our clients to liberate their creative freedom and come up with innovative concepts that combine technologies and features from different markets to create something new. Dubai-based agency Eventagrate has made quite an impact with one such idea, which we would like to present in this article: the AR stage.

Virtual and Augmented Reality have been highly sought-after technologies for many brands and events during the last years. And while their application areas seem a bit restricted, the team at Eventagrate found ways to bring AR onto a live stage. They started with a project for customer Viacom in India supervised by Ahmed Naggar, before bringing the concept to their home turf in the UAE for the Opening Ceremony Children’s Film Festival in Sharjah in October 2018.

The setup of Eventagrate’s AR stages is quite straightforward: a camera films the stage during the presentation, the filmed image is streamed through Ventuz, where real-time graphics are composited onto the feed. For this purpose, Ventuz supports a wide variety of commercial capture hardware from vendors such as Deltacast, AJA, Blackmagic, DataPath, DVS, and embraces new standards such as NDI. The added graphics are prepared in size, position and tilt to look as though they are a part of the physical stage, serving for the augmented part of the reality. The final result is then displayed on a large LED backdrop wall. In the case of the Children’s Film Festival in Sharjah the LED wall had a size of 30 meters. “The center piece of the LED wall – a width of 12 meters – was left for simple backdrop graphics, while we showed the feed of the AR stage on both side areas”, said Ghandi Al Zughbi, Ventuz Operator and Designer at Eventagrate.

The thrill for the audience lies in the contrast between what they see on stage and what the AR stream is showing them to be happening on stage. For the presenter, AR offers unique ways to make use of the entire space and interact with the content. Immersing the presenter in a virtual scene does not only raise the entertainment value of the presentation, but also provides innovative opportunities to communicate information.

Because Ventuz offers all its features in one comprehensive toolbox, the combination of technologies that seem aimed at different markets can be achieved quite fluently. AR is most commonly used in the broadcast industry, where it enhances sports or news studios in the form of Virtual Sets. The workflow requires features like Alpha Blending, Masks and Camera Tracking, which have been a part of the Ventuz ecosystem for many years. Combining these features with stage graphics pipelines and live show playout is easily done, and by adding the powerful content authoring tools to the mix, creatives are given free range to bring innovative concepts to life.

In the case of the Children’s Film Festival, the content focused around film production and movie experiences. The host, Sheikha Jawaher, led the audience through five different scenarios, each of which was loaded with 3D models: a film set, a movie theater, a factory and more. All of these models had to be created inhouse by 3D Designer Tijo Thomas, and then imported into Ventuz. “I like using the Cinema 4D Live Link in Ventuz”, said Ghandi. “It allows us to work more flexibly with our 3D content and create models and the real-time environment simultaneously.” In fact, the Live Link between Ventuz and Cinema 4D holds many workflow improvements for the creation of AR and VR content, as we demonstrated in a talk at IBC 2018.

This level of efficiency was desperately needed, since Ghandi, Tijo, Technical Director Miguel Galvao, and Project Manager Sabrina Lakel had only ten days production time. The last two days on set were mainly used to adapt the content to the real camera position. “You can take close estimations beforehand”, explained Ghandi, “but the final camera image will always differ a little bit from the virtual camera we used in production. A few tweaks in position, lense specs and focal length, and we are good to go.”

The result was stunning, and the opening ceremony of the Children’s Film Festival in Sharjah was a great success. Sabrina had the pleasure to watch the show from the back and told us that she had never seen so many people film the show with their mobile phones. “Not even 24 hours after the event, I got an email from one of our Saudi clients with a video, asking me if we could do a project like this. And the video showed our own project. That was amazing”, said Sabrina.

It seems that we will see more AR stages coming out of Dubai in the future, but although the concept is already quite popular, the team at Eventagrate will not stop perfecting it. “I think the next step will be adding a tracking system”, said Ghandi. “This will allow us to easily integrate a backup camera and maybe even work with moving cameras.”

Here, the Eventagrate team can fall back on a number of integrations that are already available in Ventuz, among them NCam (read more about the cooperation in this press release) and Trackmen. To learn more about tracking for AR, click here.

 

Feature Rollup

• Live Video merged with 3D content
• Live Link to Cinema 4D for a seamless 3D workflow
• Masks for smooth integration of 3D content into the video stream
• Real-time nature of Ventuz for last-minute adjustments on site