Art is not a reflection of reality – it is the reality of a reflection. Pioneering director Jean-Luc Godard used these words to describe his iconoclastic approach to film making. They apply equally well to the “virtual” productions that this New York City lighting designer and his team at Boum Creative have been pursuing with growing intensity of late.

Drawing on an extensive array of design, sequencing, and control tools, many of which could not have been imagined a few short years ago, the team at Boum (Danielowski, Mat Leland, Tanner Simpson, and Corey Hatch), are fusing the virtual and physical to create Extended Reality productions.

Some of their offerings are meticulously realistic. Others create fantastical scenes that transcend time and place. The goal, as the legendary French-Swiss film director so artfully put it, is not always to reflect reality, but to create reflections that immerse and transform, taking us to worlds that go far beyond the real one that lies outside our front door.

Although COVID-19 has provided an added impetus to the Extended Reality trend, Boum Creative has been at it for some time before the pandemic hit, moving forward another step with each new tool that allows them to achieve their vision. Danielowski shared some insights with us into this emerging form of expression.

Boum Creative has done quite a bit of remarkable work in theatre, events and retail. Why have you decided to enter this new venture?
“We have been working with Disguise Media Servers for over three years. So, this isn’t a new venture for us. This is an evolution of the work we’ve been doing, where we’re taking new and emerging technologies into our field to push digital experiences to the next level. The emergence of Notch and Unreal along with GPU improvements, allow us to render in real time what used to have to be done in post.”

Ok, so how would you describe what you’re doing in real time? How is it different from an ordinary livestream show that is videoed and distributed on Twitch?
“Disguise came up with a statement that pretty well describes what this is all about. It reads, ‘Extended Reality (xR) is technology that allows you to blend virtual and physical worlds together using augmented reality (AR) and Mixed Reality (MR) in live production environments to create fully immersive experiences.’”

Aside from being a lighting designer what skills have you had to develop to do this?
“As a team, Boum has had to dig into a number of new technologies to make all this work together. As previously mentioned, this workflow hinges on the media server but beyond that we’ve had to improve our skills in Notch, C4D, and CAD. As for completely new skills, we’ve had to learn Unreal Engine, Pixotope, camera tracking, and most importantly, a workflow to get all of these technologies to come together in a way that works for us.”

What kinds of collaborators have you had to add to your team?
“We’ve teamed up with a few different teams for content creation. We do this in house as well, but content will drive this technology and we’re always excited to work with new artists. We are also partnering with production companies that already own LED tiles and have interest in xR stages.”

Where is the talent positioned when they perform for these productions?
“The Talent stands on a LED stage complete with a floor. As long as there is LED behind them, we can create an immersive world around them. From there, talent is able to see — and directly interact with — the environment we place them in.”

From a lighting perspective what is the key to making these artists come off naturally, as if they are in a “real” environment?
“One of the bonuses of using LED is that you get a natural light from the content on the screen. Little things that would take hours to fix in post, such as reflections on glasses or silverware, the light in a performer’s eyes, all of this, is given to you ‘naturally’ from the LED. From there, the LD’s job is to make everyone else’s work pop. Make sure that color temperatures match the scene environment that’s been made, separate the subject from the background, and endlessly make adjustments until it looks seamless.

So, does lighting play a similar role in an extended reality show as it does in a live one or an ordinary livestream?
“This question is a little more difficult to answer. Lighting a subject on the Xr stage is very similar to lighting the subject for any film shoot. However, utilizing Notch or Unreal you can bring in virtual fixtures. In doing this you’re essentially creating visualization software that can interact with physical (real) lights as well as virtual lights. Once you go down that rabbit hole, what you’re able to do is almost endless.”

What are the hardest aspects of a real environment to capture in a virtual production?
“As unsurprising as it may seem, lighting inside of the virtual world is one of the hardest aspects to create. Being able to coordinate real world fixtures and virtual world fixtures means when I turn a physical ellipsoidal on in the air, I can create a virtual shadow inside the LED floor to go with the real shadow. This creates a stronger impression of reality by pairing the two worlds together. All that being said, the virtual lighting rig needs to mirror the real-world rig.”

Is the goal of your creation to mimic reality as closely as possible, or to create the most immersive sense of reality as possible? In other words, if you have to choose a color or look that may not be 100% in keeping with they way things look in a real environment, would you do it if that meant creating a more intense reaction among viewers?
“The goal is to be as versatile as possible. The beauty of the virtual rig is we can be in photorealistic sets as well as fantastical/cartoonish sets. A set we’re currently working on looks very much like a desert, except the colors are surreal. I will definitely play with the color on in this situation as I’m lending lighting to the video environment.”

Where do you see your new service being used? Who do you see as your clients?
“Extended Reality is for anyone and everyone. It’s great for TV and film. Similar technology is being used for large scenescapes on the Mandalorian. Our stage will complement the large volumetric stages and allow for more intimate scenes. It will also be great for Corporate events and for companies that want to get past the simplistic Zoom conferences. The xR stage will elevate experiences in these situations.”

Is this experience going to change the way you light live events?
“I don’t believe this will change how we light events. As with anything, we’ll learn and take those lessons forward. It’s likely we’ll blend virtual with live experiences in the future, but there will always be nuances to live shows that will set them apart.”

Can you provide a brief overview of the process involved in designing a virtual show? Where do you begin? What are the key factors that you weigh?
“The planning that has gone into building our xR studio is a process in and of itself. We’ve spent weeks ironing out the details to make the studio as versatile as possible. Fortunately, Mat has lots of experience in recording studios and we’ve built the stage in that fashion. With the stage built out and the need to build individual shows, we listen to our clients and find out what their needs are. For some clients we’ll utilize stock sets and for others we’ll take their vision and create a set or even a world that their event can take place in.”

What are the big technical challenges that will have to be met to make virtual shows even more engaging in the future?
“As with all video, content is king. With content getting more and more realistic, hardware will be the lynchpin. We will need the graphics card manufacturers to keep pushing technology to advance the capabilities of any virtual show that will happen in the future.”

So, how popular do you see virtual shows becoming in the future?
“Forbes claims that extended reality will be a $209billion industry by 2022. How much of that will be with live shows or streams remains to be seen. If their prediction is right, we’re happy that we’ve already started.”