In 2009, fledgling video game startup Riot Games launched its first title: League of Legends. Fast forward by a decade and that very title was celebrating its 10th anniversary, having since grown into a gaming juggernaut as well as Riot’s tentpole franchise. As part of the festivities, Riot announced an ambitious extension of League of Legends’ universe: an animated series called Arcane to be released through Netflix.
On November 19th, 2021, Riot Games launched its Undercity Nights event in Los Angeles, a once-in-a-lifetime experiential pop-up coinciding with Arcane’s season finale.
A two-day hybrid festival welcomed attendees to Arcane’s undercity of Zaun. Event proceedings featured live League of Legends competition, virtual meet-and-greet opportunities with the show’s voice actors and creators, a concert featuring artists from the soundtrack and more broadcast worldwide.
AOIN Involvement
The event’s broadcast was produced by the BC Live team, who enlisted AOIN to design and integrate a real-time augmented reality environment enveloping home viewers in the setting as much as those attending in person.
These AR set extension elements allowed the practical environment to expand beyond the venue’s physical confines, allowing viewers to gaze into the undercity and soak in the city of Zaun.
To bring this vision to life, AOIN transformed an offline rendering of the undercity into an AR environment optimized for real-time playback. Additionally, the team designed custom holdouts and masking post-process tools for runtime compositing within Unreal Engine.
AOIN worked closely with Stype and BC Live to coordinate a delicate load-in procedure resulting in AR camera accuracy down to the millimeter, a process crucial for sustaining the digital illusion and for ensuring the AR holdout tools worked as expected.
Technical Challenges
One of our primary challenges for this project was developing a custom solution for integrating AR elements into an existing physical set built to emulate the show’s undercity. Arcane viewers were already intimately familiar with the setting, so reproducing Zaun’s look and feel was of the utmost importance.
Onsite AR integration for this project was particularly difficult, as while most of the scenic and production elements were already in place at the time of integration, others were still arriving onsite for showtime. This meant that AOIN needed to develop a custom solution for ensuring AR element accuracy while working around existing production elements and coordinating closely with the core production team to avoid delays and stoppages.
Our Solution
To accommodate this dynamic environment, All of it Now developed a custom AR holdouts tool enabling AR elements to be properly masked and occluded by real-world elements, seamlessly melding the digital and physical worlds into one.
To achieve this, AOIN used 3D models of the fabricated set as masking elements for the AR objects. These masking elements unlocked tremendous creative potential, allowing for a much more dynamic AR environment, spanning far wider and deeper than a traditional AR broadcast would normally accommodate.
Augmented reality elements are typically overlaid on top of the camera feed; it is as if someone had applied a sticker to a camera’s lens. Given the shifting perspective of the camera, a simple composite would severely constrain AR element positioning and create other occlusion issues.
By innovating this holdouts tool, production elements like the lighting rig and camera jib were hidden, preserving the immersive quality of the show’s environment.
The Result
By leveraging existing game assets for integration into the physical set as AR elements and developing the AR holdout occlusion tool, AOIN transported broadcast audiences to Zaun’s streets in all of their grimy glory with an experience exclusively tailored for digital screens.
The broadcast featured the AR elements prominently, using them frequently dramatic camera flythroughs, transitions and as backdrops for segments hosted by the event’s MCs, bringing Arcane fans closer to the show than ever… all without needing to brave the vile stench of the undercity’s air.
