Pentakill
Live interactive virtual concert with Riot Games
From late May to September of 2021, I worked on an amazing collaboration with Riot Games, The Mill and Wave, creating a live interactive virtual concert for Riot’s fully virtual metal band, Pentakill.
We performed the show in September, live from Digital Domain’s motion capture studio, utilizing their optical-tracking-based mocap system called Giant for motion capture and head-mounted iPhones for face capture.
Challenges
The goal was to give fans a brand new way to experience Pentakill, through an interactive live broadcast of their music interwoven into a narrative arc.
Originally we had planned to do all the music live, but given the tight time frame and the risks involved, we had to unfortunately scrap that and pre-record the performance. However we did add a live moment where the main villain, Viego, references someone on stream. This was a great moment that took advantage of our live mocap systems and ability to embed the performer in the live stream.
Many of the challenges we faced included:
- Very short time frame to deliver a highly polished live virtual concert
- Small team on the Wave side (about 3 - 5 people working in engine)
- Having to setup infrastructure for The Mill to integrate their assets, given that their team was not used to Unity HDRP, rather Unreal Engine, and had never worked on something quite like this
- Very complex and dynamic camera systems needed for the 7 performers
- Designing interactivity for the broadcast
My Role
I was the director on the Wave side, where we handled all technical aspects of creating and performing the show live, as well as designing the interactive concept of the show and building out the infastructure in Unity to support the large team and short time frame.
Highlights
- Advised and helped design interactivity for the show during the concept phase with Riot.
- Created a cinematic camera system and an avatar offset animation system for use by animators from The Mill.
- Established a content integration pipeline for a team of close to 20 people.
- Art polish on camera moves for the final shots to remove awkward or unflattering camera angles on the avatars.
- Created the interactions involving a live Zoom video feed of viewers.
- Advised on Unity-specific workflows for rendering, lighting, post-processing, VFX and more.
The Process
The Mill created all of the art assets, and we worked heavily with them during production to hook everything together and ensure we were building things the right way for a live + interactive broadcast. In the early days of production, myself and some others at Wave sat with art directors in the Riot music division and brainstormed ways to turn the epic narrative they had conceived into a format that could be performed as a live and interactive virtual concert akin to what Wave does.
Interactivity
We came up with a team-based mechanic where 3 teams competed against each other throughout the show to see who could click most during various interactions such as “power up Mordekaiser”, with a leaderboard at the end showing the winner. Because of this live element, the show could be repeated multiple times with different outcomes.
We also did our in-game Zoom feature where people in a pre-selected Zoom call get to see their faces appear inside the game
Interactive moment for the audience to power up one of the characters in the show
A live mocap performance where Viego talks to people on the live stream
