From split screen fan chat to CG crowds and Zoom matrices, the pandemic has super served virtual fan engagement.
Much more than the action on the pitch, court or rink, the shared emotional rollercoaster that accompanies it makes live sport so empowering. So when professional sports resumed behind closed doors, sportscasters and major leagues endeavored to bring fans with them.
Virtual fan experiments gravitated in early May from rudimentary cardboard cutouts in the stands of soccer matches to pitchside LEDs displaying thousands of fans on Zoom as a means of incorporating supporters into the game and giving some semblance of home team atmosphere.
The virtual concept has increased in sophistication. When the NBA resumed the 2019–20 NBA regular season and the 2020 playoffs in July, it set up a “bubble” isolation zone at ESPN’s Wide World of Sports Complex in Walt Disney World. The intent was to deliver a home game feel for teams and fans.
Three courts were created in the complex, wrapped by giant LED screens and topped by graphics displays to fill the void left by the lack of fans. A database of music, audio cues and graphics ordinarily used by the teams in their arenas was adapted to customize the courts for the designated ‘home team’. A bank of in game reactions was also developed to support big plays and help support the natural momentum of the game. All of this was in addition to the 400 virtual fans that were displayed during normal gameplay.
“Major leagues and broadcasters want a more engaging environment for their fans and that often comes down to more shots, more angles served to second and third screens.”
—MICKEY MILLER, CEO, VISLINK
Fuse Technical Group was behind the installation, which included 13 disguise gx 2c media servers with disguise Designer software. The team used disguise’s Sockpuppet workflow to enable them to respond quickly to game play and the evolving nature of the overall game experience.
Since the NFL’s return in September, select games are being accompanied by a Fan Mosaic of 30 home team fans displayed on stadium screens and broadcast. These fans see a dual-screen display of the live game next to a gallery view of fellow fans. Each fan video feed is isolated and mixed into the Fan Mosaic display. NFL sponsor Bud Light has created a special Showtime Cam that puts the spotlight on fans from the Fan Mosaic and fan tweets on LED screens installed at each end zone after a touchdown.
Since July, Fox Sports’ MLB broadcasts have carpeted stadiums like Wrigley Field in CG crowds. The technology unites camera tracking technology from Sports Media Technologies, virtual graphics designed by Silver Spoon and realtime graphics processing from Pixotope based on Unreal Engine 3D. The result allows thousands of fans to be dynamically created, controlled and synched with the live pictures with between 2 and 8 frames delay.
“Fox acknowledges this is not real, but what they’re doing is giving the viewer a much better experience,” says Tom Shelburne, Director of Sales & Business Development, Pixotope — part of The Future Group, whose technology processes and synchronizes the individual video-conferenced feeds.
“We can add 42,000 unique individuals. We can change the color of their clothing, make them sit, stand, jump, cheer, do high fives or wave. We can alter the density of the audience. Even the shadow of the sun during the live game is taken into account to light the CG crowd. Anyone can deliver a virtual experience, but not everyone can make it photoreal. If you move the camera around the field and the crowd is misaligned even slightly, the illusion falls apart.”
Journey to The ThunderDome
WWE, however, is acknowledged as the gold standard when it comes to virtual fans. Perhaps that’s because of the sport’s normally intimate relationship with supporters cheering and booing every smackdown.
The journey to the ThunderDome, WWE’s virtual videoconferencing crowd experience, started at the NFL Draft. Virtual event and mixed reality specialist The Famous Group (TFG) was slated to do several large-scale activations during the Draft in Las Vegas, including a projection mapping show on the Bellagio fountain. Plans segued into a virtual Draft show when the pandemic hit.
“Over several weeks, this was distilled down to delivering live fans into NFL commissioner Roger Goodell’s basement,” explains Andrew Isaacson, EVP, TFG. “There was definitely some pioneering work to do, but we were able to deliver 1,000 NFL fans from around the world into the virtual Draft show with very low latency. There is also a lot of audience management that goes on behind the scenes. This includes directing individual people during the live show, moderation and removal of problematic people (there are always bad guys trying to get in!), as well as making sure the fans are decked out in all their team gear so that the experience looks visually awesome.”
TFG took the learnings from the NFL Draft and designed a new system to deliver thousands of fans into the WWE ThunderDome. The components of this system were developed rather quickly and are still ongoing but are designed for live broadcast and are considerably different than TEAMS or Zoom.
“We can add 42,000 unique individuals. We can change the color of their clothing, make them sit, stand, jump, cheer, do high fives or wave. We can alter the density of the audience. Even the shadow of the sun during the live game is taken into account to light the CG crowd. Anyone can deliver a virtual experience, but not everyone can make it photoreal. If you move the camera around the field and the crowd is misaligned even slightly, the illusion falls apart.”
—TOM SHELBURNE, DIRECTOR OF SALES & BUSINESS DEVELOPMENT, PIXOTOPE
Isaacson describes it as a combination of a custom registration system in the cloud, a robust low latency video cloud service platform and a custom touch screen system that manages each individual virtual seat in the ThunderDome. This is coupled with moderation tools that allow audience managers to shuffle people around in order to feature certain fans in prime camera locations as well as remove any misbehaving fans. There’s also a VIP room for special guest celebrities.
A quarter of a million fans registered across 30 live shows with 5,000 participating in each. Eighty percent of fans who go through the experience return for more, says Isaacson
“This has been a huge success for WWE and is a real showcase for the future of live events production even as fans return,” Isaacson says.
Europe Goes for “Authentic” Approach
In Europe, things are a little different. The English Premier League — the biggest international sports franchise — clubs have eschewed virtual fans and instead provide greater sponsorship opportunities to their partners. Others have given a nod to a club’s history and fan culture with banners representing famous victories, chants and cities. Other sports like Premiership Rugby and cricket have also been conservative.
“There may well be a prosaic reason,” says Carlo De Marchis, Chief Evangelist at sports tech provider Deltatre. “Put simply, there is the initial hurdle of having the right technology in place in a very short period of time. U.S. broadcasters may have been more prepared for this. Also, there may be a difference in approach and outlook between the markets. In the U.S. there is a lot of ‘mixed-reality’ solutions already in broadcast, whereas in Europe there has been a little more resistance to this.”
One enhancement that European sports have had in common since behind-closed-doors resumption is giving viewers the option to opt in or out of the artificial crowd noise sourced from EA Sport’s FIFA video game. Given the choice of enhanced audio or ‘purist’ sound, 70-80% of BT Sport viewers are choosing the former.
“Sophisticated well-executed virtual fan noise can add a layer of realism and help to give the impression of ‘normality’ with noises tweaked to reflect action on the field; a bad tackle, an easy opportunity to score missed, a penalty appeal,” says De Marchis. “On the other hand, with ‘naked’ audio you can get a sense of player interaction, as well as the nature of instruction from coaching staff, and the genuine reactions on the field provided they’re loud – or perhaps angry – enough. However, others have said that this option can give the feeling of a training game, and not a top-level, intense competition.”
As Crowds Return, Will Virtual Fans Fade Away?
As fans are being allowed back into stadiums in limited numbers, a new challenge is appearing. Will broadcasters continue to use virtual noise, or hope that those that in attendance contribute enough to an authentic atmosphere?
“The value proposition right now is bringing fans into the events where there are no fans,” says Greg Harvey, Partner, TFG. “The value proposition will change when fans are back to how do we monetize the platform and scale the platform.”
Some innovations that have been introduced over the past year will become a fixture. “Specifically, the fan walls that have connected communities of supporters of a team from all around the world,” says De Marchis. “Perhaps there’s more room for fans to directly connect to fans other fans at home before and after a game. If you’re living in a city hundreds of miles from your team’s stadium, now you could have a direct connection to someone in or near the stadium.”
“There was definitely some pioneering work to do, but we were able to deliver 1,000 NFL fans from around the world into the virtual Draft show with very low latency.”
—ANDREW ISAACSON, EVP, TFG
Harvey agrees: “The exciting part of this is the ability to bring fans from all over the globe into an event. You are no longer limited by physical seating or capacity. Imagine being a sports team with a global fan base and able to bring an additional five to ten thousand fans into a live game and give them unique access,” he says.
“It is a very exciting concept. It’s important to keep in mind this is really in its infancy, but we could see a new world were virtual season tickets become standard alongside physical season tickets.”
Mickey Miller, CEO at Vislink believes fans want more point of view shots and data as lockdown eases. He says, “Major leagues and broadcasters want a more engaging environment for their fans and that often comes down to more shots, more angles served to second and third screens.”
Vislink works with MotoGP organizers Dorna Sport to supply 128 wireless camera shots (80 on bikes alone) for each race delivering super intimate shots.
“Even with stands full of fans, the home environment will be key and the ability for fans to watch views of their favorite player or hole or shot will change how we view sport.”
Full live production workflows in the cloud combined with ML-driven automation promise to finally deliver hundreds of customized versions of a sportscast. Technology being tested by Amazon Web Services called Cloud Digital Interface means that the potential number of variations from a single event is limitless.
“From a control point of view, you can do this with a much smaller number of people who could be based anywhere,” says Gareth Capon, CEO of Grabyo, which has hooked its production tools into CDI. “Using CDI in the cloud offers significantly more flexibility at a much lower price point than traditional host broadcast production out of a truck.”
Holographic Cheerleaders
The faster connections and data speeds of 5G network rollout means more fans can be delivered virtually to all kinds of digital experience through their mobile devices both inside and outside of an event or television broadcast. For example, the ‘Watch Together’ second screen solution rolled out by BT Sport and Sky Sports will only be enhanced with greater bandwidth and millisecond latency.
Other advances include real-time rendering from game engines like Unreal Engine, computer vision, cloud solutions of all kinds from video delivery to GPU rendering to production tools, volumetric scanning, AR and mixed reality tech.
“Live events and virtual fans are the perfect showcase for all this amazing new technology to deliver incredible visual immersive real-time experiences,” says Harvey. “There is a real merging of the physical world and the digital world happening. This is just the tip of the iceberg for what’s is possible.”
He adds: “The cool thing about MR and AR is that we can bring fans outside of the two-dimensional video display and into the live physical space for a richer interaction with the venue. In the next three to five years you will probably see volumetric holograms of fans dancing on the field and the Super Bowl half-time show. This is truly an exciting time!”