AWS teams with FOX Sports for AI-powered highlights
Details
Editor
| 25 July 2023
Driven by fans’ expectations that digitisation and the continued advancement of broadcast delivery technology should lead to a dynamic viewing experience, FOX Sports has revealed how it has deployed Amazon Web Services (AWS) AI-powered technology for its Catch Up With Highlights feature in broadcast coverage.
AWS fox 25July2023
Whether streaming a game live or recording to view on-demand, AWS says audiences are increasingly looking for standout elements in the coverage of their favourite teams, from compelling graphics and data insights to timely replays and highlights. When the games in question are some of the most anticipated in sports, the stakes are even higher.

Using Amazon’s open source Media Replay Engine (MRE) framework for automating video clipping and replay generation, alongside AWS infrastructure and solutions, Catch Up With Highlights is an end-to-end solution that can ingest broadcast streams, identify valuable match moments through AI-trained computer vision, and then encode delivery streams with metadata across user devices. With AWS already the foundation of Fox Corporation media workflows, the development team was able to stand up the complex new feature in about eight weeks.

The Catch Up With Highlights workflow includes AWS Lambda event-driven serverless compute, and computer vision with Amazon Rekognition. Sports data is housed in an Amazon DynamoDB NoSQL database, then piped into MRE. Live video is quickly delivered to viewers via AWS Elemental MediaConnect, with videos stored in Amazon Simple Storage Service (Amazon S3), and Amazon ElastiCache for caching live clips.

The feature saw use in the US broadcast coverage of the FIFA World Cup Qatar 2022 for which 31 recap videos were published autonomously across 61 matches during the Men’s World Cup in 2022.Catch Up TV is currently available for the FIFA Women’s World Cup, MLB and USFL broadcasts; FOX is bringing the feature to football broadcasts in the autumn of 2023 and more personalised content is on the horizon.

“Soccer is often referred to as ‘the beautiful game’ because of the flowing and artistic style of play and objectively it’s a simple sport: score more goals than the opposition. However, training models and developing logic around a game that lacks set patterns is anything but simple. To further compound the complexities, it was our first time publicly releasing a platform feature that ingests multiple concurrent streams, truncates clips for inference, contextualises and optimises each segment, and transcodes and delivers directly into our CMS in a matter of moments of an event occurring,” explained FOX’s vice president of data products and machine learning (ML) strategy, Phil Martin.

“We’ve really rallied around the idea of revolutionising fandom. We believe that sports should be enjoyable and accessible for everyone, which is why we've designed our platform to make it easy to engage with and enjoy.We understand that the level of play at events like the World Cup is exceptional, and we wanted our feature to reflect that standard. To achieve this, we developed a cutting-edge AI technology to create a highlights engine that can dynamically retell the game's story in real-time, highlighting key moments and events as they unfold. This ensures that our users can stay up-to-date with the action and experience the excitement of the game, no matter where they are…AWS has been a trusted partner of ours and together we were able to take another step forward in revolutionising fandom with data all while on the world’s biggest sporting stage.”