YLE Sandbox

Live AI Game Show Judge

Probably most of media companies are wondering how sentiments of audience, actors, show hosts, or other human beings could be used to create added value in the future. One could e.g. recommend content based on sentiment of a VOD service user or automatically create trailer out of the most sentimental moments in a title.

While there are few different ways of studying sentiments of human beings Yle wanted to learn if sentiment can be analyzed from faces in video and we decided to run an experiment where the AI of Valossa Labs judged a live joke battle during Lammen Loiskeet festival in Kajaani, Finland, in the beginning of July 2019.

In this joint project, Valossa deployed facial expression recognition for live video streaming workflow and integrated the cognitive AI to Yle’s live production equipment to measure actors in real-time.

The artificial intelligence model for capturing human facial behaviour has been trained with huge amount of data to recognise even the tiniest expressions of six basic emotions: anger, disgust, fear, happiness, sadness and surprise, and neutral face.

Valossa AI models were first experimented with a popular science video from Yle, where broadcasting stars Viki and Köpi were participating in scary human experiments. Yle published a feature article of emotion recognition and AI. We also created on-screen game show graphics to illustrate the judgements of AI in real-time. In the second stage, live analysis unit was deployed to the Yle SNG truck and tested for live broadcasting.

The AI results were integrated to the live stream via dynamic graphics that showed real-time judgment and running scores for the competing teams based on detected microexpressions. A feat that has not been seen in a live game show before (as far as we know it). The aim was to bring audiences on the edge of their seats with the novel real-time cognitive augmentations.

What did we learn? AI performed game show judge task very well. Technology functioned well during broadcasting and it brought new game show experience for the audience. AI can generate lot of data, e.g. sentiment information separately for each video frame and one must pay attention on how to present and visualize the data for an end user.

For more info in Valossa’s blog: https://valossa.com/live-streaming-game-show/

A video on the experiment shall be published here soon.

Related works

  • Kielikylpy 2.0

    Kielikylpy 2.0 is a spoken language training game where students learning Finnish language take part in an interactive story using their voice. The purpose is to encourage students to use and practice their Finnish in the safety of their own home and device. In the story a friend and the user are heading to a […]

    Crowdsourced content production with media management

    BCaster delivered  an app used during the Sideways festival. The app consists of regular information sharing pages as well as possibility to create content. The content creation was done with built-in camera functionality and when the user took a photo or video, it was automatically uploaded to cloud service. The User Generated Content (UGC) was […]

    Paradise

    Paradise is a VR experience that takes you to Enewetak Atoll in the middle of the Pacific Ocean. You can explore the site where the US carried out a large number of nuclear tests in the 1940’s and 1950’s, and see what life on Enewetak looks like today. You will also witness a nuclear blast, […]

    Your Face When

    The goal of the project was to try how a Facebook lens can be used in storytelling. When using such a lens the user of application can experience scripted story in his own face and the team wanted to explore what this means in the context of writing a story that is supposed to create […]