Live AI Game Show Judge
Probably most of media companies are wondering how sentiments of audience, actors, show hosts, or other human beings could be used to create added value in the future. One could e.g. recommend content based on sentiment of a VOD service user or automatically create trailer out of the most sentimental moments in a title.
While there are few different ways of studying sentiments of human beings Yle wanted to learn if sentiment can be analyzed from faces in video and we decided to run an experiment where the AI of Valossa Labs judged a live joke battle during Lammen Loiskeet festival in Kajaani, Finland, in the beginning of July 2019.
In this joint project, Valossa deployed facial expression recognition for live video streaming workflow and integrated the cognitive AI to Yle’s live production equipment to measure actors in real-time.
The artificial intelligence model for capturing human facial behaviour has been trained with huge amount of data to recognise even the tiniest expressions of six basic emotions: anger, disgust, fear, happiness, sadness and surprise, and neutral face.
Valossa AI models were first experimented with a popular science video from Yle, where broadcasting stars Viki and Köpi were participating in scary human experiments. Yle published a feature article of emotion recognition and AI. We also created on-screen game show graphics to illustrate the judgements of AI in real-time. In the second stage, live analysis unit was deployed to the Yle SNG truck and tested for live broadcasting.
The AI results were integrated to the live stream via dynamic graphics that showed real-time judgment and running scores for the competing teams based on detected microexpressions. A feat that has not been seen in a live game show before (as far as we know it). The aim was to bring audiences on the edge of their seats with the novel real-time cognitive augmentations.
What did we learn? AI performed game show judge task very well. Technology functioned well during broadcasting and it brought new game show experience for the audience. AI can generate lot of data, e.g. sentiment information separately for each video frame and one must pay attention on how to present and visualize the data for an end user.
For more info in Valossa’s blog: https://valossa.com/live-streaming-game-show/
A video on the experiment shall be published here soon.