Sony has a prototype AI version of Eloy from Horizon that can talk to humans in real time

Sony has a prototype AI version of Eloy, the protagonist of the Horizon series, demonstrating the capabilities of artificial intelligence in creating interactive characters. The prototype is able to hold conversations, respond to requests, and even interact with the game world.

Here's What We Know

In a video posted on YouTube, Sony's Director of Engineering Sharvin Raghubarjal showed how AI Eloy can answer questions and interact with the player. For example, she tells about her story in the Horizon series.

Eloi AI also demonstrates the ability to "transport" the player to the virtual world of the game. In the video, she accompanies Raghubarjal to The Scalding Spear settlement from Horizon Forbidden West, describing its features.

Sony uses advanced technologies to create Eloy's AI. OpenAI Whisper (speech-to-text) is used for speech processing and GPT-4/Llama 3 for dialogues and decision making. The voice is generated using Sony's own Emotional Voice Synthesis (EVS) system, and facial animation is generated using Mockingbird technology.

The video was removed from YouTube following a copyright complaint from Muso TNT Ltd, a content protection company. The reason for the removal remains unclear, but there is speculation that the video may have been a leak of internal Sony material.

However, The Verge website has a gif with an AI version of Eloy.

Source: The Verge