DeepMind has unveiled an AI agent capable of performing tasks in unfamiliar 3D games

Google DeepMind has demonstrated a new artificial intelligence agent called SIMA (Scalable Instructable Multiworld Agent) that can understand and perform tasks in 3D games based on natural language instructions.

Here's What We Know

DeepMind's new development is the first to show the AI's ability to follow commands in a wide range of game worlds and environments it has not previously encountered. Nine commercial games were used to train SIMA, including No Man's Sky, Teardown, Valheim and Goat Simulator 3.

Researchers recorded players' keyboard and mouse actions to train SIMA to perceive natural language instructions and control the game character accordingly. The agent can perform tasks of varying levels of complexity, from navigation to interacting with objects and the user interface.

In tests, SIMA demonstrated its ability to effectively generalise its experience in some games to other unfamiliar environments. Its performance in unknown games was almost the same as an agent trained specifically on those games.

However, language instructions from the user are necessary for SIMA to perform optimally. Without them, the agent acted incoherently and chaotically.

DeepMind sees great potential in creating universal language-driven AI agents based on SIMA. In the future, the company expects to develop systems that can safely perform a wide range of tasks based on human instructions in virtual and real-world environments.

Source: Engadget