Google’s GameNGen can generate Doom at real-time speed on Stable Diffusion

Google researchers have created a new AI system that plays Doom without using a game engine. The system, called GameNGen, generates real-time gameplay at 20 frames per second on a single chip.

GameNGen is the first AI to fully simulate a complex video game with high-quality graphics and interactivity. It runs on a single Tensor Processing Unit (TPU) and handles Doom's 3D environments and fast-paced action efficiently.

The system uses a diffusion model to predict each frame of gameplay. This approach allows it to generate real-time interactive experiences in a complex environment without relying on traditional game engine components like manually coded software for game state management and visual rendering.

"We present GameNGen, the first game engine powered entirely by a neural model that enables real-time interaction with a complex environment over long trajectories at high quality,"

The researchers wrote this in their paper published on arXiv.

Doom, released in 1993, has long served as a technological benchmark. People have ported it to various platforms, including microwaves and digital cameras. Now, it's pushing the boundaries of AI capabilities in generating complex, interactive environments in real-time.

Does real-time diffusion generation have future?

GameNGen's development could disrupt the $200 billion global gaming industry. It has the potential to reduce development time and costs, democratize game creation, and enable new genres of dynamically evolving games.

The implications extend beyond gaming. Industries like virtual reality, autonomous vehicles, and smart cities could benefit from this technology. Possible applications include training and testing simulations, operational management, interactive educational experiences, healthcare simulations, and remote work environments.

However, challenges remain. The system requires significant computational power for modern, graphically intensive games. It's currently tailored to a specific game (Doom), and developing a general-purpose AI game engine remains challenging.

Looking to the future, we might see AI-created and AI-powered games that adapt in real-time to player actions and generate content on the fly. This could lead to reduced game development time and costs, further democratizing game creation.

As AI continues to advance, it's blurring the lines between human creativity and machine intelligence. The future of virtual experiences may only be limited by machine imagination.