Diffusion Models Are Real-Time Game Engines

🌈 Abstract

The paper presents GameNGen, a neural model that can interactively simulate the classic game DOOM at over 20 frames per second. GameNGen is trained in two phases: (1) an RL agent learns to play the game and the training sessions are recorded, and (2) a diffusion model is trained to produce the next frame, conditioned on the sequence of past frames and actions. The paper demonstrates that GameNGen can achieve a visual quality comparable to the original game, with human raters only slightly better than random chance at distinguishing short clips of the game from the simulation.

🙋 Q&A

[01] Interactive World Simulation

1. What is an Interactive Environment? An Interactive Environment consists of:

• A space of latent states
• A space of partial projections of the latent space
• A partial projection function
• A set of actions
• A transition probability function

2. What is the Interactive World Simulation objective? The objective is to minimize the distance between observations from the environment and the simulation when enacting the agent's policy.

3. What are the two objectives for training the generative model?

1. Teacher forcing objective: The conditioning observations are obtained from the environment.
2. Auto-regressive objective: The conditioning observations are obtained from the simulation.

[02] GameNGen

1. How is the training data collected for GameNGen? The training data is collected in two stages:

1. An RL agent is trained to play the game, and the agent's actions and observations during training are recorded.
2. The recorded trajectories from the RL agent are used as the training dataset for the generative diffusion model.

2. How does GameNGen condition the diffusion model on actions and observations?

• For actions, an embedding is learned for each action and concatenated to the input.
• For observations (frames), the frames are encoded into latent space using an auto-encoder and concatenated to the input.

3. How does GameNGen mitigate auto-regressive drift? GameNGen uses noise augmentation, where a varying amount of Gaussian noise is added to the encoded frames during training. This allows the network to correct information sampled in previous frames and preserve frame quality over time.

4. What is the impact of the number of past observations used as context? Increasing the context length improves generation quality, but the improvement becomes marginal after a certain point, suggesting the need for a more sophisticated architecture to effectively leverage longer contexts.

5. How does the agent-generated data compare to random policy data for training the generative model? The agent-generated data performs slightly better, especially for medium-difficulty scenarios, but the random policy data also works surprisingly well, indicating the model can learn meaningful heuristics even from random exploration.

[03] Results

1. How does GameNGen perform in terms of image quality metrics?

• Single-frame PSNR: 29.4, comparable to lossy JPEG compression
• Single-frame LPIPS: 0.125
• Video quality (FVD): 38.8 for 16-frame sequences, 55.2 for 32-frame sequences

2. How do human raters perform in distinguishing the simulation from the real game? Human raters only choose the real game over the simulation in 58-60% of the cases for short clips, indicating the simulation quality is close to the original game.

3. What are the key limitations of GameNGen?

• Limited memory: The model only has access to around 3 seconds of history, limiting its ability to maintain long-term game state.
• Differences between agent and human play: The agent's exploration and behavior still differs from human players in some cases.

</output_format>