Applications and Extensions

Real-World Applications

World Models have been applied to various domains:

Robotics

• Sim-to-Real Transfer
• Manipulation Tasks
• Tool use and dexterous manipulation

Autonomous Driving

• Scene Understanding
• Decision Making
• Safe trajectory planning

Game AI

• Procedural Content Generation
• NPC Behavior
• Adaptive opponents

Recent Extensions

DreamerV1/V2/V3

The Dreamer series extends World Models with actor-critic training and discrete latent spaces.

PlaNet

Planning Network combines latent dynamics model with model predictive control.

IRIS

Imagination with Reinforcement learning In Simulation uses transformer-based world models.

Genie

Generative Interactive Environments learns from unlabeled video data.

Future Directions

• Foundation World Models
• Hierarchical World Models
• Multimodal World Models