Cosmic Dreams: Lucid Dreaming in Latent Space via Temporal Context Windows
Generative world models allow robots to simulate possible futures for planning, but modeling rigid-body dynamics in compressed latent spaces is difficult. Standard deterministic methods produce blurry predictions, while probabilistic ones add noise. By systematically testing 11 architectures, the authors show that stable and sharp predictions depend on temporal context, not model size. Their lightweight ResNet-Rollout model, with a 4-frame context window, achieves a Root Mean Square Error of 71.08 pixels—a 57% reduction compared to the best 84-million-parameter baseline—while training in 15 minutes. The results indicate that addressing partial observability of velocity via history windows is more effective than scaling model size for deterministic physics modeling.