RISE: Adaptive Imagination for World Action Models
Abstract
World Action Models (WAMs) enhance autonomous driving planning by integrating predicted future world states into the action generation process. However, current methods allocate a fixed imagination budget to all scenes, leading to inefficient computation and suboptimal performance in varying safety-critical situations. To address this, we introduce RISE (Refining Imagination through SElective Rollout), a system-level adaptive imagination framework that makes sequential decisions—either to continue (ROLL) or stop (STOP)—based on the anticipated planning benefit of further imagination. At each step, a Latent Evaluator estimates the risk level revealed by the current action prefix and the potential improvement in planning if imagination proceeds, while a Rollout Gate weighs this expected benefit against the additional computational cost.
Given that real-world driving logs capture only one realized future, we introduce CounterDrive, a counterfactual dataset containing diverse outcomes and risk levels, to enrich future dynamic modeling and provide localized risk supervision. Every retained sample undergoes expert verification and annotation for trajectory validity, incident onset, and causal category, offering a reusable resource for safety-critical world modeling research.
Experiments conducted on NAVSIM and nuScenes demonstrate that RISE achieves the best overall planning performance while significantly reducing unnecessary rollouts. Additional transfer experiments confirm its plug-and-play generalizability across various WAM architectures, establishing RISE as a robust and efficient solution for adaptive imagination in autonomous driving systems.
Subjects: Computer Vision and Pattern Recognition (cs.CV)
Cite as: arXiv:2608.20430 [cs.CV]
(DOI: 10.48550/arXiv.2608.20430)
via ArXiv CV