FutureWorld A Plug-and-Play Causal Next-State Predictor for Interactive Video World Exploration

Guangyuan Li^1,2 Kelu Yao^3,* Shuolin Xu⁴ Xinlei Yu² Wei Xing¹ Lei Zhao^1,* Peng-Tao Jiang⁵ Xiaogang Xu¹ Xiaobin Hu^2,*

¹Zhejiang University ²National University of Singapore ³Zhejiang Lab ⁴Bournemouth University ⁵vivo BlueImage ^*Corresponding authors

Read the paper Watch videos Code

Comparison of historical retrieval and next-state prediction on a ruined street. History reuse accumulates distortions; the plug-in predictor keeps the scene structure. — Reusing generated history can carry blurred details, geometric distortion, and semantic drift into later chunks. FutureWorld predicts the next latent from the current state and the next action, then uses that prior to guide the video generator.

Interactive video world models aim to follow user actions while maintaining scene consistency over extended interactions. History-based methods reuse generated latents to preserve scene content, but three limitations of this reuse contribute to error recirculation. (1) The latent generated at each interaction is retained as history, potentially containing blurred details, geometric distortions, or semantic drift. (2) Although retrieved historical content matches the current scene, it can still contain errors that are carried into the next chunk. (3) New outputs then become history for later interactions, potentially reinforcing the same errors through repeated reuse.

To limit this cycle, we predict the future scene state from the current generated latent and the next action before generating the next interaction’s video chunk. This future-state prior guides the next chunk, with the aim of reducing the errors carried forward from history. We propose a plug-and-play Causal Next-Chunk Predictor (C-NCP) with cross-model adaptability and prediction-guided modulation. C-NCP is integrated into four base generators and evaluated across three benchmarks for interactive long-video generation and action-conditioned world modeling. The experiments show reduced error accumulation and improved long-horizon consistency across generation settings.

01 — Model

One predictor, four frozen generators.

Causal Next-Chunk Predictor

VAE-specific input adapters map each generator’s latent into a shared space. A causal spatiotemporal trunk forecasts the next chunk, and an output head returns it to that model’s latent channels.

Two-stage training

Stage I mixes predicted and ground-truth chunks so early forecasts stay learnable. Stage II drops the ground-truth addition and trains the same predictor on its own inter-chunk feedback. Within each chunk, teacher forcing remains.

Autoregressive inference

At each interaction the predictor reads the current latent and the next action. Modulation writes that future-state prior into a frozen autoregressive video DiT before the new chunk joins history.

02 — Results

The same gain, on every base model.

All reported videos are 60 seconds. On RealEstate10K-test, Sekai-val, and WBench, adding C-NCP improved every reported metric for all four generators. Scores below are copied from the manuscript tables.

Qualitative comparisons over 60-second interactive rollouts with Matrix-Game 3, HY-World 1.5, LingBot-World Fast, and SANA-WM. Each pair shows the original generator above and the predictor-augmented result below. — Qualitative comparisons over 60-second interactive rollouts with four base generators. Each pair shows the original generator above and its predictor-augmented counterpart below, with six frames from left to right. Click the figure to enlarge it.

User study · overall preference

One hundred participants each compared all 20 video pairs, five pairs for every base generator. They preferred the predictor-augmented video on scene consistency, action adherence, temporal consistency, visual quality, and overall preference. The margin is larger on consistency and action following than on visual quality.

Base modelBase + C-NCP

03 — Demonstrations