Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation

Chenjian Gao1,2*, Zhihao Hu2*, Jianqi Ma2, Jun Zhang2†, Weidong Zhang2, Tianfan Xue1†
1MMLab, The Chinese University of Hong Kong 2Tencent AIPD

*Equal contribution   †Corresponding authors

Paper Code

Train on 5 seconds. Generate far beyond.

Real-score predictions for an owl rollout: video-level DMD retains historical artifacts, while RMD independently provides cleaner appearance targets.
Video-level DMD can retain artifacts from the generated history. RMD scores each chunk independently, providing a clean target despite degraded context.

TL;DR

Rollout-Marginal Distillation (RMD) computes DMD supervision independently for each generated chunk rather than jointly over the video. Real and fake scores are evaluated without temporal context, providing a cleaner visual-quality target. Video-level refinement then restores temporal coherence. Trained on 5-second rollouts, RMD helps autoregressive video generators preserve visual quality beyond the training horizon.

Rollout-Marginal Training

A causal generator produces a rollout. Adapted score networks supervise chunks independently, and gradients flow through differentiable causal replay.
Independent appearance supervision, followed by video-level refinement. Teachers and critics are used only during training.

Long-Horizon Generation

Multi-Prompt Generation

Comparisons

BibTeX

@article{gao2026rmd,
  title   = {Rollout-Marginal Distillation for Long-Horizon Autoregressive Video Generation},
  author  = {Gao, Chenjian and Hu, Zhihao and Ma, Jianqi and Zhang, Jun and Zhang, Weidong and Xue, Tianfan},
  journal = {arXiv preprint arXiv:2609.37925},
  year    = {2026}
}