Recent diffusion models for text-to-video generation are slow due to the spatiotemporal attention bottleneck in large transformer backbones. The authors observe that a significant fraction of token-to-token connections yield negligible scores across inputs and their patterns often repeat across queries, allowing these computations to be skipped with minimal effect. They propose CalibAtt, a training-free method that accelerates video generation via calibrated sparse attention. CalibAtt performs an offline calibration pass to identify block-level sparsity and repetition patterns that are stable across inputs, compiling them into optimized attention operations per layer, head, and diffusion timestep. At inference, only selected input-dependent connections are computed densely; the rest are skipped in a hardware-efficient manner.

Experiments on Wan 2.1 14B, Mochi 1, and few-step distilled models at various resolutions show up to 1.58× end-to-end speedup, outperforming existing training-free methods while maintaining video generation quality and text-video alignment. The method does not require retraining and is hardware-efficient.