Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

Multimodal large language models often use visual chain-of-thought (Visual CoT) to reason about spatial, temporal, and embodied environments by generating intermediate reasoning images.

While this provides an intuitive mechanism for visual foresight, it introduces substantial inference overhead, which is especially problematic for proactive video reasoning.

This work asks whether models can learn to think visually during training while reasoning directly at inference, avoiding the cost of generating intermediate images.

The authors introduce Internalized Visual Thinking (IVT), a post-training framework that jointly optimizes textual prediction and next-embedding prediction over unlabeled videos.

Given a partially observed video, IVT predicts latent representations of future frames together with the target textual answer, encouraging the model to capture motion, object transitions, interactions, and latent intent.

At inference, IVT generates the answer directly without synthesizing or re-encoding future frames.

Controlled studies are conducted across target representations, decoder designs, prediction horizons, data mixtures, training curricula, and predictive objectives.

Results show that IVT improves over text-only post-training across all six evaluation settings while retaining the same efficient inference pathway.

Compared with Visual CoT, IVT achieves comparable or better performance and reduces end-to-end latency by more than 5×.

The findings suggest that explicit pixel-space generation at inference time, as used in Visual CoT, may not be necessary for proactive video reasoning; predictive world modeling can be internalized during training to produce multimodal reasoners that are both more accurate and substantially more efficient.

Beyond Visual CoT: Internalized Visual Thinking for Proactive Video Reasoning

View Original