TGPO: Incentivizing Temporal Awareness in Egocentric Video Understanding

Multimodal large language models (MLLMs) have made strides in visual understanding, but they consistently fail at temporal reasoning in egocentric video. The root cause is that standard training objectives do not explicitly reward temporal ordering, so models learn to exploit spatial shortcuts from individual frames instead of understanding event sequences. This undermines tasks like causal coherence and grounding, which are essential for interpreting first-person narratives.

To fix this, the authors propose Temporal Global Policy Optimization (TGPO), a reinforcement learning with verifiable rewards (RLVR) algorithm. TGPO contrasts model outputs from temporally ordered versus shuffled video frames to derive calibrated, globally normalized reward signals that explicitly favor temporally coherent reasoning. It integrates with GRPO and GSPO to enable cold-start RL training and suppress the spatial shortcut behaviors learned by existing MLLMs. Experiments across five egocentric video benchmarks show consistent improvements in temporal grounding and causal coherence, outperforming prior RL-based video reasoning approaches.

For builders, TGPO offers a simple and scalable pathway to instill temporal awareness without modifying model architecture. The key insight is that contrasting ordered vs. shuffled sequences provides a verifiable reward that directly addresses the temporal blind spot in current MLLMs. This approach is particularly relevant for egocentric video understanding, where event ordering is critical. It suggests that RL-based reward shaping, rather than larger datasets or more parameters, can be an effective lever for temporal reasoning.