Unified multimodal models that handle interleaved text and image sequences are still structurally fragmented. The authors identify three common failure modes: discrete tokenization harms visual fidelity, combining causal text generation with iterative diffusion denoising imposes structural asymmetry, and adapting pretrained vision-language models for generation can degrade their understanding. Their key observation is that autoregressive normalizing flows are structurally equivalent to autoregressive Transformers, sharing the causal mask, KV-cache mechanism, and left-to-right order of LLMs. That makes flows a natural fit for unified multimodal generation that is continuous, single-pass, and purely causal.

STARFlow2 implements this idea on the Pretzel architecture. Pretzel vertically interleaves a frozen pretrained VLM stream with a TARFlow stream using residual skip connections, while both streams operate under the same causal mask. The frozen VLM retains multimodal understanding, and the TARFlow stream provides continuous high-fidelity image generation. The result is structural unification under one causal mechanism rather than a hybrid of causal text modeling and iterative denoising.

The design also addresses efficiency and practicality. A deep-shallow flow design and a unified FAE latent space allow cache-friendly interleaved generation: both text and visual outputs can enter the KV-cache directly without re-encoding. This avoids the typical re-encoding overhead when alternating between modalities.

Experiments across image generation and multimodal understanding benchmarks show strong performance, and the authors frame autoregressive flows as a viable foundation for unified multimodal modeling. The work is by researchers from UIUC and Apple.