NVIDIA has added four open-weight checkpoints to its Cosmos 3 model family, all post-trained examples of Cosmos3-Nano with 16B trainable parameters, released under OpenMDW1.1 for commercial and non-commercial use. They target interactive world simulation and multi-sensor multi-view driving scenes: Transfer-Auto generates multi-view driving data with varied weather, lighting, and time of day; Sim-Auto streams synchronized camera and LiDAR observations; Sim-Transfer turns simulator controls and a text prompt into photorealistic video; Sim-Bimanual predicts video from robot or camera actions.

Cosmos3-Nano-Transfer-Auto is a bidirectional model for offline synthetic driving data generation. It generates synchronized video across up to 11 camera views, including fisheyes, and supports joint RGB-LiDAR generation. Per-camera World Scenario Map (WSM) controls specify scene structure and traffic motion, while text prompts guide appearance; optional initial RGB images anchor scene appearance, and joint LiDAR generation uses aligned HD-map range-map controls. Transfer-Auto can be post-trained on a custom sensor rig and domain, then used directly for offline data generation or causally trained and distilled into a streaming variant. Cosmos3-Nano-Sim-Auto is that distilled streaming variant: it generates successive small chunks of synchronized camera video and LiDAR in a few denoising steps, using generated history and updated driving controls for fast, interactive rollouts. Closed-loop interaction requires a simulation runtime and WSM renderer to update controls from driving inputs; for a new sensor rig or domain, the recommended path is to adapt Transfer-Auto first and then distill it into Sim-Auto.

Cosmos3-Nano-Sim-Transfer uses spatial controls—depth, segmentation, edges, and blur—plus a text prompt to render color observations from a physics simulator’s state; physics and ground-truth state remain with the simulator. Cosmos3-Nano-Sim-Bimanual simulates how an environment evolves in response to a bimanual robot’s actions, supporting interactive two-arm simulation for YAM and AgiBot embodiments, controlled by a policy or joysticks. Its mixed-action example uses 59 control values padded to 64 during training, and the action schema defines coordinate frames, rotations, gripper calibration, validity masks, and control domains. Timing must align: T transitions need T+1 observations, and released inference actions are already normalized.

The models use Cosmos 3’s Mixture-of-Transformers backbone, which connects separate text-processing and visual-generation networks through attention. The generator predicts each short segment from the prompt, available controls, and generated history; denoising refines compressed video latents into a segment. Temporal-causal attention reads available history and controls while blocking future target observations, and a key/value cache retains attention information for the next prediction. After the initial image, each latent time step represents four RGB frames; Transfer predicts one step per chunk, while the optimized Bimanual setup predicts two. The four-stage training recipe covers bidirectional adaptation from complete recorded clips, teacher forcing, consistency distillation to generate each chunk in four denoising steps, and self-forcing on the student’s generated history. Self-forcing matters because a slightly mispredicted object becomes context for later frames, so errors can compound; distillation and caching reduce observation cost.

The post-training recipes start from Cosmos3-Nano or compatible base weights and cover all four stages; full training checkpoints should be kept to resume, while exported inference weights omit optimizer and other training state. Evaluation should use held-out episodes, since a successful short run verifies execution, not convergence. Requirements are Linux, an NVIDIA GPU, Docker, and NVIDIA Container Toolkit; the published vLLM-Omni setup was tested on one GB200 and is not a minimum-memory specification. The stated motivation is to give robotics and autonomous-driving researchers an open foundation for building responsive simulated worlds, while acknowledging the work ahead: measuring where the models faithfully support policy learning and evaluation, understanding their limits, and improving where they fall short.