Frontier vision-language models (VLMs) are capable at UI navigation, but their high inference costs make them impractical for everyday use. Smaller models trained with supervised fine-tuning (SFT) or offline reinforcement learning often memorize static trajectories; as a result, execution is brittle and the agents cannot recover from errors. To address this, the paper proposes an online reinforcement learning framework that trains UI navigation agents directly on live iOS applications.

The framework decouples the agent into two specialized components: a Planning agent and a UI Grounding model. An LLM-as-a-Judge provides a comprehensive reward signal by combining dense, step-level feedback for intermediate progress with a terminal reward for overall task completion. The agent is optimized with Proximal Policy Optimization (PPO) using step-level advantage estimation, so updates are driven by progress on individual steps rather than only by the final outcome.

The paper also introduces GiGPPO, a neighbor-conditioned value estimation method for the critic. GiGPPO stabilizes value learning by leveraging reward signals from visually similar states across concurrent trajectories, which reduces variance when many agents explore in parallel. On the infrastructure side, the training paradigm is supported by an efficient system that orchestrates hundreds of concurrent virtual iOS devices.

Evaluation is performed on UINavBench, iOSWorld, and AndroidWorld. Online-RL-trained agents substantially outperform both SFT and terminal-only RL baselines while accelerating training convergence. Notably, the 8B RL agents achieve nearly a sevenfold improvement over the SFT baseline in out-of-distribution generalization to Android environments, despite being trained on iOS. On real iOS applications, these 8B agents outperform Gemini-3.5-Flash and tie with Gemini-3.1-Pro.

Overall, the results indicate that dense online RL with separated planning and grounding components can produce UI navigation agents that are more robust and more generalizable than models that fit static trajectories, and that scalable virtual-device infrastructure is a practical enabler for this training approach.