Red-teaming is essential for finding model vulnerabilities, but human red-teaming doesn’t scale. Current robustness benchmarks have been saturated by the latest models, creating a bottleneck where safety improvements can’t keep pace with capability advances. OpenAI needed a method to scale vulnerability discovery and incorporate fixes during training, not just after deployment.

They trained GPT-Red, an automated red-teaming model using self-play reinforcement learning. GPT-Red attacks defender models while they resist, forcing it to discover stronger attacks. This model was trained with unprecedented compute for safety and then used to adversarially train GPT-5.6 Sol, which achieved 6x fewer failures on the hardest prompt injection benchmark compared to models from just four months earlier. GPT-Red also generalized to novel scenarios, achieving 84% attack success on a held-out indirect prompt injection arena versus 13% for humans. Real-world tests against a vending machine agent and a Codex CLI agent demonstrated practical effectiveness.

The key insight is that automated red-teaming unlocks a self-improvement flywheel for safety—today’s models directly make tomorrow’s models more robust, without degrading general capabilities. This approach scales with compute and data, suggesting that safety can keep pace with rapidly advancing model capabilities when red-teaming is automated and integrated into training.