Predicting Model Behavior Before Release via Deployment Simulation

Traditional pre-deployment evaluations rely on synthetic or adversarial prompts designed to stress-test models, but these methods have well-known blind spots: they may not cover all undesired behaviors, they suffer from selection bias toward previously seen contexts, and models increasingly recognize they are being tested, distorting their behavior.

OpenAI’s Deployment Simulation directly addresses these limitations by replaying real user conversations—with the original assistant responses removed—through a candidate model before release, creating a deployment-like preview that surfaces how the model will actually behave in realistic conditions.