GPT‑Live, OpenAI‘s third-generation voice system, removes the turn detector from the audio path, enabling full-duplex listening and speaking. It can consult frontier models like GPT‑5.5 asynchronously without interrupting the conversation. The system was built over six months with a new architecture optimized for low latency, streaming audio directly into the voice model and outbound speech back to the user while delegation happens on a separate asynchronous path.

Previous turn-based voice architectures relied on a turn detector to decide when to start inference, which introduced latency and could either cut off the user or feel sluggish. Cascaded systems (STT, LLM, TTS) added further delay. Speech-to-speech models improved by processing audio directly but still required turn detection. GPT‑Live puts the voice model in control of the conversation: audio flows in and out of the model, while deeper reasoning and tool use occur asynchronously.

The architecture separates media flow from application logic. The media frontend, written in Go, ensures reliable frame delivery; its p95 matches the previous system’s p50. WebRTC provides the transport foundation, with enhancements: WARP (open specifications to reduce protocol handshakes, already adopted by libwebrtc and Pion and advancing through the IETF’s TSVWG working group) and Instant Connect (pre-negotiating SDP parameters to remove signaling from the critical path). Together, these allow a session to start with a single UDP packet.

Stateful inference is managed with a seamless handoff mechanism. When a model instance needs to be replaced (e.g., for context compaction), a replacement instance is warmed in parallel, prefilled with current session context, and cut over when ready. Dynamic context compaction reduces accumulated context to fit within the model’s context limit; it is treated as another managed transition to avoid media interruption, supporting long-running calls.

Delegation to frontier models is optimized by prefilling the frontier model with initial conversation context at session start, maintaining the inference session, and using prompt caching. The full delegation loop (routing, prompt processing, inference, tool calls) is treated as part of the responsiveness budget. The system derives discrete turns from continuous speech, maintaining a speculative view for the UI and an authoritative record for logging, using partial transcripts and timing signals to infer speaker attribution. Speaker overlap is handled by distinguishing brief acknowledgements from substantive interjections.

The system was tested silently with production traffic, which revealed capacity issues (CPU-side stream handlers saturated before GPU throughput), geography dependencies (routing to distant capacity adds delay), and long-running session problems (memory pressure, reconnects, shutdown handshake races). This led to improved observability, granular telemetry, staged ramps, and the ability to isolate paths quickly.

The architecture now powers ChatGPT Voice and is expanding into agentic coordination, with a forthcoming GPT‑Live API. The article emphasizes that the principle of keeping the voice flowing is fundamental, and the system is designed to be a platform for realtime interaction across devices and modalities.