Moving a production phone agent from OpenAI's Realtime API to GPT-Live: what delegation changes, three failures that produce no error message, and what breaks when turn boundaries disappear.
OpenAI shipped GPT-Live in the API on 10 September 2026. We moved a production phone agent onto it the next day. This is what actually happened, including the parts that cost us two deploys and two rollbacks.
The short version: it is not a model swap. Three of its failure modes produce no error message at all, and the useful details are the ones that only show up on a real phone call.
The Realtime API runs one model that speaks, reasons, and picks tools. GPT-Live splits those jobs. A voice model handles the conversation and hands task work to a separate backend model. The voice model cannot call a tool at all — it can only delegate.
That single sentence is the whole migration. Almost everything below follows from it.
The mechanical changes are easy to enumerate. New endpoint. session.start instead of session.update, and you wait for session.started before sending anything else. Audio events get renamed. Tool definitions move from the session to the delegation config, and results go back as response.item.create followed by response.create — which now means "continue the backend", not "speak".
The hard changes are the ones with no replacement. There are no turn boundaries. No response.done, no barge-in event, no completed-transcription event. Anything in your code keyed to a turn needs a new trigger, and you have to invent it.
This one cost the most time, so it goes first.
GPT-Live decides when to talk based on frame progress across a running audio stream. Send it no audio and it stays silent forever. No speech, no error event, no close. The socket is open, the handshake succeeded, the session ID is real, and nothing happens.
Every instinct says a silent socket means a rejected handshake. Ours did. We checked entitlements, re-read the session config, and redeployed twice before tailing the worker and seeing session.started arriving normally the whole time — followed by a single session.usage.updated reporting zero seconds, and then nothing.
Real calls stream continuously from the phone provider, so this never bites in production. It bites every synthetic test you have. Our health check now streams μ-law silence for the life of the session, and the model speaks in about a second.
If you take one thing from this post: your test harness has to supply silence. A full-duplex model has no concept of "your turn" without a clock, and audio *is* the clock.
This is Cloudflare Workers specific, but the shape of it will be familiar on any runtime with an explicit accept step.
A Workers WebSocket starts dispatching the instant you call accept(). Messages that arrive before a listener exists are gone. On the Realtime API this was survivable, because nothing important waited on the session acknowledgment. On GPT-Live, the greeting and every queued command wait on session.started. Lose it and the call is silent for its entire duration.
Our connect function now returns an unaccepted socket. The caller attaches listeners, then accepts and sends the handshake.
The testing lesson is sharper than the fix. Our first test for this passed with the bug present, because the fake socket's accept() was an empty function — so listener ordering could not possibly matter. We changed the stub to deliver a frame *during* accept(), the way a fast server does. Only then did reversing the order fail.
A mock that cannot exhibit the failure is not covering the behaviour, however many assertions it carries.
Move every tool to the backend and the voice model is left with no idea what is possible. It does not fail loudly. It politely declines things the system can do.
The symptom that gave it away was the agent saying it could not end the call. "Hang up" does not look like backend work, so it never got delegated, and calls simply kept running.
The fix is prompt-level, and the prompting guide is explicit about it: the live prompt has to list what the backend can actually do. We generate that list from each tool's own description rather than maintaining a second copy, and state plainly that ending the call and bringing another person onto it are actions the model must ask for rather than expect to happen.
Anything conversational-but-actionable has this problem. Audit for it before you ship.
Transcripts. Fragments arrive with no item ID and no completion event, and the two speakers overlap freely, because that is the point of full duplex. We buffer each speaker's fragments and flush after a gap. What you get is a display and memory unit, not an authoritative turn — group them loosely and let late text revise earlier rows.
Hanging up. Previously the agent hung up on a turn boundary. Now the hangup waits for queued provider audio to drain, on a timer.
Barge-in. There is no interruption event, and audio already handed to the phone provider keeps playing after the model stops. We drop that tail only once the model has gone quiet *and* a meaningful amount is still queued, so that a deliberate overlap — the model saying "mm-hmm" while you talk — survives. Our thresholds are reasoned rather than measured, which we would rather admit than dress up.
Filler audio. We had a short "thinking" sound covering tool latency. The first instinct was to delete it, since full duplex keeps the conversation moving on its own. We put it back. Duplex sometimes talks over it, and that is still better than an unexplained silence.
The voice layer is billed by duration, not tokens — $0.05 per minute at launch, reported as a cumulative seconds count. They are snapshots, not increments, so keep the newest rather than adding them up. The delegated backend bills separately as ordinary tokens, and you read those from a completion event nested inside a response envelope.
If you meter usage per customer, that is two meters where you previously had one, and the backend one is easy to miss entirely.
On our measurements, first audio landed in about 1.1 seconds against 1.5 to 2.1 seconds for the Realtime API on the same path. Delegated tool calls returned in 26 milliseconds to 1.7 seconds depending on the tool. Conversation feels meaningfully more natural, which is the actual point — the model can acknowledge you while you are still speaking instead of waiting for a turn to end.
We are not finished. We are still chasing sessions that close roughly forty seconds in and reconnect, and we have not yet measured our own barge-in thresholds against recorded calls. Anyone claiming a clean one-day migration of a production voice product is showing you a demo.
The API is good. The mental model is the work.
A dedicated number, realtime conversation, transfers to you when a call needs judgment, and a transcript of every call. Plans start at $49/month.
Get started →