WebSocket mode — and the cheap win hiding inside it
A persistent socket plus incremental inputs speeds long tool-heavy rollouts by up to ~40%. But there's a transport-independent Phase 0 the SDK can adopt today, because it currently replays the whole transcript every turn.
1What the feature is
WebSocket mode keeps a persistent connection to /v1/responses. Each turn is a response.create event that carries only the new input items plus previous_response_id — not the whole transcript. For rollouts with 20+ tool calls, OpenAI reports up to ~40% faster end-to-end execution. There's also a generate:false warmup, a 60-minute connection cap, and defined reconnect/recovery behaviour.
previous_response_id + incremental input). The second is useful even without the first.2What the SDK does today
The Responses path calls litellm_responses / litellm_aresponses over HTTP, defaults to store=false, and replays the full input every turn. Grep the whole llm.py and you won't find previous_response_id — it's never used (llm.py:1555).
again
3The cheap Phase 0
You don't need a websocket to stop re-uploading the growing transcript. For stored responses, adopting previous_response_id continuation on the existing HTTP path already sends only new items each turn. That's a transport-independent win and the natural first step.
resp_123
+ only new items
4The full gap & its dependency
The websocket transport itself is the heavier lift and the lowest-confidence part:
| Piece | Status / note |
|---|---|
Phase 0: previous_response_id on HTTP | doable now; measure token/latency impact |
| WebSocket transport | blocked on LiteLLM exposing a websocket Responses transport (or a direct-client path) |
Warmup, reconnect, 60-min, previous_response_not_found | Phase 2, after transport lands |
4Quiz
Five questions to check you actually understood it — not trivia. Click an answer to see whether it's right and why.