WebSockets keep one connection open so a client and server can send events in either direction. That makes them useful for realtime AI sessions where the user can speak, interrupt, send tool results, or change session state while the model is producing output.

They are not required for every “streaming AI” feature. If a server only sends text tokens to a browser, HTTP streaming or Server-Sent Events (SSE) is usually simpler. Browser voice applications often prefer WebRTC because it is designed for realtime media.

HTTP versus WebSocket

Ordinary HTTP follows request and response:

Client → prompt request
Server → streamed or complete response

A WebSocket is bidirectional after an HTTP upgrade:

Client → audio frame
Server → transcription event
Server → model audio
Client → interruption event
Server → cancel output
Server → tool request
Client → tool result

The OpenAI Realtime API supports low-latency multimodal sessions over WebRTC, WebSocket, and SIP. That is a concrete WebSocket use case, not a reason to convert every model endpoint.

Choose the transport from the interaction

Need	Prefer
complete request/response	HTTP
server-to-client text tokens	HTTP streaming or SSE
browser microphone and model audio	WebRTC
server-to-server realtime event stream	WebSocket
asynchronous completion hours later	webhook + durable job state

Transport choice should follow directionality, media, latency, network infrastructure, and recovery requirements.

Minimal browser example

const socket = new WebSocket('wss://example.com/realtime');

socket.addEventListener('open', () => {
  socket.send(JSON.stringify({
    type: 'session.start',
    clientEventId: crypto.randomUUID(),
  }));
});

socket.addEventListener('message', (message) => {
  const event = JSON.parse(message.data);
  if (event.type === 'response.delta') renderDelta(event.delta);
  if (event.type === 'tool.approval_required') showApproval(event);
});

Use wss:// in production. Authenticate with a short-lived session credential rather than exposing a long-lived model-provider key in browser code.

Design an event protocol

Raw WebSockets only move frames. Your application needs an event envelope:

{
  "id": "evt_123",
  "type": "response.delta",
  "sessionId": "sess_456",
  "sequence": 42,
  "payload": { "text": "partial output" }
}

Version events and define which side may send each type. Sequence numbers help detect gaps or stale messages. Do not execute a tool merely because an incoming JSON object names it; authorize tool calls against server-side user and policy state.

Reconnects and durable state

Connections disappear during network changes, deployments, idle timeouts, and server failures. Clients should reconnect with bounded exponential backoff. The server should restore from durable conversation/job state rather than assuming the previous process still remembers the session.

Decide whether missed events can be replayed. If yes, persist them with sequence IDs or provide an HTTP endpoint that returns current state. A WebSocket is a delivery channel, not the source of truth.

Interruption and backpressure

Voice agents must handle barge-in: when new speech begins, cancel or suppress output that is no longer relevant. Tie audio and text deltas to a response ID so late frames can be ignored.

Backpressure occurs when events arrive faster than a client or downstream model can process them. Bound queues, pause producers where the protocol allows it, and drop only explicitly disposable events such as high-frequency telemetry. Unbounded in-memory queues turn a slow client into a server outage.

Scaling

A persistent connection stays attached to an instance. Horizontal systems need compatible load balancing, cross-instance event delivery, connection draining during deploys, and shared durable state. Cloudflare’s realtime chat tutorial is one current example using WebSockets with Durable Objects for stateful coordination.

Capacity depends on message rate, payload size, TLS, application work, memory, file descriptors, and provider limits. Avoid universal “connections per server” claims; load-test the actual event pattern.

When not to use WebSockets

Do not use them for CRUD, occasional notifications, or one-way text output merely because “realtime” sounds modern. They add connection lifecycle, authentication refresh, replay, load-balancer, deployment, and observability work.

For one-way model output, use streaming AI responses in Node.js. For protocol internals and production scaling, continue with how WebSockets work under the hood. For an end-to-end local voice example, see the Whisper and Ollama voice assistant.

WebSockets are the right tool when both sides genuinely need a live event channel. Reliable realtime AI still depends on durable state, explicit event semantics, bounded resources, and server-side authorization outside that channel.