Realtime API

The gateway proxies OpenAI's Realtime API for bidirectional voice and multimodal interactions. WebSocket frames pass through in both directions. The gateway meters usage from response.done events and transcription completion events, then applies the API key's billing mode.

https://gateway.assistant-billing.dev/v1/realtime?model=gpt-realtime

The gateway base URL uses the https:// scheme, as with every other endpoint. Browser WebSocket clients must rewrite it to wss:// before calling new WebSocket(); Node.js clients using fetch() with Upgrade: websocket should keep https://.

Only the openai provider is supported for Realtime. Requests routed to any other provider will return 404.

Endpoints

WebSocket upgrade

GET /v1/realtime

Opens a persistent WebSocket to OpenAI's Realtime API. The gateway forwards every frame unmodified in both directions. Pass the model as a query parameter:

https://gateway.assistant-billing.dev/v1/realtime?model=gpt-realtime

If model is omitted, gpt-realtime is used as the default.

Authentication

Because browser WebSocket constructors cannot set custom headers, the gateway accepts the API key through three mechanisms, in priority order:

1. Sec-WebSocket-Protocol subprotocol (browser-friendly, follows OpenAI's own convention):

   Sec-WebSocket-Protocol: openai-insecure-api-key.sk_ab_live_xyz789, openai-beta.realtime-v1
   

   The auth subprotocol is stripped; the first non-auth subprotocol is echoed back in the 101 response.

2. ?api_key= query parameter:

   https://gateway.assistant-billing.dev/v1/realtime?model=gpt-realtime&api_key=sk_ab_live_xyz789
   

3. Authorization: Bearer header (server-to-server clients only):

   Authorization: Bearer sk_ab_live_xyz789
   

The API key must have the realtime endpoint allowed in its permissions (or "endpoints": "*").

Browser client

// Gateway base URL is https://; browsers must rewrite to wss:// for WebSocket.
const baseUrl = "https://gateway.assistant-billing.dev"
const wsUrl = baseUrl.replace(/^https:/, "wss:") + "/v1/realtime?model=gpt-realtime"
 
const apiKey = "sk_ab_live_xyz789"
const ws = new WebSocket(wsUrl, [
  `openai-insecure-api-key.${apiKey}`,
  "openai-beta.realtime-v1",
])
 
ws.addEventListener("open", () => {
  ws.send(JSON.stringify({
    type: "session.update",
    session: { modalities: ["text", "audio"], voice: "alloy" },
  }))
})
 
ws.addEventListener("message", (event) => {
  const msg = JSON.parse(event.data)
  console.log(msg.type, msg)
})

Billing and metering

Usage is metered per turn from response.done events. For each turn the gateway:

1. Extracts response.usage, including input_token_details.audio_tokens, output_token_details.audio_tokens, and cached audio tokens.
2. Calculates cost with the resolved model's own text input, text output, audio input, cached audio input, and audio output rates.
3. Applies the key's billing mode asynchronously. A customer balance reference is deducted when present, and platform wallet mode also deducts from the project wallet. Usage on an operator provider key without a customer balance is logged without a balance deduction.
4. Writes a request log entry with endpoint: "realtime" and metadata.event_type: "realtime_turn".

For transcription sessions, the gateway also meters conversation.item.input_audio_transcription.completed events with usage and logs them as realtime_transcription. These events are billed with the resolved model's rates under the same billing mode.

The session itself also produces these log entries:

All turns share the same sessionId for easy grouping in analytics.

Session timeout

A hard cap limits how long a single WebSocket can stay open. When reached, the gateway closes both sides with close code 1001 (Session timeout).

Response headers

The 101 Switching Protocols response includes:

Metered WebRTC calls

POST /v1/realtime/calls

Send a multipart form with an SDP offer in sdp and a JSON string in session. The session must include type and model; the gateway uses the resolved OpenAI model when it forwards the request. An OpenAI provider key is required. The response is 201 Created with an application/sdp answer and a Location header of the form /v1/realtime/calls/{call_id}.

curl https://gateway.assistant-billing.dev/v1/realtime/calls \
  -H "Authorization: Bearer sk_ab_live_xyz789" \
  -F 'sdp=v=0...' \
  -F 'session={"type":"realtime","model":"gpt-realtime"}'

The gateway attaches an OpenAI sideband WebSocket to the new call and starts its meter before returning the SDP answer. If the sideband connection cannot start, the gateway returns 502 instead of the answer. response.done and transcription completion events follow the same rates and billing mode as a gateway WebSocket session. If the sideband connection drops while the call continues, for example across a gateway deploy, the meter reattaches within a minute and resumes metering; usage during the gap is not replayed. A turn whose charge is refused for insufficient balance ends the call.

Ephemeral client secrets

POST /v1/realtime/sessions

Mints a short lived client secret for browser clients that connect directly to OpenAI over WebRTC. The gateway cannot meter the resulting session. Both /v1/realtime/sessions and /v1/realtime/client_secrets are refused when the request would bill the platform wallet or a customer balance. Use /v1/realtime/calls for a metered WebRTC call. With an operator OpenAI key and no customer balance, only the secret creation request is logged.

curl https://gateway.assistant-billing.dev/v1/realtime/sessions \
  -H "Authorization: Bearer sk_ab_live_xyz789" \
  -H "Content-Type: application/json" \
  -d '{
    "model": "gpt-realtime",
    "voice": "alloy"
  }'

The response matches OpenAI's schema. See OpenAI's Realtime docs for the full payload. /v1/realtime/client_secrets is an alias for the same endpoint.

For browser WebRTC with gateway metering, use /v1/realtime/calls.

Error handling

For /v1/realtime/calls, malformed multipart fields or session JSON return 400; a missing call ID or failed sideband connection returns 502.

Once the socket is open, network errors surface as WebSocket close frames:

• 1001: Session exceeded REALTIME_MAX_SESSION_SEC.
• 1008: A turn's charge was refused for insufficient balance.
• 1011: Internal proxy error (forwarding failure, upstream drop).

Upstream application errors (e.g. type: "error" events from OpenAI) are forwarded to the client unchanged and logged with event_type: "realtime_error".