Overview概览#

You supply a character description and a scene. PopVid returns a live audio-video stream plus a structured event feed describing what the character is doing. 你提供角色描述与场景，PopVid 返回一路实时音视频流，以及一份描述角色当前行为的结构化事件流。

Quickstart快速开始#

Endpoints and environments接入地址与环境

Production and test run on different hosts. Neither address is a constant you should compile into your client — take both from the places below and keep them in configuration. 正式环境与测试环境是两套不同的域名。这两个地址都不是可以写死进客户端的常量 —— 请按下表取值，并放在配置里。

1. On your server1. 在你的服务端

// Both come from configuration. Production is the default below; your test
// environment uses a different host, issued with your test key.
const BASE = process.env.POPVID_BASE_URL ?? "https://popvid.ai/api/public/v1";
const AUTH = { Authorization: `Bearer ${POPVID_API_KEY}`, "Content-Type": "application/json" };

// One call: defines the session, reserves capacity, and mints the credential
// for one browser. It does not start the stream — `session.start` does.
const { session, credentials } = await fetch(`${BASE}/connections`, {
  method: "POST",
  headers: AUTH,
  body: JSON.stringify({
    model: "r2-realtime-v1",
    character: { name: "Aria", prompt: "A calm archivist who speaks in short sentences." },
    scene: { prompt: "A dim library at night, rain on the windows." },
    limits: { max_duration_ms: 600000 },
    credentials_ttl_ms: 600000
  })
}).then((r) => r.json());

// Send `credentials` to your client. Never send POPVID_API_KEY.

// 两者都来自配置。下面的默认值是正式环境；测试环境是另一个域名，
// 随测试密钥一并提供。
const BASE = process.env.POPVID_BASE_URL ?? "https://popvid.ai/api/public/v1";
const AUTH = { Authorization: `Bearer ${POPVID_API_KEY}`, "Content-Type": "application/json" };

// 一次调用：定义会话、预留容量，并为单个浏览器签发凭证。
// 此时仍未启动推流 —— 推流由 `session.start` 触发。
const { session, credentials } = await fetch(`${BASE}/connections`, {
  method: "POST",
  headers: AUTH,
  body: JSON.stringify({
    model: "r2-realtime-v1",
    character: { name: "Aria", prompt: "A calm archivist who speaks in short sentences." },
    scene: { prompt: "A dim library at night, rain on the windows." },
    limits: { max_duration_ms: 600000 },
    credentials_ttl_ms: 600000
  })
}).then((r) => r.json());

// 把 credentials 下发给客户端；绝不要下发 POPVID_API_KEY。

2. On your client2. 在你的客户端

// Use control_url exactly as returned — never a hardcoded WSS host.
const ws = new WebSocket(
  `${credentials.control_url}?session_id=${credentials.session_id}`,
  ["r2.v1", `r2.token.${credentials.control_token}`]
);
const pc = new RTCPeerConnection({ iceServers: credentials.ice_servers });
pc.ontrack = (e) => { videoElement.srcObject = e.streams[0]; };

// Billing starts here, not at session.ready — report it as soon as media is up.
pc.onconnectionstatechange = () => {
  if (pc.connectionState === "connected") send("media.connected", {});
};

let lastEventId = null;

ws.onopen = () => send("session.start", {});

ws.onmessage = async (raw) => {
  const msg = JSON.parse(raw.data);
  lastEventId = msg.id;                       // keep for reconnects

  switch (msg.type) {
    case "session.ready":
      pc.addTransceiver("video", { direction: "recvonly" });
      pc.addTransceiver("audio", { direction: "recvonly" });
      await pc.setLocalDescription(await pc.createOffer());
      await iceGatheringComplete(pc);         // required — see Media
      send("media.offer", { sdp: pc.localDescription.sdp });
      break;

    case "media.answer":
      await pc.setRemoteDescription({ type: "answer", sdp: msg.data.sdp });
      break;

    case "turn.text":       showCharacterLine(msg.data.turn_id, msg.data.text); break;
    case "turn.visible":    hideThinkingIndicator(msg.data.turn_id);            break;
    case "usage.tick":      updateBudget(msg.data.budget_remaining_ms);         break;
    case "session.renewed": updateBudget(msg.data.budget_remaining_ms);         break;
    case "session.ended":   teardown(msg.data.reason);                          break;
    case "error":           handleError(msg.data);                              break;
  }
};

function send(type, data) {
  ws.send(JSON.stringify({ type, id: `c-${Date.now()}`, data }));
}

function say(text) {
  send("turn.submit", { turn_id: `turn_${crypto.randomUUID()}`, text });
}

// Use control_url exactly as returned — never a hardcoded WSS host.
const ws = new WebSocket(
  `${credentials.control_url}?session_id=${credentials.session_id}`,
  ["r2.v1", `r2.token.${credentials.control_token}`]
);
const pc = new RTCPeerConnection({ iceServers: credentials.ice_servers });
pc.ontrack = (e) => { videoElement.srcObject = e.streams[0]; };

// 计费从这一刻开始，不是从 session.ready 开始 —— 媒体一通就上报。
pc.onconnectionstatechange = () => {
  if (pc.connectionState === "connected") send("media.connected", {});
};

let lastEventId = null;

ws.onopen = () => send("session.start", {});

ws.onmessage = async (raw) => {
  const msg = JSON.parse(raw.data);
  lastEventId = msg.id;                       // 保留，用于断线重连

  switch (msg.type) {
    case "session.ready":
      pc.addTransceiver("video", { direction: "recvonly" });
      pc.addTransceiver("audio", { direction: "recvonly" });
      await pc.setLocalDescription(await pc.createOffer());
      await iceGatheringComplete(pc);         // 必需 —— 见「媒体」一节
      send("media.offer", { sdp: pc.localDescription.sdp });
      break;

    case "media.answer":
      await pc.setRemoteDescription({ type: "answer", sdp: msg.data.sdp });
      break;

    case "turn.text":       showCharacterLine(msg.data.turn_id, msg.data.text); break;
    case "turn.visible":    hideThinkingIndicator(msg.data.turn_id);            break;
    case "usage.tick":      updateBudget(msg.data.budget_remaining_ms);         break;
    case "session.renewed": updateBudget(msg.data.budget_remaining_ms);         break;
    case "session.ended":   teardown(msg.data.reason);                          break;
    case "error":           handleError(msg.data);                              break;
  }
};

function send(type, data) {
  ws.send(JSON.stringify({ type, id: `c-${Date.now()}`, data }));
}

function say(text) {
  send("turn.submit", { turn_id: `turn_${crypto.randomUUID()}`, text });
}

That is a complete integration: one HTTP call on your server, one WebSocket and one peer connection on your client. 这就是完整的对接：服务端一个 HTTP 调用，客户端一个 WebSocket 加一个 PeerConnection。

Authentication鉴权#

API keysAPI Key

Long-lived keys are prefixed pk_live_ and authenticate every REST call: 长期密钥以 pk_live_ 开头，用于所有 REST 调用的鉴权：

Authorization: Bearer pk_live_...

Keys are shown once at creation and stored only as a hash. If a key is lost, rotate it. 密钥只在创建时展示一次，服务端仅保存哈希。密钥丢失请直接轮换。

Control tokens控制令牌

POST /connections returns a JWT bound to a single session_id, carrying that session's budget and an expiry. If it leaks, the blast radius is one session and one budget. POST /connections 返回一个 JWT，绑定单个 session_id，携带该会话的额度与过期时间。即使泄露，影响面也仅限一个会话、一份额度。

The default lifetime is 10 minutes; request up to 60 minutes with credentials_ttl_ms. Set it to at least the longest a session can run — currently 5 minutes. The token is checked when a socket connects, so one that outlives the session covers every reconnect inside it — including resuming after a drop. 默认有效期 10 分钟，可通过 credentials_ttl_ms 申请最长 60 分钟。请把它设为不小于会话可能达到的最长时长 —— 当前为 5 分钟。令牌只在建立连接时校验，因此只要它的有效期覆盖整场会话，会话内的每一次重连（含掉线后续传）都不需要再取新凭证。

REST API#

Create a session创建会话#

Defines the session, reserves capacity, and mints the credential your client needs — in one call. Does not start the stream — that happens when your client sends session.start. 一次调用完成三件事：定义会话、预留容量、签发客户端所需的凭证。不会启动推流 —— 推流在客户端发送 session.start 时才开始。

Request请求体

{
  "model": "r2-realtime-v1",
  "character": { "name": "Aria", "prompt": "A calm archivist.",
                 "voice_ref_url": "https://cdn.example.com/aria-voice.mp3" },
  "scene": { "prompt": "A dim library at night." },
  "seed_image_url": "https://cdn.example.com/aria.jpg",
  "language": "en",
  "history": [{ "role": "user", "content": "..." }, { "role": "character", "content": "..." }],
  "limits": { "max_turns": 200, "turn_rate_per_min": 20 },
  "credentials_ttl_ms": 600000,
  "metadata": { "your_key": "your_value" }
}

Response 201响应 201

{
  "session": {
    "session_id": "sess_8fJ2kL9mQ4nR7tVw3xYz1a",
    "status": "created",
    "model": "r2-realtime-v1",
    "created_at_ms": 1755750000000,
    "reservation_expires_at_ms": 1755750060000,
    "limits": { "max_duration_ms": 5000, "max_turns": 200, "turn_rate_per_min": 20 },
    "media": {
      "video": { "width": 576, "height": 768, "fps": 24, "codec": "h264" },
      "audio": { "codec": "opus", "sample_rate": 48000, "channels": 1 }
    },
    "metadata": { "your_key": "your_value" }
  },
  "credentials": {
    "session_id": "sess_8fJ2kL9mQ4nR7tVw3xYz1a",
    "control_token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...",
    "control_url": "wss://<realtime-host>/public/v1/realtime",
    "expires_at_ms": 1755750600000,
    "ice_servers": [{ "urls": ["stun:stun.l.google.com:19302"] }],
    "budget": {
      "max_duration_ms": 5000,
      "max_turns": 200,
      "remaining_duration_ms": 5000,
      "remaining_turns": 200
    }
  }
}

control_url is a complete URL and its host differs between environments — use it exactly as returned. Do not rebuild it from a constant. control_url 是一个完整 URL，其域名按环境不同 —— 请原样使用，不要用常量自行拼接。

Pass credentials.ice_servers straight to new RTCPeerConnection({ iceServers }). It is a list — its contents may change without notice, and your client should not assume a fixed shape. 把 credentials.ice_servers 直接传给 new RTCPeerConnection({ iceServers })。它是一个列表，内容可能随时变化，客户端不要假设固定结构。

Send only the credentials object to your client. Keep session on your server — session.session_id is what you pass to close a session. 只把 credentials 下发给客户端，session 留在服务端 —— 关闭会话时要用的正是 session.session_id。

Common failures: 409 no_capacity, 422 content_rejected (no session is created and no capacity is held), 409 budget_exhausted (your API Energy balance cannot cover the first block). 常见失败：409 no_capacity、422 content_rejected（不会创建会话，也不占用容量）、409 budget_exhausted（API Energy 余额不足以支付首个计费块）。

Close a session关闭会话#

Ends the session, releases capacity, and returns final usage. Idempotent — repeat calls return the same settlement. 结束会话、释放容量并返回最终用量。幂等 —— 重复调用返回相同的结算结果。

{
  "session_id": "sess_8fJ2kL9mQ4nR7tVw3xYz1a",
  "status": "ended",
  "reason": "server_closed",
  "started_at_ms": 1755750010000,
  "ended_at_ms": 1755750310000,
  "duration_ms": 300000,
  "billed_ms": 300000,
  "turns_used": 42,
  "metadata": { "your_key": "your_value" }
}

A connected client receives session.ended before the socket closes. 仍在连接中的客户端会在 socket 关闭前收到 session.ended。

Realtime API实时 API#

Connecting建立连接#

{credentials.control_url}?session_id=sess_...&resume_from=evt_...

Credentials travel in the WebSocket subprotocol — browsers cannot set custom headers on a WebSocket: 凭证通过 WebSocket 子协议传递 —— 浏览器无法为 WebSocket 设置自定义请求头：

new WebSocket(url, ["r2.v1", `r2.token.${control_token}`]);

Non-browser clients may send Authorization: Bearer <control_token> instead. 非浏览器客户端可以改用 Authorization: Bearer <control_token>。

Handshake close codes握手关闭码

Heartbeats心跳

The server sends a protocol-level ping every 5 seconds; browsers answer automatically, so there is nothing to implement. Three missed responses (15 seconds) end the session with idle_timeout. There is no application-level heartbeat message — do not send extra frames to keep the connection alive. 服务端每 5 秒发送一次协议级 ping，浏览器会自动响应，你无需实现任何逻辑。连续 3 次无响应（15 秒）会以 idle_timeout 结束会话。没有应用层心跳消息 —— 不要为了保活额外发帧。

Message envelope消息信封#

Every message shares one shape.所有消息共用同一结构。

Client → server客户端 → 服务端

{ "type": "turn.submit", "id": "c-17", "data": { "turn_id": "turn_...", "text": "Hello" } }

Server → client服务端 → 客户端

{
  "type": "turn.visible",
  "id": "evt_0K3xQ9mN2pL7vR4tYb1cZa",
  "ts_ms": 1755750000123,
  "session_id": "sess_8fJ2kL9mQ4nR7tVw3xYz1a",
  "data": { "turn_id": "turn_...", "latency_ms": 4210 },
  "debug": null
}

id on a server message increases monotonically within a session and is your resume cursor. debug is null unless the session was created with "debug": true; its contents are diagnostic and not part of the API contract. 服务端消息的 id 在同一会话内单调递增，同时也是你的续传游标。除非创建会话时带了 "debug": true，否则 debug 恒为 null；其内容仅供诊断，不属于 API 契约。

Client events客户端事件#

session.start

Starts the stream. Succeeds once per session.启动推流。每个会话只能成功一次。

{ "type": "session.start", "id": "c-1",
  "data": { "seed_image_url": "..." } }

seed_image_url is an optional override of what you set at creation, and carries the same 3:4 requirement. Everything else — character, scene, history, budget — is fixed at creation time. Responds with session.ready. seed_image_url 是对创建时配置的可选覆盖，同样要求 3:4 比例。其余内容 —— 角色、场景、历史、额度 —— 在创建时即固定。响应为 session.ready。

media.offer

{ "type": "media.offer", "id": "c-2", "data": { "sdp": "v=0\r\no=- ..." } }

Requires session.ready first. Responds with media.answer. 必须先收到 session.ready。响应为 media.answer。

media.connected

{ "type": "media.connected", "id": "c-3", "data": {} }

Send this once, as soon as your RTCPeerConnection reaches connected. Requires session.ready first. The data object is empty — the server timestamps it on arrival. There is no reply event. 在你的 RTCPeerConnection 进入 connected 后立即发送一次。必须先收到 session.ready。data 为空对象 —— 服务端以收到的时刻为准。没有对应的响应事件。

turn.submit

{ "type": "turn.submit", "id": "c-17",
  "data": { "turn_id": "turn_9dK2mP...", "text": "You look different today.",
            "client_sent_at_ms": 1755750123456 } }

session.close

{ "type": "session.close", "id": "c-99", "data": { "reason": "client_closed" } }

Responds with session.ended, then closes with code 1000. 响应 session.ended，随后以关闭码 1000 断开连接。

Server events服务端事件#

Payloads事件负载

session.ready      { "expires_at_ms": …, "media": {…}, "limits": {…} }
media.answer       { "sdp": "v=0\r\n…" }
turn.text          { "turn_id": "turn_…", "text": "I changed my coat." }
turn.prompt_ready  { "turn_id": "turn_…" }
turn.started       { "turn_id": "turn_…", "est_ms": 3200 }
media.clip         { "seq": 42, "kind": "turn", "turn_id": "turn_…" }
turn.visible       { "turn_id": "turn_…", "latency_ms": 4210 }
usage.tick         { "billed_ms": 12000, "turns_used": 3, "budget_ms": 15000,
                     "budget_remaining_ms": 3000, "turns_remaining": 197,
                     "expires_at_ms": 1755750600000 }
session.renewed    { "charged": true, "extended_ms": 5000, "segments": 4,
                     "budget_ms": 20000, "budget_remaining_ms": 8000,
                     "expires_at_ms": 1755750605000 }
session.ended      { "reason": "client_closed", "duration_ms": 300000,
                     "billed_ms": 300000, "turns_used": 42 }

media.clip.kind is idle or turn. turn_id is null unless kind is turn. A common use is showing an idle affordance in your UI while kind is idle, so users can tell the character is waiting rather than replying. media.clip.kind 取值为 idle 或 turn。除非 kind 为 turn，否则 turn_id 为 null。一个常见用法是在 kind 为 idle 时于界面上给出「待机中」提示，让用户能区分角色是在等待还是在回应。

turn.started.est_ms is an estimate for progress indication, not a commitment. turn.visible.latency_ms measures from our receipt of turn.submit to first frame sent; it excludes your client's jitter buffer and decode time, so it is a lower bound. turn.started.est_ms 只是用于进度提示的估算，不构成承诺。turn.visible.latency_ms 统计的是从我们收到 turn.submit 到发出首帧的时间，不含客户端抖动缓冲与解码耗时，因此是一个下界。

session.ended.reason

client_closed · server_closed · budget_exhausted · idle_timeout · reservation_expired · superseded · media_failed · internal_error · viewer_gone

media_failed also covers "media was never connected": if the WebRTC connection is not established within 30 seconds of session.ready, the session ends and nothing is billed. media_failed 也包含「媒体从未连通」这一种：session.ready 之后 30 秒内没有建立 WebRTC 连接，会话即结束，且不计费。

viewer_gone means the renderer saw your WebRTC connection drop and released the session immediately, rather than waiting for the heartbeat to time out. It is the same outcome as idle_timeout, reached sooner. Renegotiating media does not trigger it. viewer_gone 表示渲染端观察到你的 WebRTC 连接已断开，并立即释放了会话，而不是等心跳超时。结果与 idle_timeout 相同，只是更早。重新协商媒体不会触发它。

Delivery guarantees投递保证#

Reconnecting断线重连#

Events are retained for 5 minutes, but the session itself is held for only 10 seconds after the socket drops. Reconnect within that window or the session ends with idle_timeout and the renderer is released to someone else — retry promptly rather than backing off. Reconnect with the last id you received: 事件保留 5 分钟，但连接断开后会话只为你保留 10 秒。请在这个窗口内重连，否则会话会以 idle_timeout 结束、渲染器被释放给他人 —— 应当尽快重试，不要长退避。用你收到的最后一个 id 重连：

{credentials.control_url}?session_id=sess_…&resume_from=evt_…

Everything after that cursor is replayed, then live delivery resumes. Replayed events are byte-identical to the originals, including id and ts_ms — deduplicate by id. This is the only situation that produces duplicates. 游标之后的所有事件会被重放，随后恢复实时投递。重放事件与原事件逐字节一致，包括 id 与 ts_ms —— 请按 id 去重。这是唯一会产生重复的场景。

If the cursor has aged out you receive error with code resume_window_expired (fatal: false) and only live events afterwards; rebuild your UI state. 若游标已过期，你会收到 code 为 resume_window_expired 的 error（fatal: false），此后只有实时事件；此时请重建 UI 状态。

Media媒体#

Media flows directly between the renderer and your client over SRTP. Signaling travels on the WebSocket. 媒体经 SRTP 在渲染器与你的客户端之间直连传输，信令走 WebSocket。

Negotiating协商

const pc = new RTCPeerConnection({ iceServers: credentials.ice_servers });
pc.addTransceiver("video", { direction: "recvonly" });
pc.addTransceiver("audio", { direction: "recvonly" });

await pc.setLocalDescription(await pc.createOffer());
await iceGatheringComplete(pc);           // required
send("media.offer", { sdp: pc.localDescription.sdp });

const pc = new RTCPeerConnection({ iceServers: credentials.ice_servers });
pc.addTransceiver("video", { direction: "recvonly" });
pc.addTransceiver("audio", { direction: "recvonly" });

await pc.setLocalDescription(await pc.createOffer());
await iceGatheringComplete(pc);           // 必需
send("media.offer", { sdp: pc.localDescription.sdp });

Receiving接收

Audio is generated together with the video, so picture and sound are inherently in sync — no client-side alignment is needed. 音频与视频一同生成，音画天然同步 —— 客户端无需做对齐。

Disconnecting and recovering断开与恢复

A real session, message by message真实调用样例#

Every message of one real session, in the order it happened — captured from an integrator's iOS app on a physical device (react-native-webrtc + WebSocket). The JSON is verbatim from the capture apart from the redactions noted below. 一场真实会话从建连到结算的全部报文，按发生顺序排列 —— 采集自接入方 iOS App 真机（react-native-webrtc + WebSocket）。除下面声明的脱敏项外，所有 JSON 均为原始抓取。

1 · Connect — one REST call for every credential① 建连 —— 一次 REST 调用拿到全部凭证#

A single POST /connections does three things: define the session, reserve capacity (a 60-second window), and mint the client credential. Nothing is streaming yet. The request carries Authorization: Bearer pk_live_…, so this step belongs on your own server. 一次 POST /connections 完成三件事：定义会话、预留容量（60 秒窗口）、签发客户端凭证。此时尚未启动推流。请求头带 Authorization: Bearer pk_live_…，这一步应发生在接入方自己的服务端。

{
  "model": "r2-realtime-v1",
  "character": {
    "name": "🌹 Diana Moreau 🌹",
    "prompt": "You are the character in this story. Stay in character, reply in short spoken lines, and never mention being an AI."
  },
  "language": "en",
  "limits": {
    "max_duration_ms": 240000,
    "max_turns": 200,
    "turn_rate_per_min": 20
  },
  "credentials_ttl_ms": 600000,
  "scene": {
    "prompt": "A casual real-time video chat with the user.\n\nWhen the user message is a third-person line starting with \"The user ...\", it describes the user's own expression or action, not something done to you. React to what you observe about them."
  },
  "seed_image_url": "https://cdn.example.com/aria/last_frame.png",
  "metadata": {
    "story_id": "story_20260620192023_611069",
    "episode_id": "episode_20260620192023Gn7v"
  }
}

{
  "session": {
    "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
    "status": "created",
    "model": "r2-realtime-v1",
    "created_at_ms": 1788240929526,
    "reservation_expires_at_ms": 1788240989526,
    "limits": {
      "max_duration_ms": 240000,
      "max_turns": 200,
      "turn_rate_per_min": 20
    },
    "media": {
      "video": {
        "width": 576,
        "height": 768,
        "fps": 24,
        "codec": "h264"
      },
      "audio": {
        "codec": "opus",
        "sample_rate": 48000,
        "channels": 1
      }
    },
    "metadata": {
      "story_id": "story_20260620192023_611069",
      "episode_id": "episode_20260620192023Gn7v"
    }
  },
  "credentials": {
    "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
    "control_token": "eyJhbGciOiJIUzI1NiIsInR5……<token 已打码，实际 311 字符>",
    "control_url": "wss://<realtime-host>/public/v1/realtime",
    "expires_at_ms": 1788241529594,
    "ice_servers": [
      {
        "urls": [
          "stun:stun.l.google.com:19302"
        ]
      }
    ],
    "budget": {
      "max_duration_ms": 240000,
      "max_turns": 200,
      "remaining_duration_ms": 240000,
      "remaining_turns": 200
    }
  }
}

2 · Control plane — WebSocket and starting the stream② 控制面 —— WebSocket 连接与起流#

The client opens the WebSocket at credentials.control_url with ?session_id=…. The credential travels in the subprotocol — ["r2.v1", "r2.token.<control_token>"] from a browser, or subprotocol r2.v1 plus an Authorization: Bearer <control_token> header from a non-browser client, which is what this capture does. Never put the token in the query string. 客户端拿 credentials.control_url 建 WebSocket，URL 带 ?session_id=…。凭证走子协议 ["r2.v1", "r2.token.<control_token>"]（浏览器），或子协议 r2.v1 + Authorization: Bearer <control_token> 请求头（非浏览器客户端 —— 本样例即此方式）。不要把令牌放进 query string。

{
  "type": "session.start",
  "id": "c-1",
  "data": {
    "seed_image_url": "https://cdn.example.com/aria/last_frame.png"
  }
}

{
  "type": "session.ready",
  "id": "evt_0000B9xILXXdj2SBlmxso4",
  "ts_ms": 1788240930806,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "expires_at_ms": 1788241170804,
    "limits": {
      "max_duration_ms": 240000,
      "max_turns": 200,
      "turn_rate_per_min": 20
    },
    "media": {
      "audio": {
        "channels": 1,
        "codec": "opus",
        "sample_rate": 48000
      },
      "video": {
        "codec": "h264",
        "fps": 24,
        "height": 768,
        "width": 576
      }
    }
  },
  "debug": null
}

3 · Media plane — WebRTC negotiation③ 媒体面 —— WebRTC 协商#

Two m-lines, one offer, one answer. v1 does not support trickle ICE: wait for local candidate gathering to finish before sending the offer, and the answer likewise carries a complete set. Feed credentials.ice_servers straight into the peer connection. 两条 m-line、offer/answer 各一次。v1 不支持 Trickle ICE：必须等本地 ICE 候选收集完成再发 offer，answer 同样携带完整候选集。ICE 服务器直接用 credentials.ice_servers。

{
  "type": "media.offer",
  "id": "c-2",
  "data": {
    "sdp": "<完整 SDP 见下方折叠块，共 6375 字符>"
  }
}

{
  "type": "media.answer",
  "id": "evt_0000B9xILrImG0Rt3UAIPw",
  "ts_ms": 1788240934261,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "sdp": "<完整 SDP 见下方折叠块，共 3144 字符>"
  },
  "debug": null
}

{
  "type": "media.clip",
  "id": "evt_0000B9xIMXh0eImuWnpF1E",
  "ts_ms": 1788240941675,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "kind": "idle",
    "seq": 0,
    "turn_id": null
  },
  "debug": null
}

4 · A turn — turn.submit and its five events④ 对话轮次 —— turn.submit 与五个回执事件#

One turn = one turn.submit from the client, then turn.text → turn.prompt_ready → turn.started → media.clip → turn.visible from the server. Apart from ready and ended, every event is best-effort: do not build a state machine that needs all five to advance. Turn 1 is expanded in full below; turns 2 and 3 are summarised. 一轮 = 客户端一条 turn.submit，服务端按序回 turn.text → turn.prompt_ready → turn.started → media.clip → turn.visible。除 ready/ended 外所有事件都是尽力投递，不要做必须集齐五个事件才推进的状态机。下面完整展开第 1 轮，第 2、3 轮摘要列出。

{
  "type": "turn.submit",
  "id": "c-3",
  "data": {
    "turn_id": "turn_5420882c-c732-49fa-b0db-cc8f9c3a851a",
    "text": "The user pats you",
    "client_sent_at_ms": 1788240942507
  }
}

{
  "type": "turn.text",
  "id": "evt_0000B9xIMiZGDS7qFxHhy4",
  "ts_ms": 1788240943578,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "text": "Hey! Don't just pat me like that, it's ticklish!",
    "turn_id": "turn_5420882c-c732-49fa-b0db-cc8f9c3a851a"
  },
  "debug": null
}

{
  "type": "turn.prompt_ready",
  "id": "evt_0000B9xIMid9yxFES36p2u",
  "ts_ms": 1788240943588,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "turn_id": "turn_5420882c-c732-49fa-b0db-cc8f9c3a851a"
  },
  "debug": null
}

{
  "type": "turn.started",
  "id": "evt_0000B9xIMioVImJp1EPsjA",
  "ts_ms": 1788240943621,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "est_ms": 1400,
    "turn_id": "turn_5420882c-c732-49fa-b0db-cc8f9c3a851a"
  },
  "debug": null
}

{
  "type": "media.clip",
  "id": "evt_0000B9xIMrjW8gg4gN9Whk",
  "ts_ms": 1788240945181,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "kind": "turn",
    "seq": 1,
    "turn_id": "turn_5420882c-c732-49fa-b0db-cc8f9c3a851a"
  },
  "debug": null
}

{
  "type": "turn.visible",
  "id": "evt_0000B9xIMruVTpT5ESIIpk",
  "ts_ms": 1788240945212,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "latency_ms": 2487
  },
  "debug": null
}

Turns 2 and 3 produced exactly the same event sequence as turn 1; only the differing fields are listed. 第 2、3 轮事件序列与第 1 轮完全一致，仅列差异字段。

5 · Usage heartbeat — usage.tick⑤ 计费心跳 —— usage.tick#

One every 5 seconds. Every number is cumulative, not a delta, so losing one costs nothing — the next tick carries the correct total. When budget_remaining_ms reaches zero the server ends the session; the budget is a hard ceiling. 每 5 秒一条，全部是累计值而非增量，丢失一条无损 —— 下一条会带上正确总量。budget_remaining_ms 归零时会话被服务端直接结束，额度是硬上限。

{
  "type": "usage.tick",
  "id": "evt_0000B9xIN4sVGaadS0SO12",
  "ts_ms": 1788240947480,
  "session_id": "sess_S3V0Pb6K6qaeh2G58YalNI",
  "data": {
    "billed_ms": 16673,
    "budget_ms": 240000,
    "budget_remaining_ms": 223327,
    "turns_remaining": 199,
    "turns_used": 1
  },
  "debug": null
}

6 · Teardown — closing and settlement⑥ 收尾 —— 关会话与结算#

Close it twice over: send session.close on the WebSocket and call DELETE /sessions/{session_id} over REST, either together or the second as a fallback. Both are idempotent. Do not just close the peer connection and walk away — the session keeps billing until it times out. These messages come from a second, shorter session captured the same day (sess_krJHaNhA…). 双保险收尾：WS 发 session.close，同时（或作为兜底）REST 调 DELETE /sessions/{session_id}，两者均幂等。不要只关 PeerConnection 就走人 —— 会话在超时前持续计费。本节报文取自同日另一场短会话（sess_krJHaNhA…）。

{
  "type": "session.close",
  "id": "c-2",
  "data": {
    "reason": "client_closed"
  }
}

{
  "type": "session.ended",
  "id": "evt_<示例>",
  "ts_ms": 1788240881865,
  "session_id": "sess_krJHaNhA8JRoJyOUYdYkcb",
  "data": {
    "reason": "client_closed",
    "duration_ms": 1677,
    "billed_ms": 1677,
    "turns_used": 0
  },
  "debug": null
}

{
  "session_id": "sess_krJHaNhA8JRoJyOUYdYkcb",
  "status": "ended",
  "reason": "client_closed",
  "started_at_ms": 1788240880188,
  "ended_at_ms": 1788240881865,
  "duration_ms": 1677,
  "billed_ms": 1677,
  "turns_used": 0,
  "metadata": {
    "story_id": "story_20260620192023_611069",
    "episode_id": "episode_20260620192023Gn7v"
  }
}

Errors错误#

One error object, shared by REST and WebSocket. REST returns it under error; the WebSocket delivers it as the data of an error event. REST 与 WebSocket 共用同一个错误对象。REST 把它放在 error 字段下；WebSocket 则作为 error 事件的 data 下发。

{
  "error": {
    "code": "no_capacity",
    "message": "All realtime capacity is in use.",
    "status": 409,
    "retryable": true,
    "retry_after_ms": 5000,
    "fatal": false,
    "turn_id": null,
    "request_id": "req_4nQ8xR2mK7pL9vT3wYz1cB"
  }
}

Branch on code, never on message — messages are for humans and may change. fatal: true (WebSocket only) means the session is over and session.ended follows. Include request_id in support requests. 请基于 code 分支处理，绝不要依赖 message —— 文案面向人类阅读，可能随时变更。fatal: true（仅 WebSocket）表示会话已经结束，随后会有 session.ended。提工单时请附上 request_id。

Limits and capacity限额与容量#

Per session单会话

Per account按账号

concurrent_sessions, sessions_per_day, and turns_per_day are recorded on your key, but are not currently enforced — no request is rejected today for exceeding them, and 429 quota_exceeded is not returned. The binding constraint is renderer concurrency below, together with your API Energy balance. If your integration needs guaranteed per-account quotas, contact us. concurrent_sessions、sessions_per_day、turns_per_day 会记录在你的密钥上，但当前不做强制 —— 目前不会有请求因为超出它们而被拒绝，也不会返回 429 quota_exceeded。真正约束你的是下面的渲染器并发容量，以及你的 API Energy 余额。如果你的接入需要有保障的账号级配额，请联系我们。

Concurrency并发

Best practices最佳实践#

Not available in v1v1 暂不支持#

Versioning版本策略#

What changed · 1.6 → 1.7变更明细 · 1.6 → 1.7#

Every item below is backward-compatible: nothing was removed or renamed, and the API version is unchanged (/public/v1, r2.v1). A client written against 1.6 keeps working — but the two rows marked ⚠ change what it will observe at runtime. 下列变更全部向后兼容：没有删除或改名，API 版本未变（仍为 /public/v1、r2.v1）。按 1.6 编写的客户端可继续运行 —— 但标记 ⚠ 的两行会改变它在运行时观察到的结果。

The REST path (/public/v1) and the WebSocket subprotocol (r2.v1) carry the version together — they always move as a pair. REST 路径（/public/v1）与 WebSocket 子协议（r2.v1）共同承载版本号 —— 两者始终同步升级。

Pin nothing beyond the version string, and treat debug as absent — it is diagnostic output and changes without notice. 除版本号外不要固化任何东西；请把 debug 当作不存在 —— 它是诊断输出，随时可能变化。

Support技术支持#

Include the request_id from the error body, the session_id, and an approximate timestamp. For internal_error and upstream_unavailable these three are usually enough to locate the session end to end. 请提供错误响应中的 request_id、session_id 以及大致的时间戳。对于 internal_error 与 upstream_unavailable，这三项通常足以端到端定位会话。