## Streaming (Server-Sent Events)

With `stream: true` the model response arrives as a stream of events while tokens are generated, rather than as a single chunk at the end. The first byte reaches the client within seconds regardless of the full response length.

Response headers: `Content-Type: text/event-stream`, `Cache-Control: no-cache`, `Connection: keep-alive`. Each event is a `data: {JSON}\n\n` line, and the terminating event is `data: [DONE]\n\n`.

## Event format

```
data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"delta":{"role":"assistant","content":"CRM"}}]}

data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"delta":{"content":" is"}}]}

data: {"id":"chatcmpl-a9f6128818355f17","object":"chat.completion.chunk","model":"bitrix/bitrixgpt-5.5","choices":[{"index":0,"finish_reason":"stop","delta":{}}],"usage":{"prompt_tokens":10,"completion_tokens":15,"total_tokens":25}}

data: [DONE]
```

`usage` arrives in the last event before `[DONE]`. Accumulate `delta.content` from all events to assemble the full response text.

With function `tools`, an empty turn or a call finish without a call carries an English explanation in a separate `delta.content` event with `finish_reason: null`, before the single terminal `finish_reason: "stop"`. If text was already present, a blank line separates the explanation. It asks the model to continue with the declared tools: an attempted undeclared name may have been discarded before reaching the platform. The response remains HTTP 200; `usage` and `[DONE]` are unchanged, and the explanation adds no billable tokens. This hint is not added for `tool_choice: "none"`, structured output, actual calls or reasoning-only turns ending with `stop`.

## Stream handling example

```javascript
const response = await fetch('https://vibecode.bitrix24.com/v1/chat/completions', {
  method: 'POST',
  headers: {
    'X-Api-Key': 'YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    model: 'bitrix/bitrixgpt-5.5',
    messages: [{ role: 'user', content: 'Tell me about CRM' }],
    stream: true,
  }),
})

const reader = response.body.getReader()
const decoder = new TextDecoder()
let buffer = ''

while (true) {
  const { done, value } = await reader.read()
  if (done) break
  buffer += decoder.decode(value, { stream: true })
  const lines = buffer.split('\n')
  buffer = lines.pop() ?? ''

  for (const line of lines) {
    if (!line.startsWith('data: ')) continue
    const payload = line.slice(6)
    if (payload === '[DONE]') return
    const chunk = JSON.parse(payload)
    const delta = chunk.choices?.[0]?.delta?.content
    if (delta) process.stdout.write(delta)
  }
}
```

## Choosing a mode under load

In synchronous mode (`stream: false`) the full model response is assembled on the server and sent to the client as a single response — the first byte arrives together with the last. For a model with a long reasoning phase — `bitrix/bitrixgpt-5.5-thinking` — generation on a long context and with a detailed answer takes tens of seconds. If the client timeout is shorter than the full generation time, the connection closes without receiving a single byte — for example, `cURL error 28: Operation timed out, 0 bytes received`. On short requests generation fits within the timeout and the effect does not appear.

In streaming mode (`stream: true`) the server sends status `200` and the headers right after processing begins, and events flow as tokens are generated, including the reasoning phase. For large requests and reasoning models use `stream: true` — the client timeout on receiving the first byte then does not fire.

Guidelines for the client timeout:

- **Streaming mode (`stream: true`)** — set the timeout for receiving the first event with a margin for the model's reasoning. A reasoning model spends a few seconds before the first token; a value of 30 seconds or more covers this phase with a margin.
- **Synchronous mode (`stream: false`)** — the whole-request timeout must cover the full generation time, not just the network exchange. For a large request to a reasoning model this is tens of seconds. In addition, on a provider error or timeout a synchronous request is retried automatically, and then the final response or error arrives after a few minutes. So values of 9-15 seconds are not enough: set the whole-request timeout with a margin for such retries — a few minutes — or switch to streaming mode, where the timeout on receiving the first byte removes the problem.

## Known specifics

**There is no fallback model in the stream.** On a provider failure a synchronous request is retried with the default model, while in streaming mode the client receives the error in the last event before `data: [DONE]`.

**Errors arrive inside the stream.** Status `200` has already been sent when the platform learns of overload or of an interrupted generation, so a service event of the form `data: { "error": { "code": "pool_exhausted", "retryAfter": 5 } }` arrives before `data: [DONE]`. Read the stream to the end and check the `error` field.

**An incomplete structured response also arrives as an error event.** If the request carried `response_format` and the model did not produce complete JSON without detected repetition, `data: { "error": { "code": "structured_output_truncated", "message": "...", "type": "invalid_request_error" } }` arrives before `data: [DONE]`. This event has only three fields — `code`, `message` and `type`. The `finishReason`, `param` and `suggestedMaxTokens` fields carried by the synchronous `422` are not present here. The `message` text differs by finish reason. When generation is cut off with `finish_reason: "length"`, the message suggests retrying with a raised `max_tokens`. When the stream ends without a finish reason, the message warns that the response may be incomplete. When the model finishes on its own with another reason, the message states that there is no parseable JSON and recommends a model without reasoning. See [guaranteed JSON response](./json.md).

**An unparseable structured response arrives as its own event.** If the model answered a `response_format` request with non-empty text that does not parse as a JSON object or array, `data: { "error": { "code": "structured_output_invalid_json", "message": "...", "type": "invalid_request_error" } }` arrives before `data: [DONE]`. The event carries the same three fields — `code`, `message` and `type`. The answer chunks have already been delivered by then, so discard what you collected; the stream status stays `200` by protocol. Retrying with a raised `max_tokens` does not change the result. Recovery from the internal `reasoning_content` channel is not performed here, unlike the empty-answer case: a document appended after chunks already delivered would merge with them in the client's buffer. See [guaranteed JSON response](./json.md).

**Looping reasoning is cut off with a `reasoning_repetition` event.** If the model, before starting its answer, keeps repeating the same sentences in `reasoning_content`, the platform stops the generation without waiting for the service deadline or the `max_tokens` limit and sends `data: { "error": { "code": "reasoning_repetition", "message": "...", "type": "invalid_request_error" } }` before `data: [DONE]`. The error is not transient: the same request at the same reasoning level will loop again — send it again with reasoning turned off (`reasoning_effort: "none"`); a larger `max_tokens` will not help. Reasoning chunks already delivered stay with the client. The platform catches repetition of the same sentences, not a run of different wordings, and by default (`repetition_guard` omitted or `"off"`) for early termination looks only at the reasoning before the answer starts: a repetitive answer in `content` (for example, an array of identical objects) is not cut off. If the model sends the same text into both channels at once, what counts is how it reaches the client: for a reasoning model this text arrives in `content` and is delivered as the answer; if it arrives only as `reasoning_content`, its repetition is cut off the same way as reasoning. The exception is `response_format: {"type": "json_object"}` when the model sends the answer itself through the `reasoning_content` channel as a JSON document (the reasoning starts with `{` or `[`): the cut-off is then deferred while recovery is still possible, and recovery decides — a complete document arrives as the answer, and if nothing can be recovered, `reasoning_repetition` arrives. If the document is too long to be recovered, the repetition cuts the stream off at once. Without `response_format` there is no exception: an answer that arrives only in `reasoning_content` is cut off with a `reasoning_repetition` event when it loops — recovering the answer from the reasoning is done only for `json_object`.

**A stream also has an overall service deadline.** Beyond the idle timeout, a call now has a bound on its total time: if it does not fit, the stream ends with `data: { "error": { "code": "ai_deadline_exceeded", "type": "rate_limit_exceeded", "retryable": true, "retryAfter": <seconds> } }` before `data: [DONE]`. Unlike the idle timeout, this deadline fires even while tokens keep arriving — it is about the length of the whole answer, not about a gap in it. You can name a shorter budget of your own with the `X-AI-Deadline-Ms` request header; the details are on the [Limits and overload](/docs/ai/chat/rate-limits) page.

**A stalled response is terminated by an idle timeout.** If the model sends headers but then goes silent mid-response and delivers no new data for longer than the idle window, the platform terminates the call and sends `data: { "error": { "code": "stream_idle_timeout", "type": "server_error", "retryable": true, "retryAfter": <seconds> } }` before `data: [DONE]`. The error is retryable — retry the request, honoring `retryAfter`. A continuous flow of tokens (including reasoning tokens from thinking models) resets the idle timer and is never subject to this timeout.

**The same event also ends a stream that never started.** If the upstream accepted the request but sent no response headers within the allotted budget, the call ends with the same retryable `stream_idle_timeout` before `data: [DONE]` instead of silence. Models from every provider behave this way: stream setup takes a single attempt on one shared budget, so the wait never doubles.

**A successful stream ends with a `finish_reason` event.** If the model finished its answer but sent no terminal event, the platform adds one before `data: [DONE]` itself: an empty `delta` with `finish_reason: "tool_calls"` when the answer contained tool calls, or `finish_reason: "stop"` otherwise. If the connection to the model dropped before a finish reason arrived, the answer may be incomplete: `data: { "error": { "code": "stream_incomplete", "type": "server_error", "retryable": true, "retryAfter": <seconds> } }` arrives before `data: [DONE]`. The error is retryable — retry the request, honoring `retryAfter`.

**Repetition in truncated JSON also produces `reasoning_repetition`.** With `repetition_guard` omitted or `"off"`, for `json_object` or `json_schema`, when generation ends with `finish_reason: "length"`, non-empty incomplete JSON is checked for repetition. A detected repetition produces `reasoning_repetition` and a JSON-specific message, without advice to disable reasoning and without `param` or `suggestedMaxTokens`. This is a heuristic diagnosis: it can also trigger on a legitimate repetitive array that exceeded the budget. Check whether repetition is expected; if so, check the token budget, otherwise revise the request or use another model. The check runs after generation ends and does not shorten the wait. Short truncation without detected repetition and complete JSON at `length` retain `structured_output_truncated`; a legitimate repetitive array reaches the client in full on normal completion. In a stream, the refusal arrives as one `error` event with `code`, `message`, `hint`, and `type` before `data: [DONE]`; discard the already delivered `content` fragments. The HTTP status remains `200`.

## Early refusal on repeated JSON

With explicit `repetition_guard: "abort"`, the platform inspects normalized `content` before generation ends. It requires `json_object`/`json_schema`, no declared tools and a streaming-capable model. Cowork control/completion requests and quota announcements are excluded. Incompatible requests return ordinary JSON `400 invalid_request` with `param: "repetition_guard"` before opening a stream, reservation or inference. See [JSON response](./json.md) for details, heuristic limits and the SDK example.

Detection cancels the active model request, and with the client stream already open the client receives exactly these two events under HTTP `200`:

```text
data: {"error":{"message":"Repeated JSON content was detected and this request was stopped early.","type":"invalid_request_error","code":"reasoning_repetition","hint":"Disable repetition_guard for expected repeated content; otherwise revise the request or use another model."}}

data: [DONE]
```

Discard the accumulated partial answer. The refusal does not wait for `finish_reason: "length"` or invent a finish reason. It carries no `retryable`, `retryAfter`, `param`, `suggestedMaxTokens`, raw answer text or private provider identifiers. No retry, answer recovery or fallback follows a hit. Transport cancellation does not guarantee immediate GPU release.

The heuristic can refuse a legitimate repetitive array prefix. Complete JSON at inspection is excluded, actual tool calls disable detection, and the reasoning guard is unchanged. Leave the option off (`"off"` or omitted) for expected repetition. By default, repetitive answer content still reaches the terminal check.

## See also

- [Create a chat completion](./completions.md)
- [Request limits](./rate-limits.md)
- [Guaranteed JSON response](./json.md)
- [AI Router](/docs/ai)