> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulate.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Multilingual Fast Transcription Streaming
> Low-latency multilingual speech-to-text over WebSocket. Covers 25 languages with automatic language identification, with optional speaker labels.
Low-latency multilingual speech-to-text over WebSocket. The spoken language is identified automatically, so the client declares no language and uses no per-language endpoint. Interim partial transcripts stream while audio arrives, and a final `utterance` is emitted for each segment of speech.
This endpoint produces no emotion, accent, or PII/PHI data. Use [STT Streaming](/api-reference/stt/streaming) if you need those.
## Endpoint
```text theme={null}
wss://platform.modulate.ai/api/velma-2-stt-streaming-multilingual-vfast
```
## Authentication
The API key goes in the query string at connection time, not in a header. See [Authentication and rate limits](/guides/authentication).
## Supported audio formats
**Container formats** — `sample_rate` and `num_channels` are ignored if supplied:
`mp3`, `wav`, `flac`, `m4a`, `mp4`, `ogg`, `opus`, `webm`, `aac`, `aiff`, `wma`, `amr`, and `au`, plus 66 others
`3g2`, `3ga`, `3gp`, `3gpp`, `8svx`, `aa3`, `aac`, `ac3`, `act`, `adts`, `aif`, `aifc`, `aiff`,
`amb`, `amr`, `asf`, `at3`, `au`, `avr`, `awb`, `bwf`, `c2`, `caf`, `dss`, `dts`, `dtshd`,
`eac3`, `ec3`, `f4a`, `f4b`, `flac`, `gsm`, `iff`, `m2a`, `m2ts`, `m4a`, `m4b`, `m4r`, `m4v`,
`mka`, `mkv`, `mlp`, `mp+`, `mp1`, `mp2`, `mp3`, `mp4`, `mpa`, `mpc`, `mpga`, `mpp`, `mts`,
`oga`, `ogg`, `ogx`, `oma`, `omg`, `opus`, `paf`, `pvf`, `qcp`, `ra`, `rf64`, `rm`, `rmvb`,
`snd`, `svx`, `thd`, `ts`, `tta`, `voc`, `vqf`, `w64`, `wav`, `wave`, `weba`, `webm`, `wma`,
`wmv`
The MP4-family values (`mp4`, `m4a`, `m4b`, `m4r`, `m4v`, `3gp`, `3gpp`, `3ga`, `3g2`, `f4a`, `f4b`) must be sent in a streamable layout; otherwise the connection ends with an audio-processing error.
**Raw PCM formats** — `sample_rate` and `num_channels` are required:
`s8`, `s16le`, `s16be`, `s24le`, `s24be`, `s32le`, `s32be`, `u8`, `u16le`, `u16be`, `u24le`, `u24be`, `u32le`, `u32be`, `f32le`, `f32be`, `f64le`, `f64be`, `mulaw`, `alaw`, `g722`, `vox`
`g722` and `vox` are mono-only: `num_channels` must be `1`.
**Valid sample rates:** 8000, 11025, 16000, 22050, 32000, 44100, 48000, 96000
For lowest end-to-end latency, send `audio_format=s16le&sample_rate=16000&num_channels=1`.
## Query parameters
| Parameter | Required | Description |
| -------------- | ------------ | ---------------------------------------------------------------------------------------------------------------------------------------- |
| `api_key` | Yes | Your API key |
| `audio_format` | Yes | Container or raw PCM format of the audio you will stream |
| `sample_rate` | Raw PCM only | Source sample rate in Hz. Required for a raw format; ignored for container formats. |
| `num_channels` | Raw PCM only | Number of audio channels (1–8). Required for a raw format; ignored for container formats. |
| `endpointing` | No | `true` or `false` (default `false`). When `true`, speech is segmented into utterances at pauses. |
| `diarize` | No | `true` or `false` (default `false`). When `true`, each final `utterance` carries speaker fields — see [Speaker labels](#speaker-labels). |
`diarize` and `endpointing` are independent; all four combinations are valid.
**Worth knowing:** `diarize=true` substantially increases time-to-first-text. Speaker decisions are made over a multi-second window, so the first `partial_utterance` arrives several seconds later than on a non-diarized connection, and the gap widens under concurrency. Transcript accuracy is unaffected - the cost is latency, not quality. If you are rendering live captions, either size the interface for this or run a second, non-diarized connection for the display text.
## Language identification
The spoken language is identified automatically across these 25 languages:
Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swedish, Ukrainian.
Each `utterance` is transcribed in the language spoken, with auto-capitalization and auto-punctuation already applied. There is no `language` query parameter, and recognition quality varies by language.
**Worth knowing:** this set is smaller and narrower than it sounds. Multilingual Transcription covers 100 languages; these 25 are all European, so Chinese, Japanese, Arabic, Hindi and Korean are not among them. Audio outside the set is still transcribed, but not reliably.
**Worth knowing:** no message carries a language field. The transcript is in the detected language, but the detection itself is not reported, so a client cannot branch on it. Multilingual Fast Transcription (batch) does return `language`; the streaming endpoint does not. Use [Language Detection](/get-started/language-detection) when the code itself is needed.
## Speaker labels
With `diarize=true`, each final `utterance` carries `speaker`, a zero-based index identifying who produced it. Partials never carry it, and a label is assigned when an utterance is finalized and never revised.
The index is stable for one connection and meaningless outside it: the same person on a new connection may receive a different index, so it is not an identity that can be matched across connections.
`speaker` is `null` when diarization was requested but no speaker could be determined for that utterance. That is distinct from the field being absent, which means `diarize=true` was not requested.
### Speaker capacity
Up to **four** concurrent speakers are labeled per connection. Beyond four, additional speakers are merged into existing labels: their speech is still transcribed, but `speaker` is wrong for some utterances.
**Worth knowing:** `speakers_capped` and `speaker_purity` are the two signals that a label may be unreliable, and a client acting on speaker identity should check both. The four-speaker ceiling is lower than English Fast Transcription (streaming), which labels ten, so a conversation that diarizes cleanly there can exceed capacity here.
### Utterances that span a speaker change
Segments are cut at pauses regardless of `diarize`. An utterance spanning a fast exchange with no pause carries the speaker who produced most of it, and `speaker_purity` reports what proportion that was. `1.0` is cleanly single-speaker; lower means the label describes only the majority.
### With endpointing off
`diarize=true&endpointing=false` produces one final `utterance` for the whole stream, so a single `speaker` cannot describe it. Read `n_speakers` and treat `speaker` as indicative only.
## Connection flow
1. Connect with `api_key`, `audio_format`, and (for raw formats) `sample_rate` and `num_channels`.
2. Stream audio as **binary** WebSocket frames.
3. Receive `partial_utterance` messages while audio arrives.
4. Send an **empty text frame** (`""`) to signal end of audio.
5. Receive the final `utterance`(s), then a `done` message.
6. The connection closes.
**Worth knowing:** step 4 must be a **text** frame containing the empty string. An empty *binary* frame is treated as an audio chunk and does not end the stream, so the connection hangs until it times out with no final `utterance`.
## Server messages
### `partial_utterance`
The complete transcript so far for the current scope — the whole connection by default, or the current segment when `endpointing=true`. Not a delta: replace the displayed text on each message, never append.
```json theme={null}
{
"type": "partial_utterance",
"partial_utterance": { "text": "Buenos días, gracias por", "is_final": false }
}
```
Partials never carry speaker fields, even with `diarize=true`.
### `utterance`
The final transcript for one segment. Will not be revised.
```json theme={null}
{
"type": "utterance",
"utterance": {
"text": "Buenos días, gracias por llamar.",
"is_final": true,
"start_ms": 0,
"duration_ms": 2360,
"speaker": 0,
"n_speakers": 2,
"speakers_capped": false,
"speaker_purity": 1.0
}
}
```
| Field | Type | Description |
| --------------------------- | --------------- | ---------------------------------------------------------------------------------------------------- |
| `type` | string | Always `"utterance"` |
| `utterance.text` | string | Final transcript for this segment, in the language spoken. |
| `utterance.is_final` | boolean | Always `true` |
| `utterance.start_ms` | integer | Start of this segment, in milliseconds from the beginning of the stream. Approximate. |
| `utterance.duration_ms` | integer | Duration of this segment's speech, excluding the pause that ended it. Approximate. |
| `utterance.speaker` | integer \| null | Present only with `diarize=true`. Zero-based speaker index, `null` when none could be determined. |
| `utterance.n_speakers` | integer | Present only with `diarize=true`. Distinct speakers detected on the connection so far. |
| `utterance.speakers_capped` | boolean | Present only with `diarize=true`. `true` once all four speaker slots are in use. |
| `utterance.speaker_purity` | number | Present only with `diarize=true`. Proportion of this utterance attributed to `speaker`, from 0 to 1. |
With `diarize=false` no speaker field appears in any message.
### `done`
Sent once, immediately after the final `utterance`. The server closes after it.
```json theme={null}
{ "type": "done", "duration_ms": 14253 }
```
### `error`
Sent if something goes wrong. The connection closes after it; no further messages follow.
```json theme={null}
{ "type": "error", "error": "diarize=true is not available on this endpoint." }
```
That second message means speaker labelling is not enabled on the instance that served the connection. Retry once — instances can differ while a deployment is rolling — and if it persists, reconnect without `diarize`.
## WebSocket close codes
| Code | Meaning |
| ------ | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `1000` | Normal closure after the `done` message |
| `1003` | Invalid query parameters — missing `audio_format`, invalid `sample_rate`, invalid `endpointing`, invalid `diarize`, unsupported value. Also returned when `diarize=true` is requested but speaker labelling is not enabled on the serving instance; retry once before treating this as permanent. |
| `1011` | Internal server error. An `error` message is sent before the close. |
| `1013` | The service is temporarily at capacity. Not an account or billing condition — retry the connection. Diarized connections have a lower concurrency ceiling than plain transcription, so a `diarize=true` connection can be refused while a plain one succeeds. |
| `4001` | The `api_key` query parameter is missing or invalid |
| `4002` | Audio bytes did not match the declared raw format, or the audio could not be decoded mid-stream |
| `4003` | The request is not permitted |
| `4004` | The API key does not have access to this model |
| `4029` | Insufficient credits |
| `4030` | Concurrent request limit reached — an account-level limit on simultaneous connections, distinct from the service-capacity `1013`. Contact support to raise it. |
| `4031` | Monthly usage limit reached |
## Related
* [Transcription](/get-started/stt) — parameters, response shape, and audio formats for every transcription endpoint
* [Multilingual Fast Transcription Batch](/api-reference/stt/batch-multilingual-vfast) — the file-upload counterpart, which does return `language`
* [Which API should I use?](/guides/which-api)