Multilingual Fast Transcription Streaming

WSS

api

velma-2-stt-streaming-multilingual-vfast

Low-latency multilingual speech-to-text over WebSocket. The spoken language is identified automatically, so the client declares no language and uses no per-language endpoint. Interim partial transcripts stream while audio arrives, and a final utterance is emitted for each segment of speech. This endpoint produces no emotion, accent, or PII/PHI data. Use STT Streaming if you need those.

Endpoint

wss://platform.modulate.ai/api/velma-2-stt-streaming-multilingual-vfast

Authentication

The API key goes in the query string at connection time, not in a header. See Authentication and rate limits.

Supported audio formats

Container formats — sample_rate and num_channels are ignored if supplied: mp3, wav, flac, m4a, mp4, ogg, opus, webm, aac, aiff, wma, amr, and au, plus 66 others

All 79 accepted container values

3g2, 3ga, 3gp, 3gpp, 8svx, aa3, aac, ac3, act, adts, aif, aifc, aiff, amb, amr, asf, at3, au, avr, awb, bwf, c2, caf, dss, dts, dtshd, eac3, ec3, f4a, f4b, flac, gsm, iff, m2a, m2ts, m4a, m4b, m4r, m4v, mka, mkv, mlp, mp+, mp1, mp2, mp3, mp4, mpa, mpc, mpga, mpp, mts, oga, ogg, ogx, oma, omg, opus, paf, pvf, qcp, ra, rf64, rm, rmvb, snd, svx, thd, ts, tta, voc, vqf, w64, wav, wave, weba, webm, wma, wmv

The MP4-family values (mp4, m4a, m4b, m4r, m4v, 3gp, 3gpp, 3ga, 3g2, f4a, f4b) must be sent in a streamable layout; otherwise the connection ends with an audio-processing error. Raw PCM formats — sample_rate and num_channels are required: s8, s16le, s16be, s24le, s24be, s32le, s32be, u8, u16le, u16be, u24le, u24be, u32le, u32be, f32le, f32be, f64le, f64be, mulaw, alaw, g722, vox g722 and vox are mono-only: num_channels must be 1. Valid sample rates: 8000, 11025, 16000, 22050, 32000, 44100, 48000, 96000 For lowest end-to-end latency, send audio_format=s16le&sample_rate=16000&num_channels=1.

Query parameters

Parameter	Required	Description
`api_key`	Yes	Your API key
`audio_format`	Yes	Container or raw PCM format of the audio you will stream
`sample_rate`	Raw PCM only	Source sample rate in Hz. Required for a raw format; ignored for container formats.
`num_channels`	Raw PCM only	Number of audio channels (1–8). Required for a raw format; ignored for container formats.
`endpointing`	No	`true` or `false` (default `false`). When `true`, speech is segmented into utterances at pauses.
`diarize`	No	`true` or `false` (default `false`). When `true`, each final `utterance` carries speaker fields — see Speaker labels.

diarize and endpointing are independent; all four combinations are valid.

Worth knowing: diarize=true substantially increases time-to-first-text. Speaker decisions are made over a multi-second window, so the first partial_utterance arrives several seconds later than on a non-diarized connection, and the gap widens under concurrency. Transcript accuracy is unaffected - the cost is latency, not quality. If you are rendering live captions, either size the interface for this or run a second, non-diarized connection for the display text.

Language identification

The spoken language is identified automatically across these 25 languages: Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Russian, Slovak, Slovenian, Spanish, Swedish, Ukrainian. Each utterance is transcribed in the language spoken, with auto-capitalization and auto-punctuation already applied. There is no language query parameter, and recognition quality varies by language.

Worth knowing: this set is smaller and narrower than it sounds. Multilingual Transcription covers 100 languages; these 25 are all European, so Chinese, Japanese, Arabic, Hindi and Korean are not among them. Audio outside the set is still transcribed, but not reliably.

Worth knowing: no message carries a language field. The transcript is in the detected language, but the detection itself is not reported, so a client cannot branch on it. Multilingual Fast Transcription (batch) does return language; the streaming endpoint does not. Use Language Detection when the code itself is needed.

Speaker labels

With diarize=true, each final utterance carries speaker, a zero-based index identifying who produced it. Partials never carry it, and a label is assigned when an utterance is finalized and never revised. The index is stable for one connection and meaningless outside it: the same person on a new connection may receive a different index, so it is not an identity that can be matched across connections. speaker is null when diarization was requested but no speaker could be determined for that utterance. That is distinct from the field being absent, which means diarize=true was not requested.

Speaker capacity

Up to four concurrent speakers are labeled per connection. Beyond four, additional speakers are merged into existing labels: their speech is still transcribed, but speaker is wrong for some utterances.

Worth knowing: speakers_capped and speaker_purity are the two signals that a label may be unreliable, and a client acting on speaker identity should check both. The four-speaker ceiling is lower than English Fast Transcription (streaming), which labels ten, so a conversation that diarizes cleanly there can exceed capacity here.

Utterances that span a speaker change

Segments are cut at pauses regardless of diarize. An utterance spanning a fast exchange with no pause carries the speaker who produced most of it, and speaker_purity reports what proportion that was. 1.0 is cleanly single-speaker; lower means the label describes only the majority.

With endpointing off

diarize=true&endpointing=false produces one final utterance for the whole stream, so a single speaker cannot describe it. Read n_speakers and treat speaker as indicative only.

Connection flow

Connect with api_key, audio_format, and (for raw formats) sample_rate and num_channels.
Stream audio as binary WebSocket frames.
Receive partial_utterance messages while audio arrives.
Send an empty text frame ("") to signal end of audio.
Receive the final utterance(s), then a done message.
The connection closes.

Worth knowing: step 4 must be a text frame containing the empty string. An empty binary frame is treated as an audio chunk and does not end the stream, so the connection hangs until it times out with no final utterance.

Server messages

`partial_utterance`

The complete transcript so far for the current scope — the whole connection by default, or the current segment when endpointing=true. Not a delta: replace the displayed text on each message, never append.

{
  "type": "partial_utterance",
  "partial_utterance": { "text": "Buenos días, gracias por", "is_final": false }
}

Partials never carry speaker fields, even with diarize=true.

`utterance`

The final transcript for one segment. Will not be revised.

{
  "type": "utterance",
  "utterance": {
    "text": "Buenos días, gracias por llamar.",
    "is_final": true,
    "start_ms": 0,
    "duration_ms": 2360,
    "speaker": 0,
    "n_speakers": 2,
    "speakers_capped": false,
    "speaker_purity": 1.0
  }
}

Field	Type	Description
`type`	string	Always `"utterance"`
`utterance.text`	string	Final transcript for this segment, in the language spoken.
`utterance.is_final`	boolean	Always `true`
`utterance.start_ms`	integer	Start of this segment, in milliseconds from the beginning of the stream. Approximate.
`utterance.duration_ms`	integer	Duration of this segment’s speech, excluding the pause that ended it. Approximate.
`utterance.speaker`	integer \| null	Present only with `diarize=true`. Zero-based speaker index, `null` when none could be determined.
`utterance.n_speakers`	integer	Present only with `diarize=true`. Distinct speakers detected on the connection so far.
`utterance.speakers_capped`	boolean	Present only with `diarize=true`. `true` once all four speaker slots are in use.
`utterance.speaker_purity`	number	Present only with `diarize=true`. Proportion of this utterance attributed to `speaker`, from 0 to 1.

With diarize=false no speaker field appears in any message.

`done`

Sent once, immediately after the final utterance. The server closes after it.

{ "type": "done", "duration_ms": 14253 }

`error`

Sent if something goes wrong. The connection closes after it; no further messages follow.

{ "type": "error", "error": "diarize=true is not available on this endpoint." }

That second message means speaker labelling is not enabled on the instance that served the connection. Retry once — instances can differ while a deployment is rolling — and if it persists, reconnect without diarize.

WebSocket close codes

Code	Meaning
`1000`	Normal closure after the `done` message
`1003`	Invalid query parameters — missing `audio_format`, invalid `sample_rate`, invalid `endpointing`, invalid `diarize`, unsupported value. Also returned when `diarize=true` is requested but speaker labelling is not enabled on the serving instance; retry once before treating this as permanent.
`1011`	Internal server error. An `error` message is sent before the close.
`1013`	The service is temporarily at capacity. Not an account or billing condition — retry the connection. Diarized connections have a lower concurrency ceiling than plain transcription, so a `diarize=true` connection can be refused while a plain one succeeds.
`4001`	The `api_key` query parameter is missing or invalid
`4002`	Audio bytes did not match the declared raw format, or the audio could not be decoded mid-stream
`4003`	The request is not permitted
`4004`	The API key does not have access to this model
`4029`	Insufficient credits
`4030`	Concurrent request limit reached — an account-level limit on simultaneous connections, distinct from the service-capacity `1013`. Contact support to raise it.
`4031`	Monthly usage limit reached

Transcription — parameters, response shape, and audio formats for every transcription endpoint
Multilingual Fast Transcription Batch — the file-upload counterpart, which does return language
Which API should I use?

Messages

query

type:object

Partial Utterance Result

type:object

In-progress transcript snapshot for the CURRENT speech segment. Sent zero or more times per segment, roughly every 1.5 seconds. The text field contains the current segment's transcript so far - not a delta, and not the whole stream - with auto-capitalization and auto-punctuation already applied. Clients should REPLACE the current segment's previously displayed partial text, not append. Partials never carry speaker labels, even with diarize=true, and never carry word timestamps, even with time_stamps=true.

Final Utterance Result

type:object

The final transcript for one segment of speech, with auto-capitalization and auto-punctuation applied, plus the segment's start time and duration. With diarize=true it also carries the speaker who produced it, and with time_stamps=true the start and end time of every word. Sent once per segment (when the segment ends, and once more for the last segment after end-of-stream), and supersedes the partial_utterance messages for that segment. The full transcript is the concatenation of these texts in order.

Error

type:object

An error occurred during transcription. Sent at most once per connection; the WebSocket closes shortly after. No further messages follow an error.

Transcription Complete

type:object

Transcription completed successfully. Sent exactly once, immediately after the final utterance message. After this message the server closes the WebSocket.

Audio Data Chunk

type:string

Raw audio data chunk for transcription. Sent zero or more times by the client during the stream lifetime. Each chunk's bytes are decoded by the server according to the connection's audio_format query parameter.

End of Audio Stream

type:string

Empty text frame signaling end of audio. Sent exactly once by the client after the last audio chunk. The server responds with a final utterance message followed by a done message, then closes the connection.

Messages

Velma

Speech-to-text Transcription

Deepfake Detection

Emotion Detection

Accent Detection

PII/PHI Redaction

Music Detection

AI Music Detection

Language Detection

Audio Event Detection

Multilingual Fast Transcription Streaming

Endpoint

Authentication

Supported audio formats

Query parameters

Language identification

Speaker labels

Speaker capacity

Utterances that span a speaker change

With endpointing off

Connection flow

Server messages

`partial_utterance`

`utterance`

`done`

`error`

WebSocket close codes

​Endpoint

​Authentication

​Supported audio formats

​Query parameters

​Language identification

​Speaker labels

​Speaker capacity

​Utterances that span a speaker change

​With endpointing off

​Connection flow

​Server messages

​partial_utterance

​utterance

​done

​error

​WebSocket close codes

​Related

Endpoint

Authentication

Supported audio formats

Query parameters

Language identification

Speaker labels

Speaker capacity

Utterances that span a speaker change

With endpointing off

Connection flow

Server messages

`partial_utterance`

`utterance`

`done`

`error`

WebSocket close codes

Related