Speech-to-Text Streaming English

​

Endpoint

wss://platform.modulate.ai/api/velma-2-stt-streaming-english-v2

​

Authentication

wss://platform.modulate.ai/api/velma-2-stt-streaming-english-v2?api_key=YOUR_API_KEY&audio_format=ogg

​

Supported audio formats

​

Query parameters

​

Utterance segmentation (endpointing)

• Each partial_utterance contains the complete transcript of the current segment so far (still replace, never append).
• Each time a pause ends a segment, you immediately receive a final utterance for it — final text arrives shortly after the speaker stops, instead of at end-of-stream. Each final carries start_ms and duration_ms for its speech.
• The full transcript is the concatenation of the final utterance texts, in order.
• Every connection still ends with at least one final utterance (a stream containing no speech yields one with empty text), followed by done.

​

Speaker labels

​

Speaker capacity

​

Utterances that span a speaker change

​

With endpointing off

​

Connection flow

1. Connect to the WebSocket endpoint with api_key, audio_format, (for raw PCM) sample_rate and num_channels, and optionally endpointing and diarize.
2. Stream audio as binary WebSocket frames. Frames can be any size; 4–64 KB is typical.
3. Receive partial_utterance JSON messages every ~1.5 seconds. Each contains the complete transcript of its scope so far — replace any previously displayed partial, do not append.
4. With endpointing=true, receive a final utterance message each time a pause ends a segment.
5. Send an empty text frame ("") to signal end of audio.
6. Receive the final utterance for any content not already finalized at a pause (the whole stream with endpointing off; skipped when the last segment was already finalized mid-stream).
7. Receive a done message with total audio duration.
8. The connection closes automatically.

​

Server messages

​

partial_utterance

{
  "type": "partial_utterance",
  "partial_utterance": {
    "text": "Hello, how are you",
    "is_final": false
  }
}

​

utterance

{
  "type": "utterance",
  "utterance": {
    "text": "Hello, how are you doing today?",
    "is_final": true,
    "start_ms": 0,
    "duration_ms": 2360
  }
}

{
  "type": "utterance",
  "utterance": {
    "text": "Good morning, thanks for calling.",
    "is_final": true,
    "start_ms": 8120,
    "duration_ms": 4440,
    "speaker": 0,
    "n_speakers": 2,
    "speakers_capped": false,
    "speaker_purity": 1.0
  }
}

​

done

{
  "type": "done",
  "duration_ms": 14253
}

​

error

{
  "type": "error",
  "error": "Invalid audio_format='xyz'."
}

{ "type": "error", "error": "Invalid diarize='yes'. Expected 'true' or 'false'." }

{ "type": "error", "error": "diarize=true is not available on this endpoint." }

​

WebSocket close codes

​

Rate limits

• Concurrent connection limits apply per model; exceeding them rejects the handshake with close code 4030.

​

Examples

import asyncio
import json
import os
import websockets

API_KEY = os.environ["MODULATE_API_KEY"]
AUDIO_FILE = "recording.ogg"
CHUNK_SIZE = 8192

async def transcribe():
    url = (
        f"wss://platform.modulate.ai/api/velma-2-stt-streaming-english-v2"
        f"?api_key={API_KEY}&audio_format=ogg"
    )

    async with websockets.connect(url, max_size=None) as ws:
        async def send_audio():
            with open(AUDIO_FILE, "rb") as f:
                while chunk := f.read(CHUNK_SIZE):
                    await ws.send(chunk)
            await ws.send("")  # signal end-of-stream

        send_task = asyncio.create_task(send_audio())

        try:
            async for msg in ws:
                data = json.loads(msg)
                if data["type"] == "partial_utterance":
                    # Replace any previously displayed partial — not a delta
                    print(f"\r[partial] {data['partial_utterance']['text']}", end="", flush=True)
                elif data["type"] == "utterance":
                    u = data["utterance"]
                    # "speaker" is present only with diarize=true; None means undetermined
                    who = f" speaker {u['speaker']}" if u.get("speaker") is not None else ""
                    print(f"\n[final{who}] {u['text']}")
                elif data["type"] == "done":
                    print(f"\nDone. Duration: {data['duration_ms']}ms")
                    break
                elif data["type"] == "error":
                    print(f"\nError: {data['error']}")
                    break
        finally:
            if not send_task.done():
                send_task.cancel()

asyncio.run(transcribe())

const WebSocket = require("ws");
const fs = require("fs");

const API_KEY = process.env.MODULATE_API_KEY;
const AUDIO_FILE = "recording.ogg";
const CHUNK_SIZE = 8192;

const url = new URL("wss://platform.modulate.ai/api/velma-2-stt-streaming-english-v2");
url.searchParams.set("api_key", API_KEY);
url.searchParams.set("audio_format", "ogg");

const ws = new WebSocket(url.toString());

ws.on("open", () => {
  const stream = fs.createReadStream(AUDIO_FILE, { highWaterMark: CHUNK_SIZE });
  stream.on("data", (chunk) => ws.send(chunk));
  stream.on("end", () => ws.send(""));  // signal end-of-stream
});

ws.on("message", (data) => {
  const msg = JSON.parse(data.toString());
  if (msg.type === "partial_utterance") {
    // Each partial is the full transcript so far — replace, don't append
    process.stdout.write(`\r[partial] ${msg.partial_utterance.text}`);
  } else if (msg.type === "utterance") {
    // "speaker" is present only with diarize=true; null means undetermined
    const who = msg.utterance.speaker != null ? ` speaker ${msg.utterance.speaker}` : "";
    console.log(`\n[final${who}] ${msg.utterance.text}`);
  } else if (msg.type === "done") {
    console.log(`\nDone. Duration: ${msg.duration_ms}ms`);
    ws.close();
  } else if (msg.type === "error") {
    console.error("\nError:", msg.error);
    ws.close();
  }
});

ws.on("error", (err) => console.error("WebSocket error:", err.message));
ws.on("close", (code) => console.log(`Connection closed: ${code}`));

​

Related

• Which API should I use? — when Streaming v2 is the right choice vs STT Streaming or batch
• STT Streaming — multilingual streaming with speaker diarization and enrichments
• Authentication and rate limits