> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulate.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# PII/PHI Redaction Streaming
> Real-time PII/PHI redaction over WebSocket — receive a redacted transcript and a redacted MP3 clip per utterance.
Real-time PII/PHI redaction over WebSocket. Streams audio to the server and receives, per utterance, a redacted transcript and a redacted MP3 clip with the PII/PHI ranges silenced.
## Endpoint
```text theme={null}
wss://platform.modulate.ai/api/velma-2-pii-phi-redaction-streaming
```
## Authentication
Pass your API key as a query parameter when opening the connection.
```text theme={null}
wss://platform.modulate.ai/api/velma-2-pii-phi-redaction-streaming?api_key=YOUR_API_KEY
```
Unlike the batch endpoints, the streaming API does not use an `X-API-Key` header. The key must be in the query string at connection time.
See [Authentication and rate limits](/guides/authentication) for how to obtain and manage API keys.
## Supported audio formats
**Self-describing formats** (auto-detected from file headers — no extra parameters needed):
AAC, AIFF, FLAC, MP3, OGG, WAV, WebM
**OGG / Opus:** OGG is a container that may carry Opus-encoded audio. Pass `audio_format=ogg`, not `audio_format=opus`.
**Raw / headerless formats** (require `audio_format`, `sample_rate`, and `num_channels`):
`s8`, `s16le`, `s16be`, `s24le`, `s24be`, `s32le`, `s32be`, `u8`, `u16le`, `u16be`, `u24le`, `u24be`, `u32le`, `u32be`, `f32le`, `f32be`, `f64le`, `f64be`, `mulaw`, `alaw`
Valid sample rates: `8000`, `11025`, `16000`, `22050`, `32000`, `44100`, `48000`, `96000`
## Query parameters
| Parameter | Type | Default | Description |
| - | - | - | - |
| `api_key` | string | — | **Required.** Your API key |
| `speaker_diarization` | boolean | `true` | Identify and label distinct speakers |
| `audio_format` | string | (auto-detect) | Audio encoding format. Omit for self-describing formats; required for raw formats |
| `sample_rate` | integer | — | Sample rate in Hz. Required for raw formats only |
| `num_channels` | integer | — | Number of channels (1–8). Required for raw formats only |
| `start_redaction_padding_ms` | integer | `100` | Extra silence (ms) prepended before each redacted audio range |
| `end_redaction_padding_ms` | integer | `0` | Extra silence (ms) appended after each redacted audio range |
| `language` | string | (auto-detect) | Optional language hint as a case-insensitive ISO 639-1 code (e.g. `en`, `fr`). BCP 47 region/script subtags (e.g. `en-US`) are accepted; only the primary language subtag is used. When omitted, the language is detected automatically for each utterance |
## Connection flow
1. Connect to the WebSocket endpoint with `api_key` and any optional parameters.
2. Stream audio data as **binary** WebSocket frames. Frames can be any size.
3. Receive **frame pairs** per utterance: a JSON text frame, optionally followed by a binary MP3 frame.
4. Send an **empty text frame** (`""`) to signal end of audio.
5. Receive a `done` JSON frame, optionally followed by a final binary MP3 frame for any trailing audio.
6. The connection closes automatically.
## Server messages
The server sends frame pairs: a JSON text frame indicating the utterance, optionally followed by a binary MP3 frame. The `redacted_audio` field in the JSON tells you whether a binary frame follows.
### `utterance` (JSON + optional binary MP3)
Sent when a speech segment has been transcribed and redacted.
**JSON frame:**
```json theme={null}
{
"type": "utterance",
"utterance": {
"utterance_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"text": "Hello, my name is .",
"start_ms": 0,
"duration_ms": 3000,
"speaker": 1,
"language": "en"
},
"redacted_audio": {
"start_ms": 0,
"duration_ms": 3000
}
}
```
When `redacted_audio` is not `null`, a binary MP3 frame follows immediately. It covers the window from the last emitted audio point to the end of this utterance, with PII/PHI ranges silenced.
When `redacted_audio` is `null`, no binary frame follows. This occurs for out-of-order utterances whose audio window was already emitted in a previous clip — the redacted text is still delivered.
#### Utterance fields
| Field | Type | Description |
| - | - | - |
| `utterance_uuid` | string (UUID) | Unique identifier for this utterance |
| `text` | string | Redacted text. Each detected PII/PHI span is replaced with an empty marker tag (e.g. ``, ``, ``), with the surrounding text preserved |
| `start_ms` | integer | Start time in milliseconds from the beginning of the stream |
| `duration_ms` | integer | Duration of the utterance in milliseconds |
| `speaker` | integer | Speaker number, 1-indexed. Consistent within a connection |
| `language` | string | Detected language code (e.g. `"en"`, `"fr"`) |
#### Redacted audio info fields
| Field | Type | Description |
| - | - | - |
| `start_ms` | integer | Start time of the MP3 clip in milliseconds from the beginning of the stream |
| `duration_ms` | integer | Duration of the MP3 clip in milliseconds |
### `done` (JSON + optional binary MP3)
Sent after all audio has been processed, in response to the end-of-stream signal.
```json theme={null}
{
"type": "done",
"duration_ms": 45000,
"trailing_redacted_audio": {
"start_ms": 43000,
"duration_ms": 2000
}
}
```
When `trailing_redacted_audio` is not `null`, a binary MP3 frame follows containing any remaining audio after the last utterance, with any applicable PII/PHI ranges silenced.
When `trailing_redacted_audio` is `null`, no binary frame follows.
### `error`
Sent if redaction fails during processing. The connection closes after this message. No binary frame follows.
```json theme={null}
{
"type": "error",
"error": "Internal server error"
}
```
## WebSocket close codes
| Code | Meaning |
| - | - |
| `1000` | Normal closure after a successful `done` message |
| `1003` | Invalid connection parameters (unsupported `audio_format`, invalid `sample_rate`, `num_channels`, or `language`; raw format missing `sample_rate`/`num_channels`) |
| `1011` | Internal server error, or processing ended before completion |
| `4001` | The `api_key` query parameter is missing or invalid |
| `4003` | The request is not permitted |
| `4004` | The API key does not have access to this model |
| `4029` | Insufficient credits |
| `4030` | Concurrent request limit reached |
| `4031` | Monthly usage limit reached |
An `error` JSON message is sent before the connection closes (except on `1000`).
## Rate limits
* Concurrent connection limits apply per model.
* Connections that exceed limits are rejected during the WebSocket handshake with close code `4030`.
See [Authentication and rate limits](/guides/authentication) for retry guidance.
## Redaction tags
Each detected PII/PHI span is replaced with an empty marker tag in the transcript text: `` for health information and `` for personal information, where CATEGORY identifies the detected entity type. The surrounding text is preserved. For more detail, see the **PII/PHI Redaction (Batch)** API reference.
Currently, all entity types the model can detect are redacted. Per-entity configurability is planned for a future release.
## Examples
```python theme={null}
import asyncio
import json
import aiohttp
API_KEY = "YOUR_API_KEY"
AUDIO_FILE = "recording.ogg"
CHUNK_SIZE = 8192
async def redact_streaming():
url = (
"wss://platform.modulate.ai/api/velma-2-pii-phi-redaction-streaming"
f"?api_key={API_KEY}"
"&speaker_diarization=true"
"&start_redaction_padding_ms=100"
"&end_redaction_padding_ms=0"
)
utterances = []
audio_clips = []
async with aiohttp.ClientSession() as session:
async with session.ws_connect(url) as ws:
async def send_audio():
with open(AUDIO_FILE, "rb") as f:
while chunk := f.read(CHUNK_SIZE):
await ws.send_bytes(chunk)
await asyncio.sleep(CHUNK_SIZE / 4000)
await ws.send_str("")
send_task = asyncio.create_task(send_audio())
try:
done = False
async for msg in ws:
if msg.type == aiohttp.WSMsgType.TEXT:
data = json.loads(msg.data)
if data["type"] == "utterance":
u = data["utterance"]
utterances.append(u)
print(f"[Speaker {u['speaker']}] ({u['language']}) {u['start_ms']}ms: {u['text']}")
elif data["type"] == "done":
print(f"\nDone. Duration: {data['duration_ms']}ms")
done = True
if not data.get("trailing_redacted_audio"):
break
elif data["type"] == "error":
print(f"Error: {data['error']}")
break
elif msg.type == aiohttp.WSMsgType.BINARY:
audio_clips.append(msg.data)
if done:
break
elif msg.type in (
aiohttp.WSMsgType.ERROR,
aiohttp.WSMsgType.CLOSE,
aiohttp.WSMsgType.CLOSED,
):
break
finally:
if not send_task.done():
send_task.cancel()
if audio_clips:
with open("redacted.mp3", "wb") as f:
for clip in audio_clips:
f.write(clip)
asyncio.run(redact_streaming())
```
```javascript theme={null}
const WebSocket = require("ws");
const fs = require("fs");
const API_KEY = "YOUR_API_KEY";
const AUDIO_FILE = "recording.ogg";
const CHUNK_SIZE = 8192;
const url = new URL(
"wss://platform.modulate.ai/api/velma-2-pii-phi-redaction-streaming"
);
url.searchParams.set("api_key", API_KEY);
url.searchParams.set("speaker_diarization", "true");
url.searchParams.set("start_redaction_padding_ms", "100");
url.searchParams.set("end_redaction_padding_ms", "0");
const ws = new WebSocket(url.toString());
const utterances = [];
const audioClips = [];
let isDone = false;
ws.on("open", () => {
const stream = fs.createReadStream(AUDIO_FILE, { highWaterMark: CHUNK_SIZE });
stream.on("data", (chunk) => ws.send(chunk));
stream.on("end", () => ws.send(""));
});
ws.on("message", (data, isBinary) => {
if (isBinary) {
audioClips.push(data);
if (isDone) finalize();
return;
}
const msg = JSON.parse(data.toString());
if (msg.type === "utterance") {
utterances.push(msg.utterance);
console.log(
`[Speaker ${msg.utterance.speaker}] (${msg.utterance.language}) ` +
`${msg.utterance.start_ms}ms: ${msg.utterance.text}`
);
} else if (msg.type === "done") {
console.log(`\nDone. Duration: ${msg.duration_ms}ms`);
isDone = true;
if (!msg.trailing_redacted_audio) finalize();
} else if (msg.type === "error") {
console.error("Error:", msg.error);
ws.close();
}
});
function finalize() {
if (audioClips.length > 0) {
const combined = Buffer.concat(audioClips);
fs.writeFileSync("redacted.mp3", combined);
}
ws.close();
}
ws.on("error", (err) => console.error("WebSocket error:", err.message));
```
WebSocket APIs cannot be tested with cURL. For command-line testing, use [`websocat`](https://github.com/vi/websocat).
## Related
* [Which API should I use?](/guides/which-api) — PII/PHI redaction vs PII/PHI tagging, batch vs streaming
* [Transcription](/get-started/stt#signals) — PII/PHI tagging option in the STT transcription APIs
* [Authentication and rate limits](/guides/authentication)
## AsyncAPI
````yaml api/velma_2_pii_phi_redaction_streaming.yaml velma2PiiPhiRedactionStreaming
id: velma2PiiPhiRedactionStreaming
title: Velma2 pii phi redaction streaming
description: >
Establish a WebSocket connection for real-time PII/PHI redaction. The service
transcribes audio in real-time, detects PII/PHI in the text, redacts the
transcript, and returns redacted audio clips as binary MP3 frames.
## Connection Flow
1. Connect with `api_key` and optional query parameters
2. Connection will be accepted if authentication succeeds and limits are not
exceeded
3. Stream audio data as binary WebSocket frames
4. Receive utterance results as JSON+binary frame pairs (redacted text + MP3
clip)
5. Send empty text frame ("") when audio stream ends
6. Receive final "done" message with optional trailing audio
7. Connection closes automatically after completion
## Error Handling
If an error occurs during processing, the server sends an error JSON message
and
closes the connection. Connection rejections during handshake use WebSocket
close codes.
## WebSocket Close Codes
- `1000`: Normal closure after the `done` message, or after the
connection completes.
- `1003`: Invalid connection parameter (e.g. unsupported `audio_format`,
`sample_rate`,
`num_channels`, an invalid `language`, or a raw format missing
`sample_rate`/`num_channels`). An `error` message is sent before the close.
- `1011`: Internal server error, or processing ended before completion.
An `error` message is sent before the close; when processing ends
without a `done` message, the `error` message is
"Unable to complete the request. Please try again."
- `4001`: The `api_key` query parameter is missing or invalid. An
`error` message is sent before the close.
- `4003`: The request is not permitted. An `error` message is sent
before the close.
- `4004`: The API key does not have access to this model. An `error`
message is sent before the close.
- `4029`: The request could not be completed due to insufficient credits.
An `error` message is sent before the close.
- `4030`: Concurrent request limit reached. An `error` message is sent
before the close.
- `4031`: Monthly usage limit reached. An `error` message is sent before
the close.
servers:
- id: production
protocol: wss
host: platform.modulate.ai
bindings: []
variables: []
address: /api/velma-2-pii-phi-redaction-streaming
parameters: []
bindings:
- protocol: ws
version: latest
value:
query:
type: object
required:
- api_key
properties:
api_key:
type: string
description: >
Your API key for authentication. Required. Must be provided as a
query
parameter when establishing the WebSocket connection; the
connection is
rejected during the handshake if it is missing.
speaker_diarization:
type: boolean
default: true
description: >
Enable speaker diarization to identify different speakers in the
audio stream.
When enabled, each utterance includes a speaker identifier
(1-indexed integer).
audio_format:
type: string
description: >
Audio encoding format. Determines how the service interprets
incoming audio data.
**Self-describing formats** (`wav`, `mp3`, `ogg`, `flac`, `webm`,
`aac`, `aiff`):
These include container formats (AIFF, OGG, WAV, WebM) and codecs
whose streams
carry per-frame headers (AAC/ADTS, FLAC, MP3). When omitted
(default), the service
auto-detects the format from these headers. You may specify the
format explicitly
to override auto-detection. Do not specify `sample_rate` or
`num_channels` for
self-describing formats - the headers already carry this metadata.
**Raw/headerless formats** (`s8`..`f64be`, `mulaw`, `alaw`):
These formats are raw sample data with no headers. When using a
raw format, you
must also specify `sample_rate` and `num_channels` so the service
knows how to
interpret the byte stream. Omitting either will result in an
error.
enum:
- s8
- s16le
- s16be
- s24le
- s24be
- s32le
- s32be
- u8
- u16le
- u16be
- u24le
- u24be
- u32le
- u32be
- f32le
- f32be
- f64le
- f64be
- mulaw
- alaw
- wav
- mp3
- ogg
- flac
- webm
- aac
- aiff
sample_rate:
type: integer
description: >
Audio sample rate in Hz. Required when `audio_format` is a
raw/headerless format
(PCM, mu-law, A-law). Do not specify for self-describing formats
(WAV, MP3, OGG,
etc.) - their headers already include this metadata. Cannot be
specified without
`audio_format`.
enum:
- 8000
- 11025
- 16000
- 22050
- 32000
- 44100
- 48000
- 96000
num_channels:
type: integer
minimum: 1
maximum: 8
description: >
Number of audio channels (1-8). Required when `audio_format` is a
raw/headerless
format (PCM, mu-law, A-law). Do not specify for self-describing
formats (WAV, MP3,
OGG, etc.) - their headers already include this metadata. Use 1
for mono, 2 for
stereo. Cannot be specified without `audio_format`.
start_redaction_padding_ms:
type: integer
minimum: 0
default: 100
description: >
Additional silence in milliseconds to prepend before each redacted
audio range.
Extends the silenced region earlier in time to provide a buffer
before PII/PHI content.
end_redaction_padding_ms:
type: integer
minimum: 0
default: 0
description: >
Additional silence in milliseconds to append after each redacted
audio range.
Extends the silenced region later in time to provide a buffer
after PII/PHI content.
language:
type: string
pattern: ^[A-Za-z]{2,3}([-_][A-Za-z0-9]+)*$
description: >
Optional language hint as a case-insensitive ISO 639-1 code (e.g.
"en",
"fr"). BCP 47 region/script subtags (e.g. "en-US") are accepted;
only the
primary language subtag is used. When omitted, the language is
detected
automatically for each utterance.
schemaProperties:
- name: query
type: object
required: false
properties:
- name: api_key
type: string
description: >
Your API key for authentication. Required. Must be provided as a
query
parameter when establishing the WebSocket connection; the
connection is
rejected during the handshake if it is missing.
required: true
- name: speaker_diarization
type: boolean
description: >
Enable speaker diarization to identify different speakers in the
audio stream.
When enabled, each utterance includes a speaker identifier
(1-indexed integer).
required: false
- name: audio_format
type: string
description: >
Audio encoding format. Determines how the service interprets
incoming audio data.
**Self-describing formats** (`wav`, `mp3`, `ogg`, `flac`, `webm`,
`aac`, `aiff`):
These include container formats (AIFF, OGG, WAV, WebM) and codecs
whose streams
carry per-frame headers (AAC/ADTS, FLAC, MP3). When omitted
(default), the service
auto-detects the format from these headers. You may specify the
format explicitly
to override auto-detection. Do not specify `sample_rate` or
`num_channels` for
self-describing formats - the headers already carry this metadata.
**Raw/headerless formats** (`s8`..`f64be`, `mulaw`, `alaw`):
These formats are raw sample data with no headers. When using a
raw format, you
must also specify `sample_rate` and `num_channels` so the service
knows how to
interpret the byte stream. Omitting either will result in an
error.
enumValues:
- s8
- s16le
- s16be
- s24le
- s24be
- s32le
- s32be
- u8
- u16le
- u16be
- u24le
- u24be
- u32le
- u32be
- f32le
- f32be
- f64le
- f64be
- mulaw
- alaw
- wav
- mp3
- ogg
- flac
- webm
- aac
- aiff
required: false
- name: sample_rate
type: integer
description: >
Audio sample rate in Hz. Required when `audio_format` is a
raw/headerless format
(PCM, mu-law, A-law). Do not specify for self-describing formats
(WAV, MP3, OGG,
etc.) - their headers already include this metadata. Cannot be
specified without
`audio_format`.
enumValues:
- 8000
- 11025
- 16000
- 22050
- 32000
- 44100
- 48000
- 96000
required: false
- name: num_channels
type: integer
description: >
Number of audio channels (1-8). Required when `audio_format` is a
raw/headerless
format (PCM, mu-law, A-law). Do not specify for self-describing
formats (WAV, MP3,
OGG, etc.) - their headers already include this metadata. Use 1
for mono, 2 for
stereo. Cannot be specified without `audio_format`.
required: false
- name: start_redaction_padding_ms
type: integer
description: >
Additional silence in milliseconds to prepend before each redacted
audio range.
Extends the silenced region earlier in time to provide a buffer
before PII/PHI content.
required: false
- name: end_redaction_padding_ms
type: integer
description: >
Additional silence in milliseconds to append after each redacted
audio range.
Extends the silenced region later in time to provide a buffer
after PII/PHI content.
required: false
- name: language
type: string
description: >
Optional language hint as a case-insensitive ISO 639-1 code (e.g.
"en",
"fr"). BCP 47 region/script subtags (e.g. "en-US") are accepted;
only the
primary language subtag is used. When omitted, the language is
detected
automatically for each utterance.
required: false
operations:
- &ref_3
id: streamAudio
title: Stream audio
description: Stream audio for real-time PII/PHI redaction
type: send
messages:
- &ref_9
id: audioChunk
contentType: application/octet-stream
payload:
- type: string
format: binary
x-parser-schema-id:
name: Audio Data Chunk
description: Audio data chunk for real-time redaction
headers: []
jsonPayloadSchema:
type: string
format: binary
x-parser-schema-id:
title: Audio Data Chunk
description: Audio data chunk for real-time redaction
example: '{}'
bindings: []
extensions:
- id: x-parser-unique-object-id
value: audioChunk
- &ref_10
id: endOfStream
contentType: text/plain
payload:
- type: string
const: ''
x-parser-schema-id:
name: End of Audio Stream
description: Empty text frame signaling end of audio stream
headers: []
jsonPayloadSchema:
type: string
const: ''
x-parser-schema-id:
title: End of Audio Stream
description: Empty text frame signaling end of audio stream
example: '""'
bindings: []
extensions:
- id: x-parser-unique-object-id
value: endOfStream
bindings: []
extensions: &ref_1
- id: x-parser-unique-object-id
value: velma2PiiPhiRedactionStreaming
- &ref_2
id: receiveRedaction
title: Receive redaction
description: Receive redacted transcription results and audio clips from the server
type: receive
messages:
- &ref_4
id: utteranceResult
contentType: application/json
payload:
- name: Utterance Result
description: >
A completed utterance with redacted text. When `redacted_audio` is
not null,
the next frame from the server is a binary MP3 frame containing
the redacted
audio clip. When `redacted_audio` is null (out-of-order utterance
whose audio
was already emitted), no binary frame follows.
type: object
properties:
- name: type
type: string
description: utterance
required: true
- name: utterance
type: object
required: true
properties:
- name: utterance_uuid
type: string
description: Unique identifier for this utterance.
required: true
- name: text
type: string
description: >
The redacted text for this utterance. Each detected PII or
PHI span is
replaced with an empty marker tag - `` for
health information
and `` for personal
information, where
CATEGORY identifies the detected entity type - with the
surrounding text
preserved.
required: true
- name: start_ms
type: integer
description: >
The start time of this utterance in milliseconds, relative
to the beginning
of the audio stream.
required: true
- name: duration_ms
type: integer
description: The duration of this utterance in milliseconds.
required: true
- name: speaker
type: integer
description: >
Speaker identifier for this utterance (1-indexed). When
speaker diarization is
enabled, this identifies which speaker produced this
utterance. Speaker numbers
are consistent within a single connection but may vary
between different connections.
required: true
- name: language
type: string
description: >
The automatically detected language code for this
utterance (e.g., "en" for English,
"fr" for French).
required: true
- name: redacted_audio
type: object
description: >
Describes the audio clip that follows as the next binary
frame.
`start_ms` and `duration_ms` indicate the time window in the
original
audio stream that the clip covers (from the last emitted point
to the
end of this utterance). Null when the utterance's audio was
already
emitted in a previous window (out-of-order); in that case, no
binary
frame follows.
required: true
properties:
- name: start_ms
type: integer
description: >
Start time of the audio clip in milliseconds, relative to
the beginning
of the audio stream. This is the point from which the
binary MP3 data begins.
required: true
- name: duration_ms
type: integer
description: Duration of the audio clip in milliseconds.
required: true
headers: []
jsonPayloadSchema:
type: object
required:
- type
- utterance
- redacted_audio
properties:
type:
type: string
const: utterance
x-parser-schema-id:
utterance:
type: object
required:
- utterance_uuid
- text
- start_ms
- duration_ms
- speaker
- language
properties:
utterance_uuid:
type: string
format: uuid
description: Unique identifier for this utterance.
example: a1b2c3d4-e5f6-7890-abcd-ef1234567890
x-parser-schema-id:
text:
type: string
description: >
The redacted text for this utterance. Each detected PII or
PHI span is
replaced with an empty marker tag - `` for health
information
and `` for personal
information, where
CATEGORY identifies the detected entity type - with the
surrounding text
preserved.
example: Hello, my name is .
x-parser-schema-id:
start_ms:
type: integer
minimum: 0
description: >
The start time of this utterance in milliseconds, relative
to the beginning
of the audio stream.
example: 0
x-parser-schema-id:
duration_ms:
type: integer
minimum: 0
description: The duration of this utterance in milliseconds.
example: 2500
x-parser-schema-id:
speaker:
type: integer
description: >
Speaker identifier for this utterance (1-indexed). When
speaker diarization is
enabled, this identifies which speaker produced this
utterance. Speaker numbers
are consistent within a single connection but may vary
between different connections.
example: 1
x-parser-schema-id:
language:
type: string
description: >
The automatically detected language code for this utterance
(e.g., "en" for English,
"fr" for French).
example: en
x-parser-schema-id:
x-parser-schema-id: StreamingUtterance
redacted_audio:
oneOf:
- &ref_0
type: object
required:
- start_ms
- duration_ms
properties:
start_ms:
type: integer
minimum: 0
description: >
Start time of the audio clip in milliseconds, relative
to the beginning
of the audio stream. This is the point from which the
binary MP3 data begins.
example: 0
x-parser-schema-id:
duration_ms:
type: integer
minimum: 0
description: Duration of the audio clip in milliseconds.
example: 3000
x-parser-schema-id:
x-parser-schema-id: RedactedAudioInfo
- type: 'null'
x-parser-schema-id:
description: >
Describes the audio clip that follows as the next binary frame.
`start_ms` and `duration_ms` indicate the time window in the
original
audio stream that the clip covers (from the last emitted point
to the
end of this utterance). Null when the utterance's audio was
already
emitted in a previous window (out-of-order); in that case, no
binary
frame follows.
x-parser-schema-id:
x-parser-schema-id:
title: Utterance Result
description: >
A completed utterance with redacted text. When `redacted_audio` is not
null,
the next frame from the server is a binary MP3 frame containing the
redacted
audio clip. When `redacted_audio` is null (out-of-order utterance
whose audio
was already emitted), no binary frame follows.
example: |-
{
"type": "utterance",
"utterance": {
"utterance_uuid": "b2c3d4e5-f6a7-8901-bcde-f12345678901",
"text": "Hello, my name is .",
"start_ms": 0,
"duration_ms": 3000,
"speaker": 1,
"language": "en"
},
"redacted_audio": {
"start_ms": 0,
"duration_ms": 3000
}
}
bindings: []
extensions:
- id: x-parser-unique-object-id
value: utteranceResult
- &ref_5
id: redactedAudioClip
contentType: audio/mpeg
payload:
- type: string
format: binary
x-parser-schema-id:
name: Redacted Audio Clip
description: >
Binary MP3 data for the redacted audio window. Sent immediately
after an
utterance JSON frame that has a non-null `redacted_audio` field.
headers: []
jsonPayloadSchema:
type: string
format: binary
x-parser-schema-id:
title: Redacted Audio Clip
description: >
Binary MP3 data for the redacted audio window. Sent immediately after
an
utterance JSON frame that has a non-null `redacted_audio` field.
example: '{}'
bindings: []
extensions:
- id: x-parser-unique-object-id
value: redactedAudioClip
- &ref_6
id: errorResult
contentType: application/json
payload:
- name: Error
description: An error occurred during redaction. No binary frame follows.
type: object
properties:
- name: type
type: string
description: error
required: true
- name: error
type: string
description: Human-readable error message
required: true
headers: []
jsonPayloadSchema:
type: object
required:
- type
- error
properties:
type:
type: string
const: error
x-parser-schema-id:
error:
type: string
description: Human-readable error message
x-parser-schema-id:
x-parser-schema-id:
title: Error
description: An error occurred during redaction. No binary frame follows.
example: |-
{
"type": "error",
"error": "Internal server error"
}
bindings: []
extensions:
- id: x-parser-unique-object-id
value: errorResult
- &ref_7
id: doneResult
contentType: application/json
payload:
- name: Redaction Complete
description: >
Redaction completed successfully. When `trailing_redacted_audio`
is not null,
the next frame is a binary MP3 frame containing the remaining
audio after the
last utterance. When null, no binary frame follows.
type: object
properties:
- name: type
type: string
description: done
required: true
- name: duration_ms
type: integer
description: Total duration of audio processed in milliseconds
required: true
- name: trailing_redacted_audio
type: object
description: >
Describes any remaining audio after the last utterance. When
not null,
a binary MP3 frame follows containing the trailing audio with
any
applicable redaction applied. Null when the last utterance
already
covered the end of the audio.
required: true
properties:
- name: start_ms
type: integer
description: >
Start time of the audio clip in milliseconds, relative to
the beginning
of the audio stream. This is the point from which the
binary MP3 data begins.
required: true
- name: duration_ms
type: integer
description: Duration of the audio clip in milliseconds.
required: true
headers: []
jsonPayloadSchema:
type: object
required:
- type
- duration_ms
- trailing_redacted_audio
properties:
type:
type: string
const: done
x-parser-schema-id:
duration_ms:
type: integer
minimum: 0
description: Total duration of audio processed in milliseconds
x-parser-schema-id:
trailing_redacted_audio:
oneOf:
- *ref_0
- type: 'null'
x-parser-schema-id:
description: >
Describes any remaining audio after the last utterance. When not
null,
a binary MP3 frame follows containing the trailing audio with
any
applicable redaction applied. Null when the last utterance
already
covered the end of the audio.
x-parser-schema-id:
x-parser-schema-id:
title: Redaction Complete
description: >
Redaction completed successfully. When `trailing_redacted_audio` is
not null,
the next frame is a binary MP3 frame containing the remaining audio
after the
last utterance. When null, no binary frame follows.
example: |-
{
"type": "done",
"duration_ms": 45000,
"trailing_redacted_audio": {
"start_ms": 43000,
"duration_ms": 2000
}
}
bindings: []
extensions:
- id: x-parser-unique-object-id
value: doneResult
- &ref_8
id: trailingAudioClip
contentType: audio/mpeg
payload:
- type: string
format: binary
x-parser-schema-id:
name: Trailing Audio Clip
description: >
Binary MP3 data for the trailing audio after the last utterance.
Sent
immediately after a `done` JSON frame that has a non-null
`trailing_redacted_audio` field.
headers: []
jsonPayloadSchema:
type: string
format: binary
x-parser-schema-id:
title: Trailing Audio Clip
description: |
Binary MP3 data for the trailing audio after the last utterance. Sent
immediately after a `done` JSON frame that has a non-null
`trailing_redacted_audio` field.
example: '{}'
bindings: []
extensions:
- id: x-parser-unique-object-id
value: trailingAudioClip
bindings: []
extensions: *ref_1
sendOperations:
- *ref_2
receiveOperations:
- *ref_3
sendMessages:
- *ref_4
- *ref_5
- *ref_6
- *ref_7
- *ref_8
receiveMessages:
- *ref_9
- *ref_10
extensions:
- id: x-parser-unique-object-id
value: velma2PiiPhiRedactionStreaming
securitySchemes: []
````
This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.