> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulate.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Velma Triage
> Analyze a whole conversation for behaviors, topics, sentiment, speaker roles, and a summary, over a diarized transcript. Batch and streaming.
Velma Triage analyzes conversations rather than audio characteristics. It evaluates a configured set of behaviors against the transcript and reports which ones were detected, with the clips that triggered each detection, alongside a conversation type, per-speaker roles, topics, per-topic sentiment, and a summary.
Behaviors come from a catalog of presets, from definitions written for the deployment, or from both.
Velma transcribes as part of its analysis. The diarized transcript is returned as `clips`, so a separate transcription call on the same audio is redundant.
| | Batch | Streaming |
| -------- | --------------------------------------------------- | ----------------------------------------------------- |
| Protocol | HTTP POST | WebSocket |
| Input | A complete recording | A live or in-progress conversation |
| Output | One JSON response | Typed events as analysis develops |
| Use case | Post-call QA, compliance review, backlog processing | Live monitoring, real-time alerting, in-call coaching |
## Velma Triage (batch)
Returns `application/json`. `duration_ms`, `clips`, and `behaviors` are always present. The rest depend on configuration.
| Field | Type | Contents |
| ------------------------ | ---------------- | -------------------------------------------------------- |
| `duration_ms` | integer | Total audio duration. |
| `clips` | array | The diarized transcript, one entry per speaker turn. |
| `behaviors` | array | One entry per behavior evaluated, detected or not. |
| `conversation_type_pick` | object or null | The conversation type chosen for the session. |
| `participant_role_picks` | array | One role assignment per speaker. |
| `topics` | array of strings | Topics extracted from the conversation. |
| `topic_sentiments` | array | Per-speaker sentiment for each topic. |
| `summary` | string or null | Narrative summary. Null when `produce_summary` is false. |
Each entry in `clips` carries `clip_uuid`, `text`, `start_ms`, `duration_ms`, `speaker_label`, `language`, and the optional `emotion`, `accent`, and `deepfake_score` signals, which are null unless enabled in the `stt` block.
Each entry in `behaviors` carries `behavior_uuid`, `behavior_name`, `speaker_label`, `detected`, `confidence`, `evidence_clip_uuids`, `definitive_clip_uuid`, and `reasoning`. Entries appear for every behavior evaluated, so `detected: false` means the behavior was checked and not found.
`conversation_type_pick` and each entry in `participant_role_picks` carry a `confidence` and a `selection_source` of `inferred`, `auto_selected_single_option`, or `default`, which distinguishes a real inference from a fallback.
### Try it
```bash curl theme={null}
curl -X POST https://platform.modulate.ai/api/velma-2-batch \
-H "X-API-Key: $MODULATE_API_KEY" \
-F "upload_file=@recording.mp3" \
-F 'config={"behaviors":["preset:harassment","preset:service_churn"]}'
```
```python Python theme={null}
import os, json, requests
config = {
"behaviors": ["preset:harassment", "preset:service_churn"],
"stt": {"speaker_diarization": True, "emotion_signal": True},
"produce_summary": True,
}
response = requests.post(
"https://platform.modulate.ai/api/velma-2-batch",
headers={"X-API-Key": os.environ["MODULATE_API_KEY"]},
data={"config": json.dumps(config)},
files={"upload_file": open("recording.mp3", "rb")},
)
response.raise_for_status()
result = response.json()
for b in result["behaviors"]:
if b["detected"]:
print(f"{b['behavior_name']} - {b['speaker_label']} ({b.get('confidence')})")
print(result.get("summary"))
```
```json theme={null}
{
"duration_ms": 45000,
"clips": [
{
"clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"text": "Thanks for calling, how can I help?",
"start_ms": 0,
"duration_ms": 2800,
"speaker_label": "Speaker 1",
"language": "en",
"emotion": "Calm",
"accent": null,
"deepfake_score": null
}
],
"conversation_type_pick": {
"conversation_type_uuid": "c1d2e3f4-a5b6-7890-cdef-123456789012",
"name": "Customer support call",
"confidence": 0.93,
"selection_source": "inferred",
"detail": "Agent greeting followed by an account issue."
},
"participant_role_picks": [
{
"speaker_label": "Speaker 1",
"participant_role_uuid": "d2e3f4a5-b6c7-8901-def0-234567890123",
"name": "Support agent",
"confidence": 0.95,
"selection_source": "inferred",
"detail": "Opens the call and handles the request."
}
],
"behaviors": [
{
"behavior_uuid": "e3f4a5b6-c7d8-9012-ef01-345678901234",
"behavior_name": "Service churn",
"speaker_label": "Speaker 2",
"detected": true,
"confidence": 0.81,
"evidence_clip_uuids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"],
"definitive_clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
"reasoning": "The caller states they are considering cancelling."
}
],
"topics": ["billing", "account access"],
"topic_sentiments": [
{ "topic": "billing", "speaker_label": "Speaker 2", "sentiment_score": -0.6, "sentiment_label": "negative" }
],
"summary": "A customer called about a locked account and raised a billing concern."
}
```
### What you can configure
Configuration travels in one `config` form field, holding either the literal string `default` or a JSON-encoded object.
| `config` key | Default | Effect |
| --------------------------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `behaviors` | **nothing evaluated** | What to look for. Each entry is a `preset:` reference or a full behavior definition. |
| `conversation_types` | built-in set | Candidate conversation types. Pass `[]` for no candidates. |
| `participant_roles` | built-in set | Candidate speaker roles. Pass `[]` for no candidates. |
| `stt` | — | Transcription options: `speaker_diarization`, `emotion_signal`, `accent_signal`, `deepfake_signal`, `pii_phi_tagging`, `language`, `custom_terms`. See [Transcription](/get-started/stt) for each. |
| `produce_topics` | `true` | Set `false` to skip topic extraction. |
| `produce_topic_sentiments` | `true` | Set `false` to skip per-topic sentiment. |
| `produce_summary` | `true` | Set `false` to skip the narrative summary. |
| `default_conversation_type` | built-in fallback | Used when no candidate applies. Set to `null` to disable the fallback. |
| `default_participant_role` | built-in fallback | Used when no candidate applies. Set to `null` to disable the fallback. |
Passing a custom `config` without a `behaviors` key evaluates **no behaviors**. There is no implicit run-everything. This differs from `conversation_types` and `participant_roles`, which do fall back to built-in defaults when omitted.
Omitting `config` entirely, or sending the literal `default`, loads a curated built-in configuration that does include behaviors. To evaluate against a broad set, list the `preset:` references explicitly.
```json theme={null}
{
"behaviors": [
"preset:harassment",
"preset:service_churn",
{
"behavior_uuid": "8f14e45f-ceea-467a-9c2b-7f1e4a1b2c3d",
"name": "Unverified account change",
"short_description": "Caller requests an account change without completing verification.",
"detailed_description": "Detected when a participant asks to change account details and the agent proceeds without confirming identity through the documented verification steps.",
"applies_to_conversation_type_uuids": null,
"applies_to_participant_role_uuids": null
}
],
"conversation_types": [
{
"conversation_type_uuid": "c1d2e3f4-a5b6-7890-cdef-123456789012",
"name": "Customer support call",
"short_description": "Inbound support contact.",
"detailed_description": "A customer contacts support about an existing product or account issue."
}
],
"participant_roles": [
{
"participant_role_uuid": "d2e3f4a5-b6c7-8901-def0-234567890123",
"name": "Support agent",
"short_description": "The representative handling the call.",
"detailed_description": "Employee who greets the caller, diagnoses the issue, and takes action.",
"applies_to_conversation_type_uuids": null
}
],
"stt": {
"speaker_diarization": true,
"emotion_signal": true,
"accent_signal": false,
"deepfake_signal": true,
"pii_phi_tagging": true,
"language": "en",
"custom_terms": [
"Modulate",
{ "term": "Velma", "definition": "Conversation intelligence model", "pronunciations": ["VEL-muh"] }
]
},
"produce_topics": true,
"produce_topic_sentiments": true,
"produce_summary": true,
"default_conversation_type": {
"conversation_type_uuid": "f4a5b6c7-d8e9-0123-4567-89abcdef0123",
"name": "Unclassified conversation",
"short_description": "Fallback when no candidate applies.",
"detailed_description": "Used when none of the candidate conversation types match the audio."
},
"default_participant_role": {
"participant_role_uuid": "a5b6c7d8-e9f0-1234-5678-9abcdef01234",
"name": "Unidentified participant",
"short_description": "Fallback when no candidate role applies.",
"detailed_description": "Used when none of the candidate participant roles match a speaker.",
"applies_to_conversation_type_uuids": null
}
}
```
Preset identifiers come from `GET /api/velma-2-batch/list-presets`, which returns each preset's `identifier`, `name`, `short_description`, and `detailed_description`:
```bash theme={null}
curl https://platform.modulate.ai/api/velma-2-batch/list-presets \
-H "X-API-Key: $MODULATE_API_KEY"
```
A malformed `config`, an unknown preset identifier, or a definition missing a required field is rejected with `422`.
### Audio formats
Accepted extensions: `.aac`, `.aiff`, `.flac`, `.mov`, `.mp3`, `.mp4`, `.ogg`, `.opus`, `.wav`, `.webm`.
Maximum file size is 100 MB. Empty files are rejected with `400`.
## Velma Triage (streaming)
The same analysis, emitted as typed JSON events while the conversation is still running. Connect, send the config as the first text frame, then stream audio.
| Event | Payload |
| -------------------- | ------------------------------------------------------------------------------------------------------------------------------------------ |
| `partial_clip` | An in-progress utterance with interim `emotion`, `accent`, and `deepfake_score`. Reports `end_ms`, which grows, rather than `duration_ms`. |
| `clip` | The finalized utterance, reusing the `clip_uuid` of its partials. |
| `clip_update` | Refined `emotion` and `accent` for a clip that already finalized. |
| `behavior_detection` | One behavior's verdict, with evidence clip UUIDs and reasoning. |
| `conversation_type` | The session's conversation type pick. |
| `participant_role` | A role pick for one speaker. |
| `topics` | The current topic list. |
| `topic_sentiment` | Sentiment for one topic and speaker. |
| `summary` | The current summary text. |
| `done` | `duration_ms`. The server closes after this. |
| `error` | `error` string. The server closes after this. |
Three events have update semantics worth building for:
* `partial_clip` is a transient preview. Multiple partials share one `clip_uuid`, and the eventual `clip` reuses it, so a run of partials correlates with its final clip.
* `clip_update` can arrive any number of times for a finalized clip, always before `done`. The latest value for each field supersedes the value on the `clip` event and on any earlier update.
* `topics` and `summary` fully replace the previous event of that type. Never merge them. `topic_sentiment` supersedes an earlier event for the same topic and speaker.
### Try it
```python Python theme={null}
import os, asyncio, json, websockets
API_KEY = os.environ["MODULATE_API_KEY"]
AUDIO_FILE = "recording.mp3"
CHUNK_SIZE = 4096
config = {"behaviors": ["preset:harassment", "preset:service_churn"]}
async def analyze():
url = f"wss://platform.modulate.ai/api/velma-2-streaming?api_key={API_KEY}"
clips = {}
async with websockets.connect(url) as ws:
# The config must be the first text frame, before any audio.
await ws.send(json.dumps(config))
async def send():
with open(AUDIO_FILE, "rb") as f:
while chunk := f.read(CHUNK_SIZE):
await ws.send(chunk)
await ws.send("") # end of stream
async def receive():
async for message in ws:
event = json.loads(message)
kind = event["type"]
if kind == "partial_clip":
print(f" ... {event['partial_clip']['text']}", end="\r")
elif kind == "clip":
clip = event["clip"]
clips[clip["clip_uuid"]] = clip
print(f"[{clip['start_ms']}ms] {clip['speaker_label']}: {clip['text']}")
elif kind == "clip_update":
update = event["clip_update"]
clips.get(update["clip_uuid"], {}).update(
{k: v for k, v in update.items() if k != "clip_uuid"}
)
elif kind == "behavior_detection":
d = event["detection"]
if d["detected"]:
print(f"{d['behavior_name']} - {d['speaker_label']}")
elif kind == "summary":
print(f"Summary: {event['text']}")
elif kind == "done":
print(f"Done: {event['duration_ms']}ms")
break
elif kind == "error":
raise RuntimeError(event["error"])
await asyncio.gather(send(), receive())
asyncio.run(analyze())
```
```json theme={null}
{ "type": "partial_clip", "partial_clip": { "clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "text": "Thanks for calling, how", "start_ms": 0, "end_ms": 1400, "speaker_label": null, "emotion": null, "accent": null, "deepfake_score": null } }
{ "type": "clip", "clip": { "clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "text": "Thanks for calling, how can I help?", "start_ms": 0, "duration_ms": 2800, "speaker_label": "Speaker 1", "language": "en", "emotion": null, "accent": null, "deepfake_score": null } }
{ "type": "clip_update", "clip_update": { "clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "emotion": "Calm", "accent": "American" } }
{ "type": "behavior_detection", "detection": { "behavior_uuid": "e3f4a5b6-c7d8-9012-ef01-345678901234", "behavior_name": "Service churn", "speaker_label": "Speaker 2", "detected": true, "confidence": 0.81, "evidence_clip_uuids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"], "reasoning": "The caller states they are considering cancelling." } }
{ "type": "topics", "topics": ["billing", "account access"] }
{ "type": "done", "duration_ms": 45000 }
```
WebSocket endpoints cannot be exercised with cURL. For command-line testing use [websocat](https://github.com/vi/websocat).
### What you can configure
The config object is the same one used by batch, sent as the first text frame instead of a form field. Because it rides on that frame, `language` and `custom_terms` in the `stt` block travel with it and need no separate transport.
| Query parameter | Default | Effect |
| --------------- | ------------------ | --------------------------------------------------------------------------------------- |
| `api_key` | *required* | The API key. WebSocket connections authenticate through the query string, not a header. |
| `audio_format` | auto-detect | Optional for self-describing containers. Required for raw formats. |
| `sample_rate` | *required for raw* | Sample rate in Hz. Cannot be sent without `audio_format`. |
| `num_channels` | *required for raw* | 1 to 8. Cannot be sent without `audio_format`. |
The config frame must arrive before any audio. Sending audio first closes the connection with `1003`.
This endpoint reports authentication failure as close code `4003`. It does not use `4001`.
### Audio formats
**Self-describing formats.** `aac`, `aiff`, `flac`, `mp3`, `ogg`, `wav`, `webm`. Auto-detected when `audio_format` is omitted.
**Raw formats.** `s8`, `s16le`, `s16be`, `s24le`, `s24be`, `s32le`, `s32be`, `u8`, `u16le`, `u16be`, `u24le`, `u24be`, `u32le`, `u32be`, `f32le`, `f32be`, `f64le`, `f64be`, `mulaw`, `alaw`. Headerless, so `audio_format`, `sample_rate`, and `num_channels` are all required.
`sample_rate` accepts `8000`, `11025`, `16000`, `22050`, `32000`, `44100`, `48000`, `96000`.
## Behaviors
* [What are behaviors?](/velma/behaviors/what-are-behaviors)
* [Using behaviors](/velma/behaviors/using-behaviors)
* [Custom behaviors](/velma/behaviors/custom-behaviors)
* [Best practices](/velma/behaviors/best-practices)
* [Detection packages](/velma/detection-packages), ready-made behavior sets for fraud, trust and safety, compliance, and retention
## Combining with Deepfake Detection
Behavior detections say what a caller did, not whether the voice was synthetic. [Screen a call for voice fraud](/get-started/voice-fraud-screening) runs Velma Triage alongside [Deepfake Detection](/get-started/deepfake) on one recording and combines both outputs into a single decision.
## API reference
* [Velma Triage Batch](/api-reference/velma/batch)
* [Velma Triage Streaming](/api-reference/velma/streaming)
* [Behavior presets](/api-reference/velma/presets)