Velma Triage

Velma Triage analyzes conversations rather than audio characteristics. It evaluates a configured set of behaviors against the transcript and reports which ones were detected, with the clips that triggered each detection, alongside a conversation type, per-speaker roles, topics, per-topic sentiment, and a summary. Behaviors come from a catalog of presets, from definitions written for the deployment, or from both. Velma transcribes as part of its analysis. The diarized transcript is returned as clips, so a separate transcription call on the same audio is redundant.

	Batch	Streaming
Protocol	HTTP POST	WebSocket
Input	A complete recording	A live or in-progress conversation
Output	One JSON response	Typed events as analysis develops
Use case	Post-call QA, compliance review, backlog processing	Live monitoring, real-time alerting, in-call coaching

Velma Triage (batch)

Returns application/json. duration_ms, clips, and behaviors are always present. The rest depend on configuration.

Field	Type	Contents
`duration_ms`	integer	Total audio duration.
`clips`	array	The diarized transcript, one entry per speaker turn.
`behaviors`	array	One entry per behavior evaluated, detected or not.
`conversation_type_pick`	object or null	The conversation type chosen for the session.
`participant_role_picks`	array	One role assignment per speaker.
`topics`	array of strings	Topics extracted from the conversation.
`topic_sentiments`	array	Per-speaker sentiment for each topic.
`summary`	string or null	Narrative summary. Null when `produce_summary` is false.

Each entry in clips carries clip_uuid, text, start_ms, duration_ms, speaker_label, language, and the optional emotion, accent, and deepfake_score signals, which are null unless enabled in the stt block. Each entry in behaviors carries behavior_uuid, behavior_name, speaker_label, detected, confidence, evidence_clip_uuids, definitive_clip_uuid, and reasoning. Entries appear for every behavior evaluated, so detected: false means the behavior was checked and not found. conversation_type_pick and each entry in participant_role_picks carry a confidence and a selection_source of inferred, auto_selected_single_option, or default, which distinguishes a real inference from a fallback.

Try it

curl -X POST https://platform.modulate.ai/api/velma-2-batch \
  -H "X-API-Key: $MODULATE_API_KEY" \
  -F "upload_file=@recording.mp3" \
  -F 'config={"behaviors":["preset:harassment","preset:service_churn"]}'

import os, json, requests

config = {
    "behaviors": ["preset:harassment", "preset:service_churn"],
    "stt": {"speaker_diarization": True, "emotion_signal": True},
    "produce_summary": True,
}

response = requests.post(
    "https://platform.modulate.ai/api/velma-2-batch",
    headers={"X-API-Key": os.environ["MODULATE_API_KEY"]},
    data={"config": json.dumps(config)},
    files={"upload_file": open("recording.mp3", "rb")},
)
response.raise_for_status()
result = response.json()

for b in result["behaviors"]:
    if b["detected"]:
        print(f"{b['behavior_name']} - {b['speaker_label']} ({b.get('confidence')})")
print(result.get("summary"))

Response

{
  "duration_ms": 45000,
  "clips": [
    {
      "clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "text": "Thanks for calling, how can I help?",
      "start_ms": 0,
      "duration_ms": 2800,
      "speaker_label": "Speaker 1",
      "language": "en",
      "emotion": "Calm",
      "accent": null,
      "deepfake_score": null
    }
  ],
  "conversation_type_pick": {
    "conversation_type_uuid": "c1d2e3f4-a5b6-7890-cdef-123456789012",
    "name": "Customer support call",
    "confidence": 0.93,
    "selection_source": "inferred",
    "detail": "Agent greeting followed by an account issue."
  },
  "participant_role_picks": [
    {
      "speaker_label": "Speaker 1",
      "participant_role_uuid": "d2e3f4a5-b6c7-8901-def0-234567890123",
      "name": "Support agent",
      "confidence": 0.95,
      "selection_source": "inferred",
      "detail": "Opens the call and handles the request."
    }
  ],
  "behaviors": [
    {
      "behavior_uuid": "e3f4a5b6-c7d8-9012-ef01-345678901234",
      "behavior_name": "Service churn",
      "speaker_label": "Speaker 2",
      "detected": true,
      "confidence": 0.81,
      "evidence_clip_uuids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"],
      "definitive_clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "reasoning": "The caller states they are considering cancelling."
    }
  ],
  "topics": ["billing", "account access"],
  "topic_sentiments": [
    { "topic": "billing", "speaker_label": "Speaker 2", "sentiment_score": -0.6, "sentiment_label": "negative" }
  ],
  "summary": "A customer called about a locked account and raised a billing concern."
}

What you can configure

Configuration travels in one config form field, holding either the literal string default or a JSON-encoded object.

`config` key	Default	Effect
`behaviors`	nothing evaluated	What to look for. Each entry is a `preset:<identifier>` reference or a full behavior definition.
`conversation_types`	built-in set	Candidate conversation types. Pass `[]` for no candidates.
`participant_roles`	built-in set	Candidate speaker roles. Pass `[]` for no candidates.
`stt`	—	Transcription options: `speaker_diarization`, `emotion_signal`, `accent_signal`, `deepfake_signal`, `pii_phi_tagging`, `language`, `custom_terms`. See Transcription for each.
`produce_topics`	`true`	Set `false` to skip topic extraction.
`produce_topic_sentiments`	`true`	Set `false` to skip per-topic sentiment.
`produce_summary`	`true`	Set `false` to skip the narrative summary.
`default_conversation_type`	built-in fallback	Used when no candidate applies. Set to `null` to disable the fallback.
`default_participant_role`	built-in fallback	Used when no candidate applies. Set to `null` to disable the fallback.

Passing a custom config without a behaviors key evaluates no behaviors. There is no implicit run-everything. This differs from conversation_types and participant_roles, which do fall back to built-in defaults when omitted.Omitting config entirely, or sending the literal default, loads a curated built-in configuration that does include behaviors. To evaluate against a broad set, list the preset:<identifier> references explicitly.

A config with every key set

{
  "behaviors": [
    "preset:harassment",
    "preset:service_churn",
    {
      "behavior_uuid": "8f14e45f-ceea-467a-9c2b-7f1e4a1b2c3d",
      "name": "Unverified account change",
      "short_description": "Caller requests an account change without completing verification.",
      "detailed_description": "Detected when a participant asks to change account details and the agent proceeds without confirming identity through the documented verification steps.",
      "applies_to_conversation_type_uuids": null,
      "applies_to_participant_role_uuids": null
    }
  ],
  "conversation_types": [
    {
      "conversation_type_uuid": "c1d2e3f4-a5b6-7890-cdef-123456789012",
      "name": "Customer support call",
      "short_description": "Inbound support contact.",
      "detailed_description": "A customer contacts support about an existing product or account issue."
    }
  ],
  "participant_roles": [
    {
      "participant_role_uuid": "d2e3f4a5-b6c7-8901-def0-234567890123",
      "name": "Support agent",
      "short_description": "The representative handling the call.",
      "detailed_description": "Employee who greets the caller, diagnoses the issue, and takes action.",
      "applies_to_conversation_type_uuids": null
    }
  ],
  "stt": {
    "speaker_diarization": true,
    "emotion_signal": true,
    "accent_signal": false,
    "deepfake_signal": true,
    "pii_phi_tagging": true,
    "language": "en",
    "custom_terms": [
      "Modulate",
      { "term": "Velma", "definition": "Conversation intelligence model", "pronunciations": ["VEL-muh"] }
    ]
  },
  "produce_topics": true,
  "produce_topic_sentiments": true,
  "produce_summary": true,
  "default_conversation_type": {
    "conversation_type_uuid": "f4a5b6c7-d8e9-0123-4567-89abcdef0123",
    "name": "Unclassified conversation",
    "short_description": "Fallback when no candidate applies.",
    "detailed_description": "Used when none of the candidate conversation types match the audio."
  },
  "default_participant_role": {
    "participant_role_uuid": "a5b6c7d8-e9f0-1234-5678-9abcdef01234",
    "name": "Unidentified participant",
    "short_description": "Fallback when no candidate role applies.",
    "detailed_description": "Used when none of the candidate participant roles match a speaker.",
    "applies_to_conversation_type_uuids": null
  }
}

Preset identifiers come from GET /api/velma-2-batch/list-presets, which returns each preset’s identifier, name, short_description, and detailed_description:

curl https://platform.modulate.ai/api/velma-2-batch/list-presets \
  -H "X-API-Key: $MODULATE_API_KEY"

A malformed config, an unknown preset identifier, or a definition missing a required field is rejected with 422.

Audio formats

Accepted extensions: .aac, .aiff, .flac, .mov, .mp3, .mp4, .ogg, .opus, .wav, .webm. Maximum file size is 100 MB. Empty files are rejected with 400.

Velma Triage (streaming)

The same analysis, emitted as typed JSON events while the conversation is still running. Connect, send the config as the first text frame, then stream audio.

Event	Payload
`partial_clip`	An in-progress utterance with interim `emotion`, `accent`, and `deepfake_score`. Reports `end_ms`, which grows, rather than `duration_ms`.
`clip`	The finalized utterance, reusing the `clip_uuid` of its partials.
`clip_update`	Refined `emotion` and `accent` for a clip that already finalized.
`behavior_detection`	One behavior’s verdict, with evidence clip UUIDs and reasoning.
`conversation_type`	The session’s conversation type pick.
`participant_role`	A role pick for one speaker.
`topics`	The current topic list.
`topic_sentiment`	Sentiment for one topic and speaker.
`summary`	The current summary text.
`done`	`duration_ms`. The server closes after this.
`error`	`error` string. The server closes after this.

Three events have update semantics worth building for:

partial_clip is a transient preview. Multiple partials share one clip_uuid, and the eventual clip reuses it, so a run of partials correlates with its final clip.
clip_update can arrive any number of times for a finalized clip, always before done. The latest value for each field supersedes the value on the clip event and on any earlier update.
topics and summary fully replace the previous event of that type. Never merge them. topic_sentiment supersedes an earlier event for the same topic and speaker.

Try it

Python

import os, asyncio, json, websockets

API_KEY = os.environ["MODULATE_API_KEY"]
AUDIO_FILE = "recording.mp3"
CHUNK_SIZE = 4096

config = {"behaviors": ["preset:harassment", "preset:service_churn"]}

async def analyze():
    url = f"wss://platform.modulate.ai/api/velma-2-streaming?api_key={API_KEY}"
    clips = {}

    async with websockets.connect(url) as ws:
        # The config must be the first text frame, before any audio.
        await ws.send(json.dumps(config))

        async def send():
            with open(AUDIO_FILE, "rb") as f:
                while chunk := f.read(CHUNK_SIZE):
                    await ws.send(chunk)
            await ws.send("")  # end of stream

        async def receive():
            async for message in ws:
                event = json.loads(message)
                kind = event["type"]
                if kind == "partial_clip":
                    print(f"  ... {event['partial_clip']['text']}", end="\r")
                elif kind == "clip":
                    clip = event["clip"]
                    clips[clip["clip_uuid"]] = clip
                    print(f"[{clip['start_ms']}ms] {clip['speaker_label']}: {clip['text']}")
                elif kind == "clip_update":
                    update = event["clip_update"]
                    clips.get(update["clip_uuid"], {}).update(
                        {k: v for k, v in update.items() if k != "clip_uuid"}
                    )
                elif kind == "behavior_detection":
                    d = event["detection"]
                    if d["detected"]:
                        print(f"{d['behavior_name']} - {d['speaker_label']}")
                elif kind == "summary":
                    print(f"Summary: {event['text']}")
                elif kind == "done":
                    print(f"Done: {event['duration_ms']}ms")
                    break
                elif kind == "error":
                    raise RuntimeError(event["error"])

        await asyncio.gather(send(), receive())

asyncio.run(analyze())

Events received

{ "type": "partial_clip", "partial_clip": { "clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "text": "Thanks for calling, how", "start_ms": 0, "end_ms": 1400, "speaker_label": null, "emotion": null, "accent": null, "deepfake_score": null } }
{ "type": "clip", "clip": { "clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "text": "Thanks for calling, how can I help?", "start_ms": 0, "duration_ms": 2800, "speaker_label": "Speaker 1", "language": "en", "emotion": null, "accent": null, "deepfake_score": null } }
{ "type": "clip_update", "clip_update": { "clip_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "emotion": "Calm", "accent": "American" } }
{ "type": "behavior_detection", "detection": { "behavior_uuid": "e3f4a5b6-c7d8-9012-ef01-345678901234", "behavior_name": "Service churn", "speaker_label": "Speaker 2", "detected": true, "confidence": 0.81, "evidence_clip_uuids": ["a1b2c3d4-e5f6-7890-abcd-ef1234567890"], "reasoning": "The caller states they are considering cancelling." } }
{ "type": "topics", "topics": ["billing", "account access"] }
{ "type": "done", "duration_ms": 45000 }

WebSocket endpoints cannot be exercised with cURL. For command-line testing use websocat.

What you can configure

The config object is the same one used by batch, sent as the first text frame instead of a form field. Because it rides on that frame, language and custom_terms in the stt block travel with it and need no separate transport.

Query parameter	Default	Effect
`api_key`	required	The API key. WebSocket connections authenticate through the query string, not a header.
`audio_format`	auto-detect	Optional for self-describing containers. Required for raw formats.
`sample_rate`	required for raw	Sample rate in Hz. Cannot be sent without `audio_format`.
`num_channels`	required for raw	1 to 8. Cannot be sent without `audio_format`.

The config frame must arrive before any audio. Sending audio first closes the connection with 1003.This endpoint reports authentication failure as close code 4003. It does not use 4001.

Audio formats

Self-describing formats. aac, aiff, flac, mp3, ogg, wav, webm. Auto-detected when audio_format is omitted. Raw formats. s8, s16le, s16be, s24le, s24be, s32le, s32be, u8, u16le, u16be, u24le, u24be, u32le, u32be, f32le, f32be, f64le, f64be, mulaw, alaw. Headerless, so audio_format, sample_rate, and num_channels are all required. sample_rate accepts 8000, 11025, 16000, 22050, 32000, 44100, 48000, 96000.

Behaviors

What are behaviors?
Using behaviors
Custom behaviors
Best practices
Detection packages, ready-made behavior sets for fraud, trust and safety, compliance, and retention

Combining with Deepfake Detection

Behavior detections say what a caller did, not whether the voice was synthetic. Screen a call for voice fraud runs Velma Triage alongside Deepfake Detection on one recording and combines both outputs into a single decision.

Get started

By capability

Build end to end

Guides

Velma Triage (batch)

Try it

What you can configure

Audio formats

Velma Triage (streaming)

Try it

What you can configure

Audio formats

Behaviors

Combining with Deepfake Detection

API reference

​Velma Triage (batch)

​Try it

​What you can configure

​Audio formats

​Velma Triage (streaming)

​Try it

​What you can configure

​Audio formats

​Behaviors

​Combining with Deepfake Detection

​API reference

Velma Triage (batch)

Try it

What you can configure

Audio formats

Velma Triage (streaming)

Try it

What you can configure

Audio formats

Behaviors

Combining with Deepfake Detection

API reference