> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulate.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# Which API should I use?
> Pick the right Modulate model based on the kind of output you need, your latency requirements, language coverage, and audio constraints.
Modulate offers several models with overlapping capabilities. This guide helps you pick the right one based on the kind of output you need, your latency requirements, language coverage, and audio constraints.
## Start here: what kind of output do you need?
Start by deciding what you ultimately want back. Everything else — batch vs. streaming, language, enrichments — follows from this.
A classification of the audio itself — spoken language, synthetic voice, emotion, accent, music vs. speech, or AI-generated music — with **no transcript**.
A transcript, optionally enriched with speaker labels, emotion, accent, or PII/PHI handling.
Higher-level conversation understanding — fraud, churn, compliance risks, and other behaviors — not just what was said.
* **Acoustic signals only** → jump to [Acoustic signals](#acoustic-signals).
* **Transcription, maybe with acoustic signals** → jump to [Transcription](#transcription-maybe-with-signals).
* **Escalated behaviors** → jump to [Escalated behaviors](#escalated-behaviors).
## Quick reference
### Transcription
| | Multilingual Transcription (batch) | English Fast Transcription (batch) | Multilingual Fast Transcription (batch) | Multilingual Transcription (streaming) | English Fast Transcription (streaming) |
| ---------------------------------------- | ------------------------------------------ | ------------------------------------------ | ------------------------------------------ | ------------------------------------------ | ------------------------------------------- |
| **Use case** | Transcription with rich metadata | Fast English-only transcription | Fast multilingual transcription | Real-time transcription | Low-latency English real-time transcription |
| **Protocol** | HTTP POST | HTTP POST | HTTP POST | WebSocket | WebSocket |
| **Languages** | Multilingual | English only | Multilingual | Multilingual | English only |
| **Audio formats** | See [Audio formats](/guides/audio-formats) | See [Audio formats](/guides/audio-formats) | See [Audio formats](/guides/audio-formats) | See [Audio formats](/guides/audio-formats) | See [Audio formats](/guides/audio-formats) |
| **Max file size** | 100 MB | 100 MB | 100 MB | — (streaming) | — (streaming) |
| **Speaker diarization** | ✓ | — | — | ✓ | — |
| **Emotion detection** | ✓ | — | — | ✓ | — |
| **Accent detection** | ✓ | — | — | ✓ | — |
| **PII/PHI tagging** | ✓ | — | — | ✓ | — |
| **Deepfake scoring** | ✓ (per-utterance) | — | — | ✓ (per-utterance) | — |
| **Utterance-level output** | ✓ | — | — | ✓ | — |
| **Partial transcripts during streaming** | — | — | — | ✓ (opt-in `partial_results`) | ✓ (every \~1.5 s) |
### Deepfake Detection
| | Deepfake Detection (batch) | Deepfake Detection (streaming) |
| -------------------- | ---------------------------------------------------- | ------------------------------ |
| **Use case** | Deepfake detection on a file | Real-time deepfake detection |
| **Protocol** | HTTP POST | WebSocket |
| **Audio formats** | AAC, AIFF, FLAC, MOV, MP3, MP4, OGG, Opus, WAV, WebM | Raw PCM and container formats |
| **Max file size** | 100 MB | — (streaming) |
| **Deepfake scoring** | ✓ (per-frame) | ✓ (per-frame) |
### PII/PHI Redaction
| | PII/PHI Redaction (batch) | PII/PHI Redaction (streaming) |
| --------------------------- | ---------------------------------------------------- | ---------------------------------------------- |
| **Use case** | Transcription with audio redaction | Real-time transcription with audio redaction |
| **Protocol** | HTTP POST | WebSocket |
| **Languages** | Multilingual | Multilingual |
| **Audio formats** | AAC, AIFF, FLAC, MP3, MP4, MOV, OGG, Opus, WAV, WebM | AAC, AIFF, FLAC, MP3, OGG, WAV, WebM + raw PCM |
| **Max file size** | 100 MB | — (streaming) |
| **Speaker diarization** | ✓ | ✓ |
| **PII/PHI audio redaction** | ✓ | ✓ |
| **Utterance-level output** | ✓ | ✓ |
**PII/PHI tagging** (Transcription APIs) — wraps sensitive spans in tags within the transcript text; original content is preserved.
**PII/PHI audio redaction** (Redaction APIs) — replaces each detected PII/PHI span with an empty marker tag (e.g. ``, ``, ``) in the transcript text **and** silences the corresponding audio ranges in the returned MP3.
### Language Detection
| | Language Detection Batch |
| ------------------ | ------------------------------------------------------- |
| **Use case** | Identify the spoken language of an audio file |
| **Protocol** | HTTP POST |
| **Languages** | 100 spoken languages |
| **Audio formats** | WebM, WAV, Opus, OGG, MP4, MP3, MOV, FLAC, AIFF, AAC |
| **Max file size** | 100 MB |
| **Audio analyzed** | First 30 seconds only |
| **Output** | ISO 639-1 language code, display name, confidence score |
Language Detection is a pure classification endpoint — it returns no transcript, diarization, or enrichment data.
### Emotion Detection
| | Emotion Detection Batch |
| ----------------- | ---------------------------------------------------- |
| **Use case** | Classify the emotional tone of an audio file |
| **Protocol** | HTTP POST |
| **Audio formats** | WebM, WAV, Opus, OGG, MP4, MP3, MOV, FLAC, AIFF, AAC |
| **Max file size** | 100 MB |
| **Output** | Whole-file `emotion` label + per-window time series |
Emotion Detection is a pure classification endpoint — it returns a whole-file label and consecutive fixed-length windows, each with its own label, but no transcript. To get emotion labels alongside a transcript, use the `emotion_signal` enrichment on Multilingual Transcription instead.
### Accent Detection
| | Accent Detection Batch |
| ----------------- | ---------------------------------------------------- |
| **Use case** | Classify the speaker accent of an audio file |
| **Protocol** | HTTP POST |
| **Audio formats** | WebM, WAV, Opus, OGG, MP4, MP3, MOV, FLAC, AIFF, AAC |
| **Max file size** | 100 MB |
| **Output** | Whole-file `accent` label + per-window time series |
Accent Detection is a pure classification endpoint — it returns a whole-file label and consecutive fixed-length windows, each with its own label, but no transcript. To get accent labels alongside a transcript, use the `accent_signal` enrichment on Multilingual Transcription instead.
### Music & Speech Detection
| | Music & Speech Detection (batch) | Music & Speech Detection (streaming) |
| ------------ | ----------------------------------------------------- | --------------------------------------------- |
| **Use case** | Classify a complete file as music, speech, or neither | Real-time frame-by-frame classification |
| **Protocol** | HTTP POST | WebSocket |
| **Output** | Full response after processing | Frames emitted progressively as audio arrives |
| **Latency** | Proportional to file length | \~192 ms per frame |
### AI Music Detection
| | AI Music Detection (batch) | AI Music Detection (streaming) |
| ------------ | --------------------------------------------------- | ----------------------------------------------------------------------------------------- |
| **Use case** | Decide whether a file contains AI-generated music | Real-time per-window classification |
| **Protocol** | HTTP POST | WebSocket |
| **Output** | Clip-level verdict plus per-window breakdown | Per-window vocal AI results emitted progressively, final clip-level summary on completion |
| **Verdicts** | `ai-vocal-music`, `ai-instrumental`, `not-ai-music` | Same, resolved in the final `done` message |
AI Music Detection answers a different question from Music & Speech Detection: *is this music AI-generated?* rather than *is this music or speech?* See [AI Music Detection](/api-reference/ai-music-detection/overview).
### Velma Triage
| | Velma Triage (batch) | Velma Triage (streaming) |
| ------------ | ---------------------------------------------------------------------------------------------------- | ----------------------------------------------------- |
| **Use case** | Analyze a complete recording | Analyze a live or in-progress conversation |
| **Protocol** | HTTP POST | WebSocket |
| **Response** | Single JSON response | Stream of typed events |
| **Best for** | Post-call QA, compliance review, offline processing | Live monitoring, real-time alerting, in-call coaching |
| **Output** | Behavior detections, conversation type, roles, topics, sentiment, summary, and a diarized transcript | Same, delivered progressively as events |
Velma Triage is Modulate's conversation-understanding model — you describe the behaviors you want to detect and Velma surfaces them from the audio. See [What is Velma?](/velma/overview).
## Decision tree
### Acoustic signals
You want a classification of the audio, not a transcript. Pick by the signal you need. For anything real-time, choose the streaming variant; otherwise use batch.
* **Spoken language** → **Language Detection** (batch). Returns the detected language as an ISO 639-1 code and display name with a confidence score, from the first 30 seconds of audio. No transcript, diarization, or enrichment data.
* **Synthetic (AI-generated) voice** → **Deepfake Detection**. Use **streaming** if you need continuous monitoring throughout a live call; otherwise use **batch** for a submitted file.
* **Music vs. speech** → **Music & Speech Detection**. Classifies audio as music, speech, or neither. Use **streaming** for continuous updates on a live call or stream; otherwise use **batch**.
* **AI-generated music** → **AI Music Detection**. Returns a clip-level verdict (`ai-vocal-music`, `ai-instrumental`, or `not-ai-music`). Use **streaming** for real-time per-window results; otherwise use **batch**. This is distinct from Music & Speech Detection — it answers *is this music AI-generated?*
* **Emotion or accent** → **Emotion Detection** or **Accent Detection** (batch). Each returns a whole-file label plus a per-window time series, with no transcript. If you also need a transcript, use the `emotion_signal` and `accent_signal` enrichments on **Multilingual Transcription** (batch or streaming) instead — see [Transcription](#transcription-maybe-with-signals).
### Transcription (maybe with signals)
You want a transcript, optionally enriched. Choose based on your single most important requirement:
* **Multilingual support** → **Multilingual Transcription** for enriched output (use **streaming** if you need results as audio is captured; otherwise use **batch**), or **Multilingual Fast Transcription (batch)** when you just need a fast transcript of a complete file with no enrichments. These are the only options with broad language coverage.
* **Lowest latency (e.g. agentic AI)** → **English Fast Transcription (streaming)**. Emits a rolling partial transcript every \~1.5 seconds and a single final transcript at end-of-stream — no enrichments. If you need low latency across multiple languages, **Multilingual Transcription streaming** is the fallback, but it is not tuned for the lowest latency.
* **Only redacted results (compliance)** → **PII/PHI Redaction**. Use **streaming** for live audio or **batch** for complete files. Both return a redacted MP3 with sensitive audio ranges silenced, alongside a transcript where each PII/PHI span is replaced with an entity-type tag. If your team can instead receive results with PII **tagged but not redacted** — and remove or avoid persisting the sensitive content on your side — then **Multilingual Transcription** with `pii_phi_tagging=true` becomes an option as well.
* **Just accuracy, cost, reliability, and scale** → choose by language:
* **English only** → **English Fast Transcription**. Use **streaming** for real-time results; otherwise use **batch**. Fastest option for English audio, but no speaker diarization, emotion, accent, PII/PHI tagging, or utterance-level data — it returns a single `text` string and `duration_ms`, with a 60-second processing timeout per batch request.
* **Multilingual, fast batch turnaround** → **Multilingual Fast Transcription (batch)**. Like English Fast, it returns a single `text` string and `duration_ms` with no enrichments, but works across supported languages. Declare the spoken language with the optional `language` parameter for the fastest, most direct path; omit it and the language is detected automatically.
* **Multilingual, full feature set** → **Multilingual Transcription**. Use **streaming** for real-time results; otherwise use **batch**.
### Full-featured vs fast transcription — detailed tradeoffs
| Consideration | Choose Multilingual Transcription | Choose English Fast Transcription | Choose Multilingual Fast Transcription |
| --------------------------------------------- | ------------------------------------ | --------------------------------- | -------------------------------------- |
| Audio language | Non-English, or unknown/multilingual | English only | Non-English, or unknown/multilingual |
| Audio format | Any supported format | Any supported format | Any supported format |
| Need speaker IDs | Yes | No | No |
| Need emotion or accent signals | Yes | No | No |
| Need PII/PHI tagging | Yes | No | No |
| Need utterance timestamps | Yes | No | No |
| Need deepfake scoring alongside transcription | Yes | No | No |
| Processing speed is the top priority | Lower priority | Yes | Yes |
| Transcription quality is the top priority | Yes | Lower priority | Lower priority |
| Large-scale batch jobs in English | — | Preferred | — |
| Large-scale batch jobs in other languages | — | — | Preferred |
### Do you need deepfake detection alongside transcription?
The Multilingual Transcription batch API supports a `deepfake_signal` parameter that adds a per-utterance `deepfake_score` to transcription output. This is convenient when you already need transcription and want a deepfake signal without a second API call.
Use the dedicated Deepfake Detection APIs instead when:
* Transcription is not needed — you only want to know if audio is synthetic.
* You need frame-level results across the full audio (not just utterance-level scores).
* You need explicit `no-content` verdicts for silent regions.
* You need streaming deepfake verdicts in real time.
### Escalated behaviors
You want higher-level conversation understanding — fraud, churn, compliance risks, and other behaviors — not just what was said.
→ **Velma Triage**. You describe the behaviors you want to detect and Velma surfaces them from the audio, alongside a conversation summary, topics, sentiment, speaker roles, and a diarized transcript. Use **streaming** to monitor a live or in-progress conversation; otherwise use **batch** for a completed recording. See [What is Velma?](/velma/overview) and the [detection packages](/velma/detection-packages) for ready-made configurations.
## Common scenarios
**Call center QA on recorded English calls, high volume**
→ English Fast Transcription (batch). Use parallel requests with a semaphore to respect concurrent limits.
**Fast transcription of recorded calls in non-English or mixed languages**
→ Multilingual Fast Transcription (batch). Pass `language` when you know the spoken language to skip auto-detection; omit it to detect automatically.
**Meeting transcription with speaker attribution and emotion**
→ Multilingual Transcription (batch) with `speaker_diarization=true` and `emotion_signal=true`.
**Live interview transcription with real-time captions, multilingual or with speaker labels**
→ Multilingual Transcription streaming. Stream audio over WebSocket and display utterances as they arrive.
**Live English captions or voice assistant input with minimal latency**
→ English Fast Transcription streaming. Partial transcripts update every \~1.5 seconds — replace your displayed text with each new partial, then finalize on the `utterance` message.
**Detect AI-generated voice in a submitted audio clip**
→ Deepfake Detection batch endpoint.
**Real-time anti-spoofing check during a voice authentication flow**
→ Deepfake Detection streaming endpoint. Stream audio and act on frame verdicts as they arrive.
**Transcript of a support call with PII/PHI spans tagged for downstream review**
→ Multilingual Transcription (batch) with `pii_phi_tagging=true`. Language is auto-detected per utterance. The transcript content is preserved; sensitive spans are wrapped in tags.
**Compliance recording that must be shareable with PII/PHI silenced**
→ PII/PHI Redaction Batch API. Returns a redacted MP3 with sensitive audio ranges silenced and a transcript where each PII/PHI span is replaced with an entity-type tag.
**Route audio to the right transcription pipeline based on spoken language**
→ Language Detection batch endpoint. Send the clip, read `predicted_language_code` from the response, and route accordingly. Use the `confidence` field to catch low-certainty results and handle them separately.
**Gate a stream so you only process speech, not music or silence**
→ Music & Speech Detection streaming. Classify frames in real time and route only speech segments downstream.
**Screen uploaded tracks for AI-generated music**
→ AI Music Detection batch. Read the clip-level `primary_verdict`; rely on it rather than per-window results when judging a whole song.
**Flag fraud or compliance risks on recorded support calls**
→ Velma Triage (batch) with a detection package. Configure the behaviors you care about and read per-behavior verdicts, with the clips that triggered each detection.
**Monitor a live call for escalation or churn signals**
→ Velma Triage (streaming). Act on behavior-detection events as they arrive during the conversation.
## Related
* [Authentication and rate limits](/guides/authentication)
* [Audio formats](/guides/audio-formats)
* [Transcription enrichment features](/guides/stt-enrichment-features)