> ## Documentation Index > Fetch the complete documentation index at: https://docs.modulate.ai/llms.txt > Use this file to discover all available pages before exploring further. # Deepfake Detection > Detect synthetic (AI-generated) voice in recorded files or live audio, as per-frame verdicts with confidence scores. Deepfake Detection classifies segments of single-speaker audio as naturally produced human speech or synthetic speech. Synthetic speech covers text-to-speech systems, voice cloning, and other AI voice generation. Classification is acoustic: the model analyzes how the audio sounds, not the words spoken. Audio is scored in windows rather than as a whole, so the response locates synthetic speech in time instead of returning one verdict per file. Synthetic speech is not itself evidence of harm or deception. The model cannot distinguish someone using an accessibility tool to communicate from someone using voice cloning to defraud. Interpret results against the use case. | | Batch | Streaming | | - | - | - | | Protocol | HTTP POST | WebSocket | | Input | A complete file | Live audio frames | | Output | All frames at once, after processing | Frames as they are analyzed, then a summary | | Use case | Screening recordings, investigating a reported call | Anti-spoofing during a live voice-authentication flow | ## Deepfake Detection (batch) Returns `application/json`. Every field is always present. | Field | Type | Contents | | - | - | - | | `filename` | string or null | The uploaded filename. | | `duration_ms` | integer | Total audio duration. | | `frames` | array | Per-frame results, ordered, covering the full duration. | | `frames[].start_time_ms` | integer | Frame start, relative to the original file. | | `frames[].end_time_ms` | integer | Frame end, relative to the original file. | | `frames[].verdict` | string | `synthetic`, `non-synthetic`, or `no-content`. | | `frames[].confidence` | number | 0 to 1, confidence in the stated verdict. | | Verdict | Meaning | | - | - | | `synthetic` | The frame likely contains AI-generated speech. | | `non-synthetic` | The frame likely contains natural human speech. | | `no-content` | The frame is silent or has no usable audio. Always returned with confidence `1.0`. | `confidence` is confidence in the verdict given, not the probability that the audio is synthetic. A frame with `verdict: "non-synthetic"` and `confidence: 0.97` means the model is 97% confident the speech is natural. It does not mean a 3% chance of being synthetic. ### Try it ```bash curl + jq theme={null} curl -sS -X POST https://platform.modulate.ai/api/velma-2-synthetic-voice-detection-batch \ -H "X-API-Key: $MODULATE_API_KEY" \ -F "upload_file=@audio.mp3" \ | jq -r 'if .detail then "error: \(.detail)" else "\(.duration_ms)ms, \(.frames | length) frames", (.frames[] | " \(.start_time_ms)-\(.end_time_ms)ms \(.verdict) \(.confidence * 100 | round)%") end' ``` ```python Python theme={null} import os, requests response = requests.post( "https://platform.modulate.ai/api/velma-2-synthetic-voice-detection-batch", headers={"X-API-Key": os.environ["MODULATE_API_KEY"]}, files={"upload_file": open("audio.mp3", "rb")}, ) response.raise_for_status() result = response.json() print(f"{result['duration_ms']}ms, {len(result['frames'])} frames") for f in result["frames"]: print(f" {f['start_time_ms']}-{f['end_time_ms']}ms " f"{f['verdict']} {round(f['confidence'] * 100)}%") ``` ```javascript JavaScript theme={null} import fs from "fs"; import FormData from "form-data"; const form = new FormData(); form.append("upload_file", fs.createReadStream("audio.mp3"), { filename: "audio.mp3" }); const response = await fetch( "https://platform.modulate.ai/api/velma-2-synthetic-voice-detection-batch", { method: "POST", headers: { "X-API-Key": process.env.MODULATE_API_KEY, ...form.getHeaders() }, body: form, } ); const result = await response.json(); if (!response.ok) { console.error(`error: ${result.detail}`); process.exit(1); } console.log(`${result.duration_ms}ms, ${result.frames.length} frames`); for (const f of result.frames) { console.log( ` ${f.start_time_ms}-${f.end_time_ms}ms ` + `${f.verdict} ${Math.round(f.confidence * 100)}%` ); } ``` ```text Output theme={null} 16000ms, 4 frames 0-4000ms non-synthetic 94% 4000-8000ms non-synthetic 91% 8000-12000ms synthetic 87% 12000-16000ms no-content 100% ``` ```json theme={null} { "filename": "audio.mp3", "duration_ms": 16000, "frames": [ { "start_time_ms": 0, "end_time_ms": 4000, "verdict": "non-synthetic", "confidence": 0.94 }, { "start_time_ms": 4000, "end_time_ms": 8000, "verdict": "non-synthetic", "confidence": 0.91 }, { "start_time_ms": 8000, "end_time_ms": 12000, "verdict": "synthetic", "confidence": 0.87 }, { "start_time_ms": 12000, "end_time_ms": 16000, "verdict": "no-content", "confidence": 1.00 } ] } ``` ### What you can configure | Form field | Default | Effect | | - | - | - | | `upload_file` | *required* | The audio to analyze. This endpoint takes no other parameters. | ### Audio formats Accepted extensions include `.mp3`, `.wav`, `.flac`, `.m4a`, `.mp4`, `.ogg`, `.opus`, `.webm`, `.aac`, `.aiff`, and `.mov`, plus 85 others. `.3g2`, `.3ga`, `.3gp`, `.3gpp`, `.8svx`, `.aa3`, `.aac`, `.ac3`, `.act`, `.adts`, `.aif`, `.aifc`, `.aiff`, `.alac`, `.amb`, `.amr`, `.ape`, `.asf`, `.at3`, `.au`, `.avi`, `.avr`, `.awb`, `.bwf`, `.c2`, `.caf`, `.dss`, `.dts`, `.dtshd`, `.eac3`, `.ec3`, `.f4a`, `.f4b`, `.flac`, `.flv`, `.gsm`, `.iff`, `.m2a`, `.m2ts`, `.m4a`, `.m4b`, `.m4r`, `.m4v`, `.mka`, `.mkv`, `.mlp`, `.mmf`, `.mov`, `.mp+`, `.mp1`, `.mp2`, `.mp3`, `.mp4`, `.mpa`, `.mpc`, `.mpeg`, `.mpg`, `.mpga`, `.mpp`, `.mts`, `.mxf`, `.nist`, `.oga`, `.ogg`, `.ogx`, `.oma`, `.omg`, `.opus`, `.paf`, `.pvf`, `.qcp`, `.ra`, `.rf64`, `.rka`, `.rm`, `.rmvb`, `.sf`, `.shn`, `.snd`, `.sph`, `.svx`, `.tak`, `.thd`, `.ts`, `.tta`, `.vob`, `.voc`, `.vqf`, `.w64`, `.wav`, `.wave`, `.weba`, `.webm`, `.wma`, `.wmv`, `.wv` Maximum file size is 100 MB. Empty files are rejected with `400`. ### Behavior and constraints Audio under 0.5 seconds is rejected with `422`. Audio shorter than one 4-second window is accepted and padded before inference. Each batch frame covers a 4-second window, and the file is windowed from start to finish. Leading and trailing silence is trimmed before windowing. Frame timestamps still refer to positions in the original file, so `start_time_ms` and `end_time_ms` line up against the source audio with no offset arithmetic. `no-content` frames are classified before inference rather than by the model, so silence produces an explicit verdict instead of an arbitrary synthetic or non-synthetic guess. Filter them out when aggregating a clip-level verdict. Recommended clip length is 4 to 60 seconds. ## Deepfake Detection (streaming) Frames arrive as JSON text messages while audio streams in. A closing `done` message reports total duration and frame count. | Message | Payload | | - | - | | `frame` | `frame` object with `start_time_ms`, `end_time_ms`, `verdict`, `confidence`. Same shape as a batch frame. | | `done` | `duration_ms` and `frame_count`. The server closes after this. | | `error` | `error` string. The server closes after this. | Send audio as binary WebSocket frames of any size; the server buffers and windows internally. Send an empty text frame (`""`) to end the stream. The server then flushes remaining audio, delivers final frames, sends `done`, and closes. ### Try it ```python Python theme={null} import os, asyncio, json, websockets API_KEY = os.environ["MODULATE_API_KEY"] AUDIO_FILE = "audio.raw" CHUNK_SIZE = 8192 async def stream(): url = ( f"wss://platform.modulate.ai/api/velma-2-synthetic-voice-detection-streaming" f"?api_key={API_KEY}&audio_format=s16le&sample_rate=16000&num_channels=1" ) async with websockets.connect(url) as ws: async def send(): with open(AUDIO_FILE, "rb") as f: while chunk := f.read(CHUNK_SIZE): await ws.send(chunk) await ws.send("") # end of stream async def receive(): async for message in ws: msg = json.loads(message) if msg["type"] == "frame": f = msg["frame"] print(f" {f['start_time_ms']}-{f['end_time_ms']}ms " f"{f['verdict']} {round(f['confidence'] * 100)}%") elif msg["type"] == "done": print(f"done: {msg['duration_ms']}ms of audio, {msg['frame_count']} frames") break elif msg["type"] == "error": raise RuntimeError(msg["error"]) await asyncio.gather(send(), receive()) asyncio.run(stream()) ``` ```javascript JavaScript theme={null} import { WebSocket } from "ws"; import { createReadStream } from "fs"; const url = `wss://platform.modulate.ai/api/velma-2-synthetic-voice-detection-streaming` + `?api_key=${process.env.MODULATE_API_KEY}` + `&audio_format=s16le&sample_rate=16000&num_channels=1`; const ws = new WebSocket(url); ws.on("open", () => { const stream = createReadStream("audio.raw", { highWaterMark: 8192 }); stream.on("data", (chunk) => ws.send(chunk)); stream.on("end", () => ws.send("")); }); ws.on("message", (data) => { const msg = JSON.parse(data); if (msg.type === "frame") { const f = msg.frame; console.log( ` ${f.start_time_ms}-${f.end_time_ms}ms ` + `${f.verdict} ${Math.round(f.confidence * 100)}%` ); } else if (msg.type === "done") { console.log(`done: ${msg.duration_ms}ms of audio, ${msg.frame_count} frames`); ws.close(); } else if (msg.type === "error") { console.error(`error: ${msg.error}`); } }); // A rejected handshake closes without sending an error frame first. ws.on("close", (code) => { if (code !== 1000) console.error(`error: connection closed ${code}`); }); ws.on("error", (err) => console.error(`error: ${err.message}`)); ``` ```text Output theme={null} 0-4000ms non-synthetic 94% 4000-8000ms synthetic 91% done: 8000ms of audio, 2 frames ``` ```json theme={null} { "type": "frame", "frame": { "start_time_ms": 0, "end_time_ms": 4000, "verdict": "non-synthetic", "confidence": 0.94 } } { "type": "frame", "frame": { "start_time_ms": 4000, "end_time_ms": 8000, "verdict": "synthetic", "confidence": 0.91 } } { "type": "done", "duration_ms": 8000, "frame_count": 2 } ``` A stream ends with an empty text frame. A shell pipe sends binary frames only, so `curl` and `websocat` cannot close a stream cleanly. Use one of the clients above. ### What you can configure | Query parameter | Default | Effect | | - | - | - | | `api_key` | *required* | The API key. WebSocket connections authenticate through the query string, not a header. | | `audio_format` | *required* | How the server decodes the bytes sent. Required on every connection, including container formats. | | `sample_rate` | *required for raw* | Sample rate in Hz. Required for raw formats. Must not be sent for container formats. | | `num_channels` | *required for raw* | 1 to 8. Required for raw formats. Must not be sent for container formats. Multi-channel audio is downmixed to mono before analysis. | ### Audio formats `audio_format` is required on every connection, including self-describing containers. Omitting it closes the connection with code `1003`. This differs from Transcription and Redaction streaming, which auto-detect containers. **Container formats.** Accepted values include `mp3`, `wav`, `flac`, `m4a`, `mp4`, `ogg`, `opus`, `webm`, `aac`, `aiff`, `wma`, `amr`, and `au`, plus 66 others. The stream header carries sample rate and channel count, so `sample_rate` and `num_channels` are ignored if supplied. `3g2`, `3ga`, `3gp`, `3gpp`, `8svx`, `aa3`, `aac`, `ac3`, `act`, `adts`, `aif`, `aifc`, `aiff`, `amb`, `amr`, `asf`, `at3`, `au`, `avr`, `awb`, `bwf`, `c2`, `caf`, `dss`, `dts`, `dtshd`, `eac3`, `ec3`, `f4a`, `f4b`, `flac`, `gsm`, `iff`, `m2a`, `m2ts`, `m4a`, `m4b`, `m4r`, `m4v`, `mka`, `mkv`, `mlp`, `mp+`, `mp1`, `mp2`, `mp3`, `mp4`, `mpa`, `mpc`, `mpga`, `mpp`, `mts`, `oga`, `ogg`, `ogx`, `oma`, `omg`, `opus`, `paf`, `pvf`, `qcp`, `ra`, `rf64`, `rm`, `rmvb`, `snd`, `svx`, `thd`, `ts`, `tta`, `voc`, `vqf`, `w64`, `wav`, `wave`, `weba`, `webm`, `wma`, `wmv` The MP4-family values (`mp4`, `m4a`, `m4b`, `m4r`, `m4v`, `3gp`, `3gpp`, `3ga`, `3g2`, `f4a`, `f4b`) must be sent in a streamable layout. Anything else ends the connection with an audio-processing error. **Raw formats.** `s8`, `s16le`, `s16be`, `s24le`, `s24be`, `s32le`, `s32be`, `u8`, `u16le`, `u16be`, `u24le`, `u24be`, `u32le`, `u32be`, `f32le`, `f32be`, `f64le`, `f64be`, `mulaw`, `alaw`, `g722`, `vox`. These are headerless, so `sample_rate` and `num_channels` are both required. `g722` and `vox` are mono-only: `num_channels` must be `1`. `sample_rate` accepts `8000`, `11025`, `16000`, `22050`, `32000`, `44100`, `48000`, `96000`. Common raw configurations: | Source | `audio_format` | `sample_rate` | `num_channels` | | - | - | - | - | | Native app default | `s16le` | `16000` | `1` | | Web Audio API (`AudioWorklet`) | `f32le` | `48000` | `1` | | Native stereo capture | `s16le` | `48000` | `2` | | Telephony (mu-law) | `mulaw` | `8000` | `1` | | Telephony (A-law) | `alaw` | `8000` | `1` | `s16le` at 16 kHz mono is passed through without conversion. Every other format is decoded and resampled to 16 kHz mono first. Output is identical either way, so this only affects latency. Capture at `s16le` 16 kHz mono where the pipeline allows it. To convert a file: ```bash theme={null} ffmpeg -i audio.mp3 -ar 16000 -ac 1 -f s16le audio.raw ``` ### Behavior and constraints The first prediction is emitted once the minimum audio duration has arrived. Each subsequent prediction follows one second later, with the window growing from time zero. Once the window reaches full length it slides forward, holding a constant size. Verdicts therefore arrive incrementally rather than at end of stream. Format parameters are validated twice: at connection time, which closes with `1003`, and again while decoding audio, which closes with `4002`. The second catches both undecodable audio and a mismatch between the declared `audio_format` and the bytes actually sent. ## Frame-level detection compared with the transcription signal Multilingual Transcription accepts `deepfake_signal=true`, which adds a `deepfake_score` to each utterance. That is one score per utterance, available only when transcribing, and `null` for utterances under 0.5 seconds. The endpoints on this page score every frame across the file, return explicit `no-content` verdicts for silence, and work on live audio without producing a transcript. Use `deepfake_signal` when detection is a secondary signal alongside a transcript. Use these endpoints when detection is the goal. ## Screening with behavior analysis A synthetic verdict does not say whether the caller did anything wrong, and this page's endpoints cannot tell an AI assistant calling on a customer's behalf from a cloned voice running a fraud script. [Screen a call for voice fraud](/get-started/voice-fraud-screening) pairs these endpoints with [Velma Triage](/get-started/velma), aggregates the frame verdicts into one clip-level value, and combines both outputs into a single decision. ## API reference * [Deepfake Detection Batch](/api-reference/svd/batch) * [Deepfake Detection Streaming](/api-reference/svd/streaming) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.