> ## Documentation Index > Fetch the complete documentation index at: https://docs.modulate.ai/llms.txt > Use this file to discover all available pages before exploring further. # AI Music Detection > Detect AI-generated music, as a clip-level verdict with per-window vocal and instrumental breakdowns. Batch and streaming. AI Music Detection determines whether music was generated by a model. Vocals and instrumentals are scored separately, because a track can pair an AI voice with a human backing track or the reverse, and a single number would hide that. This is not [Music & Speech Detection](/get-started/music-detection), which separates music from speech. That model answers what kind of sound is present. This one answers where the music came from. | | Batch | Streaming | | - | - | - | | Protocol | HTTP POST | WebSocket | | Input | A complete file | Live audio frames | | Output | Clip verdict plus every window, at once | Windows as they are scored, then the clip verdict | | Use case | Screening uploads, auditing a catalog | Monitoring a live broadcast or user stream | ## Reading the fields Two families of field look alike and mean different things. | Suffix | Meaning | | - | - | | `_percentage` | How much of the audio **contains** that kind of content. | | `_ai_percentage` | How much of it reads as **AI-generated**. | A track can be 90% vocal and 5% AI-vocal. The first is about content, the second about origin. Vocal and instrumental are scored independently per window rather than routed to one or the other. A window with enough vocal content receives `vocal_ai_probability` and `vocal_ai_confidence`. A window that is not mostly silence receives `instrumental_ai_probability` and `instrumental_ai_confidence`. Either pair is `null` when the window lacks enough of that content to score, so a window can carry both pairs, one, or neither. Guard for `null` on `vocal_ai_probability`, `vocal_ai_confidence`, `instrumental_ai_probability`, and `instrumental_ai_confidence` before formatting them. Any per-window renderer that assumes a number will print garbage on a silent or instrumental-only window. ## AI Music Detection (batch) Returns `application/json`. Every field is always present, though the four per-window AI fields can be `null`. | Field | Type | Contents | | - | - | - | | `filename` | string | The submitted filename, or an empty string if none was sent. | | `duration_s` | number | Total duration in **seconds**, not milliseconds. | | `primary_verdict` | string | `ai-vocal-music`, `ai-instrumental`, or `not-ai-music`. | | `vocal_percentage` | number | Clip-level average share of vocal content, 0 to 100. | | `vocal_ai_percentage` | number | Clip-level AI-vocal score, 0 to 100. Each scored window's probability-weighted duration as a share of the full clip. Unscored windows contribute zero but still count toward the total. | | `vocal_ai_confidence` | number | Average confidence across windows scored for AI vocals. Not diluted by unscored windows. | | `instrumental_percentage` | number | Clip-level average share of instrumental content, 0 to 100. | | `instrumental_ai_percentage` | number | Clip-level AI-instrumental score, 0 to 100, duration-weighted. Mostly-silent windows do not contribute. | | `instrumental_ai_confidence` | number | Average confidence across windows scored for AI instrumentals. Zero if none were scored. | | `silence_percentage` | number | Clip-level average share containing neither vocal nor instrumental content. | | `latency_ms` | number | Server-side inference time. | | `windows` | array | Per-window breakdown, each covering 4 seconds. | | `primary_verdict` | Meaning | | - | - | | `ai-vocal-music` | AI-generated music with a detected synthetic voice. Covers AI songs and AI vocal tracks. | | `ai-instrumental` | AI-generated instrumental music with no detectable synthetic voice. | | `not-ai-music` | The clip does not appear to contain AI-generated music. | Each entry in `windows` carries `start_time_ms`, `end_time_ms`, `vocal_percentage`, `vocal_ai_probability`, `vocal_ai_confidence`, `instrumental_percentage`, `instrumental_ai_probability`, `instrumental_ai_confidence`, and `silence_percentage`. ### Try it ```bash curl + jq theme={null} curl -sS -X POST https://platform.modulate.ai/api/velma-2-ai-music-detection-batch \ -H "X-API-Key: $MODULATE_API_KEY" \ -F "upload_file=@audio.mp3" \ | jq -r 'if .detail then "error: \(.detail)" else "Verdict: \(.primary_verdict)", "Vocal AI: \(.vocal_ai_percentage)% (\(.vocal_ai_confidence * 100 | round)% confidence)", "Instrumental AI: \(.instrumental_ai_percentage)% (\(.instrumental_ai_confidence * 100 | round)% confidence)", (.windows[] | " \(.start_time_ms)-\(.end_time_ms)ms" + " vocal_ai=\(.vocal_ai_probability // "n/a")" + " instr_ai=\(.instrumental_ai_probability // "n/a")") end' ``` ```python Python theme={null} import os, requests response = requests.post( "https://platform.modulate.ai/api/velma-2-ai-music-detection-batch", headers={"X-API-Key": os.environ["MODULATE_API_KEY"]}, files={"upload_file": open("audio.mp3", "rb")}, ) response.raise_for_status() result = response.json() def prob(value): return "n/a" if value is None else value print(f"Verdict: {result['primary_verdict']}") print(f"Vocal AI: {result['vocal_ai_percentage']}% " f"({round(result['vocal_ai_confidence'] * 100)}% confidence)") print(f"Instrumental AI: {result['instrumental_ai_percentage']}% " f"({round(result['instrumental_ai_confidence'] * 100)}% confidence)") for w in result["windows"]: print(f" {w['start_time_ms']}-{w['end_time_ms']}ms " f"vocal_ai={prob(w['vocal_ai_probability'])} " f"instr_ai={prob(w['instrumental_ai_probability'])}") ``` ```javascript JavaScript theme={null} import fs from "fs"; import FormData from "form-data"; const form = new FormData(); form.append("upload_file", fs.createReadStream("audio.mp3"), { filename: "audio.mp3" }); const response = await fetch( "https://platform.modulate.ai/api/velma-2-ai-music-detection-batch", { method: "POST", headers: { "X-API-Key": process.env.MODULATE_API_KEY, ...form.getHeaders() }, body: form, } ); const result = await response.json(); if (!response.ok) { console.error(`error: ${result.detail}`); process.exit(1); } const prob = (v) => (v == null ? "n/a" : v); console.log(`Verdict: ${result.primary_verdict}`); console.log( `Vocal AI: ${result.vocal_ai_percentage}% ` + `(${Math.round(result.vocal_ai_confidence * 100)}% confidence)` ); console.log( `Instrumental AI: ${result.instrumental_ai_percentage}% ` + `(${Math.round(result.instrumental_ai_confidence * 100)}% confidence)` ); for (const w of result.windows) { console.log( ` ${w.start_time_ms}-${w.end_time_ms}ms ` + `vocal_ai=${prob(w.vocal_ai_probability)} instr_ai=${prob(w.instrumental_ai_probability)}` ); } ``` ```text Output theme={null} Verdict: ai-vocal-music Vocal AI: 56.5% (96% confidence) Instrumental AI: 10.5% (95% confidence) 0-4000ms vocal_ai=0.97 instr_ai=0.42 4000-8000ms vocal_ai=n/a instr_ai=0.88 ``` The second window scored no vocal AI probability, so it prints `n/a` rather than a number. Guard for `null` before formatting either probability. ```json theme={null} { "filename": "my_audio.mp3", "duration_s": 89.28, "primary_verdict": "ai-vocal-music", "vocal_percentage": 87.5, "vocal_ai_percentage": 56.5, "vocal_ai_confidence": 0.96, "instrumental_percentage": 64.3, "instrumental_ai_percentage": 10.5, "instrumental_ai_confidence": 0.95, "silence_percentage": 3.51, "latency_ms": 1333.0, "windows": [ { "start_time_ms": 0, "end_time_ms": 4000, "vocal_percentage": 100.0, "vocal_ai_probability": 0.97, "vocal_ai_confidence": 0.97, "instrumental_percentage": 79.0, "instrumental_ai_probability": 0.42, "instrumental_ai_confidence": 0.16, "silence_percentage": 0.0 }, { "start_time_ms": 4000, "end_time_ms": 8000, "vocal_percentage": 0.0, "vocal_ai_probability": null, "vocal_ai_confidence": null, "instrumental_percentage": 82.0, "instrumental_ai_probability": 0.88, "instrumental_ai_confidence": 0.76, "silence_percentage": 18.0 } ] } ``` ### What you can configure | Form field | Default | Effect | | - | - | - | | `upload_file` | *required* | The audio to analyze. This endpoint takes no other parameters. | ### Audio formats Accepted extensions include `.mp3`, `.wav`, `.flac`, `.m4a`, `.mp4`, `.ogg`, `.opus`, `.webm`, `.aac`, `.aiff`, and `.mov`, plus 85 others. `.3g2`, `.3ga`, `.3gp`, `.3gpp`, `.8svx`, `.aa3`, `.aac`, `.ac3`, `.act`, `.adts`, `.aif`, `.aifc`, `.aiff`, `.alac`, `.amb`, `.amr`, `.ape`, `.asf`, `.at3`, `.au`, `.avi`, `.avr`, `.awb`, `.bwf`, `.c2`, `.caf`, `.dss`, `.dts`, `.dtshd`, `.eac3`, `.ec3`, `.f4a`, `.f4b`, `.flac`, `.flv`, `.gsm`, `.iff`, `.m2a`, `.m2ts`, `.m4a`, `.m4b`, `.m4r`, `.m4v`, `.mka`, `.mkv`, `.mlp`, `.mmf`, `.mov`, `.mp+`, `.mp1`, `.mp2`, `.mp3`, `.mp4`, `.mpa`, `.mpc`, `.mpeg`, `.mpg`, `.mpga`, `.mpp`, `.mts`, `.mxf`, `.nist`, `.oga`, `.ogg`, `.ogx`, `.oma`, `.omg`, `.opus`, `.paf`, `.pvf`, `.qcp`, `.ra`, `.rf64`, `.rka`, `.rm`, `.rmvb`, `.sf`, `.shn`, `.snd`, `.sph`, `.svx`, `.tak`, `.thd`, `.ts`, `.tta`, `.vob`, `.voc`, `.vqf`, `.w64`, `.wav`, `.wave`, `.weba`, `.webm`, `.wma`, `.wmv`, `.wv` Maximum file size is 100 MB. Files above it are rejected with `413`. Empty files are rejected with `400`. ### Accuracy Use `primary_verdict` when judging a whole song or segment. Per-window results are lower accuracy by construction, each scored from four seconds of context. Heavily processed or high-production tracks are sometimes labelled AI-generated. This is a known gap targeted by future model updates. ## AI Music Detection (streaming) One `window` message per completed 4-second window, then a `done` message with the clip verdict and percentages. | Message | Payload | | - | - | | `window` | `window` object, same shape as a batch window entry. | | `done` | `duration_ms`, `window_count`, `primary_verdict`, and the six clip-level percentages and confidences. The server closes after this. | | `error` | `error` string. The server closes after this. | The `done` message is not a summary of the windows already sent. Its instrumental score is recomputed at end of stream from the full accumulated audio, which is more context than any single window receives, so it can differ from what the live windows suggested and is the more reliable number. Send audio as binary WebSocket frames of any size. Send an empty text frame (`""`) to end the stream. The server then flushes remaining audio, emits outstanding windows, runs the final instrumental analysis, sends `done`, and closes. ### Try it ```python Python theme={null} import os, asyncio, json, websockets API_KEY = os.environ["MODULATE_API_KEY"] AUDIO_FILE = "audio.mp3" CHUNK_SIZE = 65536 def prob(value): return "n/a" if value is None else value async def stream(): url = ( f"wss://platform.modulate.ai/api/velma-2-ai-music-detection-streaming" f"?api_key={API_KEY}&audio_format=mp3" ) async with websockets.connect(url) as ws: async def send(): with open(AUDIO_FILE, "rb") as f: while chunk := f.read(CHUNK_SIZE): await ws.send(chunk) await ws.send("") # end of stream async def receive(): async for message in ws: msg = json.loads(message) if msg["type"] == "window": w = msg["window"] print(f" {w['start_time_ms']}-{w['end_time_ms']}ms " f"vocal_ai={prob(w['vocal_ai_probability'])} " f"instr_ai={prob(w['instrumental_ai_probability'])}") elif msg["type"] == "done": print(f"done: {msg['duration_ms']}ms of audio, {msg['window_count']} windows") print(f"Verdict: {msg['primary_verdict']} " f"vocal_ai={msg['vocal_ai_percentage']}% " f"instr_ai={msg['instrumental_ai_percentage']}%") break elif msg["type"] == "error": raise RuntimeError(msg["error"]) await asyncio.gather(send(), receive()) asyncio.run(stream()) ``` ```javascript JavaScript theme={null} import { WebSocket } from "ws"; import { createReadStream } from "fs"; const url = `wss://platform.modulate.ai/api/velma-2-ai-music-detection-streaming` + `?api_key=${process.env.MODULATE_API_KEY}&audio_format=mp3`; const ws = new WebSocket(url); const prob = (v) => (v == null ? "n/a" : v); ws.on("open", () => { const stream = createReadStream("audio.mp3", { highWaterMark: 65536 }); stream.on("data", (chunk) => ws.send(chunk)); stream.on("end", () => ws.send("")); }); ws.on("message", (data) => { const msg = JSON.parse(data); if (msg.type === "window") { const w = msg.window; console.log( ` ${w.start_time_ms}-${w.end_time_ms}ms ` + `vocal_ai=${prob(w.vocal_ai_probability)} instr_ai=${prob(w.instrumental_ai_probability)}` ); } else if (msg.type === "done") { console.log(`done: ${msg.duration_ms}ms of audio, ${msg.window_count} windows`); console.log( `Verdict: ${msg.primary_verdict} ` + `vocal_ai=${msg.vocal_ai_percentage}% instr_ai=${msg.instrumental_ai_percentage}%` ); ws.close(); } else if (msg.type === "error") { console.error(`error: ${msg.error}`); } }); // A rejected handshake closes without sending an error frame first. ws.on("close", (code) => { if (code !== 1000) console.error(`error: connection closed ${code}`); }); ws.on("error", (err) => console.error(`error: ${err.message}`)); ``` ```text Output theme={null} 0-4000ms vocal_ai=0.97 instr_ai=0.42 4000-8000ms vocal_ai=n/a instr_ai=0.88 done: 89280ms of audio, 22 windows Verdict: ai-vocal-music vocal_ai=56.5% instr_ai=41.8% ``` The instrumental figure on `done` is recomputed from the full audio, so it is not the average of the window values printed above it. ```json theme={null} { "type": "window", "window": { "start_time_ms": 0, "end_time_ms": 4000, "vocal_percentage": 87.5, "vocal_ai_probability": 0.97, "vocal_ai_confidence": 0.97, "instrumental_percentage": 64.3, "instrumental_ai_probability": 0.42, "instrumental_ai_confidence": 0.16, "silence_percentage": 3.5 } } { "type": "window", "window": { "start_time_ms": 4000, "end_time_ms": 8000, "vocal_percentage": 0.0, "vocal_ai_probability": null, "vocal_ai_confidence": null, "instrumental_percentage": 82.0, "instrumental_ai_probability": 0.88, "instrumental_ai_confidence": 0.76, "silence_percentage": 18.0 } } { "type": "done", "duration_ms": 89280, "window_count": 22, "primary_verdict": "ai-vocal-music", "vocal_percentage": 87.5, "vocal_ai_percentage": 56.5, "vocal_ai_confidence": 0.96, "instrumental_percentage": 64.3, "instrumental_ai_percentage": 41.8, "instrumental_ai_confidence": 0.95, "silence_percentage": 3.51 } ``` A stream ends with an empty text frame. A shell pipe sends binary frames only, so `curl` and `websocat` cannot close a stream cleanly. Use one of the clients above. ### What you can configure | Query parameter | Default | Effect | | - | - | - | | `api_key` | *required* | The API key. WebSocket connections authenticate through the query string, not a header. | | `audio_format` | *required* | How the server decodes the bytes sent. Required on every connection, including container formats. | | `sample_rate` | *required for raw* | Sample rate in Hz. Required for raw formats. Must not be sent for container formats. | | `num_channels` | *required for raw* | 1 to 8. Required for raw formats. Must not be sent for container formats. | ### Audio formats `audio_format` is required on every connection, including self-describing containers. Omitting it closes the connection with code `1003`. **Container formats.** Accepted values include `mp3`, `wav`, `flac`, `m4a`, `mp4`, `ogg`, `opus`, `webm`, `aac`, `aiff`, `wma`, `amr`, and `au`, plus 66 others. The stream header carries sample rate and channel count, so `sample_rate` and `num_channels` are ignored if supplied. `3g2`, `3ga`, `3gp`, `3gpp`, `8svx`, `aa3`, `aac`, `ac3`, `act`, `adts`, `aif`, `aifc`, `aiff`, `amb`, `amr`, `asf`, `at3`, `au`, `avr`, `awb`, `bwf`, `c2`, `caf`, `dss`, `dts`, `dtshd`, `eac3`, `ec3`, `f4a`, `f4b`, `flac`, `gsm`, `iff`, `m2a`, `m2ts`, `m4a`, `m4b`, `m4r`, `m4v`, `mka`, `mkv`, `mlp`, `mp+`, `mp1`, `mp2`, `mp3`, `mp4`, `mpa`, `mpc`, `mpga`, `mpp`, `mts`, `oga`, `ogg`, `ogx`, `oma`, `omg`, `opus`, `paf`, `pvf`, `qcp`, `ra`, `rf64`, `rm`, `rmvb`, `snd`, `svx`, `thd`, `ts`, `tta`, `voc`, `vqf`, `w64`, `wav`, `wave`, `weba`, `webm`, `wma`, `wmv` The MP4-family values (`mp4`, `m4a`, `m4b`, `m4r`, `m4v`, `3gp`, `3gpp`, `3ga`, `3g2`, `f4a`, `f4b`) must be sent in a streamable layout. Anything else ends the connection with an audio-processing error. **Raw formats.** `s8`, `s16le`, `s16be`, `s24le`, `s24be`, `s32le`, `s32be`, `u8`, `u16le`, `u16be`, `u24le`, `u24be`, `u32le`, `u32be`, `f32le`, `f32be`, `f64le`, `f64be`, `mulaw`, `alaw`, `g722`, `vox`. These are headerless, so `sample_rate` and `num_channels` are both required. `g722` and `vox` are mono-only: `num_channels` must be `1`. `sample_rate` accepts `8000`, `11025`, `16000`, `22050`, `32000`, `44100`, `48000`, `96000`. To convert a file to raw PCM: ```bash theme={null} ffmpeg -i audio.mp3 -ar 16000 -ac 1 -f s16le audio.raw ``` ## API reference * [AI Music Detection Batch](/api-reference/ai-music-detection/batch) * [AI Music Detection Streaming](/api-reference/ai-music-detection/streaming) This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.