AI Music Detection
AI Music Detection Streaming
Real-time AI music detection over WebSocket. Per-window vocal-AI and instrumental-AI results are emitted progressively as audio arrives, followed by a final clip-level summary.
WSS
/
api
/
velma-2-ai-music-detection-streaming
Real-time AI music detection over WebSocket. The client streams audio and receives per-window vocal-AI and instrumental-AI results as they become available, followed by a final clip-level summary on completion.
The MP4-family values (
An
Endpoint
wss://platform.modulate.ai/api/velma-2-ai-music-detection-streaming
Authentication
Pass your API key as a query parameter on the connection URL:wss://.../velma-2-ai-music-detection-streaming?api_key=YOUR_API_KEY&audio_format=mp3
Connection parameters
| Parameter | Required | Description |
|---|---|---|
api_key | Yes | Your API key |
audio_format | Yes | Audio format - see supported formats below |
sample_rate | Raw PCM only | Sample rate in Hz |
num_channels | Raw PCM only | Number of channels (1-8) |
Supported audio formats
Container formats -sample_rate and num_channels are ignored if supplied (the headers already carry this metadata):
mp3, wav, flac, m4a, mp4, ogg, opus, webm, aac, aiff, wma, amr, and au, plus 66 others
All 79 accepted container values
All 79 accepted container values
3g2, 3ga, 3gp, 3gpp, 8svx, aa3, aac, ac3, act, adts, aif, aifc, aiff,
amb, amr, asf, at3, au, avr, awb, bwf, c2, caf, dss, dts, dtshd,
eac3, ec3, f4a, f4b, flac, gsm, iff, m2a, m2ts, m4a, m4b, m4r, m4v,
mka, mkv, mlp, mp+, mp1, mp2, mp3, mp4, mpa, mpc, mpga, mpp, mts,
oga, ogg, ogx, oma, omg, opus, paf, pvf, qcp, ra, rf64, rm, rmvb,
snd, svx, thd, ts, tta, voc, vqf, w64, wav, wave, weba, webm, wma,
wmvmp4, m4a, m4b, m4r, m4v, 3gp, 3gpp, 3ga, 3g2, f4a, f4b) must be sent in a streamable layout; otherwise the connection ends with an audio-processing error.
Raw PCM formats - sample_rate and num_channels are required:
s8, s16le, s16be, s24le, s24be, s32le, s32be, u8, u16le, u16be, u24le, u24be, u32le, u32be, f32le, f32be, f64le, f64be, mulaw, alaw, g722, vox
g722 and vox are mono-only: num_channels must be 1.
Valid sample rates: 8000, 11025, 16000, 22050, 32000, 44100, 48000, 96000
Protocol
Client -> server
| Message | Description |
|---|---|
| Binary frame | Chunk of audio bytes in the declared format (any size) |
Empty text frame "" | Signals end of stream |
Server -> client
| Message | Description |
|---|---|
{"type": "window", "window": {...}} | Per-window result - emitted for each completed 4-second window, in order |
{"type": "done", ...} | Stream complete - clip-level verdict plus instrumental AI detection |
{"type": "error", "error": "..."} | An error occurred - connection will close |
Vocal AI and instrumental AI are both scored independently on each 4-second window as audio arrives and reported in
window messages - a window can carry both scores, one, or neither, depending on how much of each content type it contains. The window-level instrumental score is computed from a single short window, so it can be less accurate than the clip-level score in the final done message, which is computed from the full accumulated audio.Window object
| Field | Type | Description |
|---|---|---|
start_time_ms | integer | Window start time in milliseconds |
end_time_ms | integer | Window end time in milliseconds |
vocal_percentage | float | Percentage of the window containing vocal content (0-100) |
vocal_ai_probability | float or null | Probability the window contains AI-generated vocals (0-1). null if the window doesn’t have enough vocal content to score, was too short to analyse, or no content was found to classify |
vocal_ai_confidence | float or null | Confidence in vocal_ai_probability (0-1); null under the same conditions as that field |
instrumental_percentage | float | Percentage of the window containing instrumental music content (0-100) |
instrumental_ai_probability | float or null | Probability the window contains AI-generated instrumental content (0-1), from a single-window analysis (lower accuracy than the done message’s clip-level score). null if the window is mostly silence, instrumental-AI detection is unavailable, or the window’s audio could not be analysed |
instrumental_ai_confidence | float or null | Confidence in instrumental_ai_probability (0-1); null under the same conditions as that field |
silence_percentage | float | Percentage of the window containing neither vocal nor instrumental content (0-100) |
vocal_ai_probability/vocal_ai_confidence and instrumental_ai_probability/instrumental_ai_confidence are gated and scored independently, so a window can carry both pairs, one, or neither.
Done object
| Field | Type | Description |
|---|---|---|
duration_ms | integer | Total duration of the streamed audio in milliseconds |
window_count | integer | Total number of windows analysed during the session |
primary_verdict | string | Clip-level classification: "ai-vocal-music", "ai-instrumental", or "not-ai-music" |
vocal_percentage | float | Average percentage of audio with vocal content, across all windows (0-100) |
vocal_ai_percentage | float | Percentage of the full clip duration classified as AI-generated vocals (0-100), recomputed at end-of-stream. May differ from what the live per-window messages suggested |
vocal_ai_confidence | float | Average confidence across frames classified as AI-generated vocals, from the end-of-stream recompute (0-1); 0 if none were found |
instrumental_percentage | float | Average percentage of audio with instrumental content, across all windows (0-100) |
instrumental_ai_percentage | float | AI detection score for the clip’s instrumental content (0-100), computed once at end-of-stream from the full accumulated audio - more reliable than the live per-window instrumental_ai_probability values |
instrumental_ai_confidence | float | Confidence in the instrumental AI assessment for the full clip (0-1) |
silence_percentage | float | Average percentage of audio with neither vocal nor instrumental content (0-100) |
WebSocket close codes
| Code | Meaning |
|---|---|
1000 | Normal closure after the done message, or after the connection completes |
1003 | Invalid or missing query parameters (unknown format, bad sample rate, missing audio_format) |
1011 | Internal server error, or processing ended before completion |
4001 | The api_key query parameter is missing or invalid |
4002 | Audio could not be decoded, or does not match the declared raw audio_format |
4003 | The request is not permitted |
4004 | The API key does not have access to this model |
4029 | The request could not be completed due to insufficient credits |
4030 | Concurrent request limit reached |
4031 | Monthly usage limit reached |
error message is sent before the connection closes for every case above except 1000.
Rate limits
- Concurrent connection limits apply per model
Messages
{
"type": "<string>",
"window": {
"start_time_ms": 123,
"end_time_ms": 123,
"vocal_percentage": 123,
"instrumental_percentage": 123,
"silence_percentage": 123
}
}{
"type": "<string>",
"duration_ms": 123,
"window_count": 123,
"primary_verdict": "<string>",
"vocal_percentage": 123,
"vocal_ai_percentage": 123,
"vocal_ai_confidence": 123,
"instrumental_percentage": 123,
"instrumental_ai_percentage": 123,
"instrumental_ai_confidence": 123,
"silence_percentage": 123
}{
"type": "<string>",
"error": "<string>"
}{}{}query
type:object
Window Result
type:object
A single 4-second window detection result.
Done
type:object
Analysis complete - final clip-level verdict has been computed.
Error
type:object
An error occurred during connection or analysis.
Audio Data
type:string
A binary frame containing audio samples.
End of Stream
type:string
An empty text frame signalling that the client has finished sending audio.
Messages
{
"type": "<string>",
"window": {
"start_time_ms": 123,
"end_time_ms": 123,
"vocal_percentage": 123,
"instrumental_percentage": 123,
"silence_percentage": 123
}
}{
"type": "<string>",
"duration_ms": 123,
"window_count": 123,
"primary_verdict": "<string>",
"vocal_percentage": 123,
"vocal_ai_percentage": 123,
"vocal_ai_confidence": 123,
"instrumental_percentage": 123,
"instrumental_ai_percentage": 123,
"instrumental_ai_confidence": 123,
"silence_percentage": 123
}{
"type": "<string>",
"error": "<string>"
}{}{}