AI Music Detection Streaming

Real-time AI music detection over WebSocket. The client streams audio and receives per-window vocal-AI and instrumental-AI results as they become available, followed by a final clip-level summary on completion.

Endpoint

wss://platform.modulate.ai/api/velma-2-ai-music-detection-streaming

Authentication

Pass your API key as a query parameter on the connection URL:

wss://.../velma-2-ai-music-detection-streaming?api_key=YOUR_API_KEY&audio_format=mp3

Connection parameters

Parameter	Required	Description
`api_key`	Yes	Your API key
`audio_format`	Yes	Audio format - see supported formats below
`sample_rate`	Raw PCM only	Sample rate in Hz
`num_channels`	Raw PCM only	Number of channels (1-8)

Supported audio formats

Container formats - sample_rate and num_channels are ignored if supplied (the headers already carry this metadata): mp3, wav, flac, m4a, mp4, ogg, opus, webm, aac, aiff, wma, amr, and au, plus 66 others

All 79 accepted container values

3g2, 3ga, 3gp, 3gpp, 8svx, aa3, aac, ac3, act, adts, aif, aifc, aiff, amb, amr, asf, at3, au, avr, awb, bwf, c2, caf, dss, dts, dtshd, eac3, ec3, f4a, f4b, flac, gsm, iff, m2a, m2ts, m4a, m4b, m4r, m4v, mka, mkv, mlp, mp+, mp1, mp2, mp3, mp4, mpa, mpc, mpga, mpp, mts, oga, ogg, ogx, oma, omg, opus, paf, pvf, qcp, ra, rf64, rm, rmvb, snd, svx, thd, ts, tta, voc, vqf, w64, wav, wave, weba, webm, wma, wmv

The MP4-family values (mp4, m4a, m4b, m4r, m4v, 3gp, 3gpp, 3ga, 3g2, f4a, f4b) must be sent in a streamable layout; otherwise the connection ends with an audio-processing error. Raw PCM formats - sample_rate and num_channels are required: s8, s16le, s16be, s24le, s24be, s32le, s32be, u8, u16le, u16be, u24le, u24be, u32le, u32be, f32le, f32be, f64le, f64be, mulaw, alaw, g722, vox g722 and vox are mono-only: num_channels must be 1. Valid sample rates: 8000, 11025, 16000, 22050, 32000, 44100, 48000, 96000

Protocol

Client -> server

Message	Description
Binary frame	Chunk of audio bytes in the declared format (any size)
Empty text frame `""`	Signals end of stream

Server -> client

Message	Description
`{"type": "window", "window": {...}}`	Per-window result - emitted for each completed 4-second window, in order
`{"type": "done", ...}`	Stream complete - clip-level verdict plus instrumental AI detection
`{"type": "error", "error": "..."}`	An error occurred - connection will close

Vocal AI and instrumental AI are both scored independently on each 4-second window as audio arrives and reported in window messages - a window can carry both scores, one, or neither, depending on how much of each content type it contains. The window-level instrumental score is computed from a single short window, so it can be less accurate than the clip-level score in the final done message, which is computed from the full accumulated audio.

Window object

Field	Type	Description
`start_time_ms`	integer	Window start time in milliseconds
`end_time_ms`	integer	Window end time in milliseconds
`vocal_percentage`	float	Percentage of the window containing vocal content (0-100)
`vocal_ai_probability`	float or `null`	Probability the window contains AI-generated vocals (0-1). `null` if the window doesn’t have enough vocal content to score, was too short to analyse, or no content was found to classify
`vocal_ai_confidence`	float or `null`	Confidence in `vocal_ai_probability` (0-1); `null` under the same conditions as that field
`instrumental_percentage`	float	Percentage of the window containing instrumental music content (0-100)
`instrumental_ai_probability`	float or `null`	Probability the window contains AI-generated instrumental content (0-1), from a single-window analysis (lower accuracy than the `done` message’s clip-level score). `null` if the window is mostly silence, instrumental-AI detection is unavailable, or the window’s audio could not be analysed
`instrumental_ai_confidence`	float or `null`	Confidence in `instrumental_ai_probability` (0-1); `null` under the same conditions as that field
`silence_percentage`	float	Percentage of the window containing neither vocal nor instrumental content (0-100)

vocal_ai_probability/vocal_ai_confidence and instrumental_ai_probability/instrumental_ai_confidence are gated and scored independently, so a window can carry both pairs, one, or neither.

Done object

Field	Type	Description
`duration_ms`	integer	Total duration of the streamed audio in milliseconds
`window_count`	integer	Total number of windows analysed during the session
`primary_verdict`	string	Clip-level classification: `"ai-vocal-music"`, `"ai-instrumental"`, or `"not-ai-music"`
`vocal_percentage`	float	Average percentage of audio with vocal content, across all windows (0-100)
`vocal_ai_percentage`	float	Percentage of the full clip duration classified as AI-generated vocals (0-100), recomputed at end-of-stream. May differ from what the live per-window messages suggested
`vocal_ai_confidence`	float	Average confidence across frames classified as AI-generated vocals, from the end-of-stream recompute (0-1); `0` if none were found
`instrumental_percentage`	float	Average percentage of audio with instrumental content, across all windows (0-100)
`instrumental_ai_percentage`	float	AI detection score for the clip’s instrumental content (0-100), computed once at end-of-stream from the full accumulated audio - more reliable than the live per-window `instrumental_ai_probability` values
`instrumental_ai_confidence`	float	Confidence in the instrumental AI assessment for the full clip (0-1)
`silence_percentage`	float	Average percentage of audio with neither vocal nor instrumental content (0-100)

WebSocket close codes

Code	Meaning
`1000`	Normal closure after the `done` message, or after the connection completes
`1003`	Invalid or missing query parameters (unknown format, bad sample rate, missing `audio_format`)
`1011`	Internal server error, or processing ended before completion
`4001`	The `api_key` query parameter is missing or invalid
`4002`	Audio could not be decoded, or does not match the declared raw `audio_format`
`4003`	The request is not permitted
`4004`	The API key does not have access to this model
`4029`	The request could not be completed due to insufficient credits
`4030`	Concurrent request limit reached
`4031`	Monthly usage limit reached

An error message is sent before the connection closes for every case above except 1000.

Rate limits

Concurrent connection limits apply per model

Velma

Speech-to-text Transcription

Deepfake Detection

Emotion Detection

Accent Detection

PII/PHI Redaction

Music Detection