> ## Documentation Index
> Fetch the complete documentation index at: https://docs.modulate.ai/llms.txt
> Use this file to discover all available pages before exploring further.

# Music Detection Streaming

> Real-time frame-level music and speech classification over WebSocket. Frames are emitted progressively as audio is streamed.

Real-time frame-level music and speech classification over WebSocket. Frames are returned progressively as audio is streamed — no need to wait for the full file to upload before results begin arriving.

## Endpoint

```text theme={null}
wss://platform.modulate.ai/api/velma-2-music-detection-streaming
```

## Authentication

Pass your API key as a query parameter on the connection URL:

```text theme={null}
wss://.../velma-2-music-detection-streaming?api_key=YOUR_API_KEY&audio_format=s16le&...
```

## Features

* **Real-time output** — frames emitted progressively after each 192ms chunk of audio
* **Music detection** — identifies frames containing music content
* **Speech detection** — identifies frames containing speech content
* **Non-exclusive labels** — music and speech are independent; both can be high simultaneously (e.g. music with vocals)
* **Any chunk size** — send audio in whatever chunk size suits your pipeline
* **Container and raw PCM support** — stream compressed files or raw PCM directly from a microphone

## Connection parameters

| Parameter | Required | Description |
| - | - | - |
| `api_key` | Yes | Your API key |
| `audio_format` | Yes | Audio format — see supported formats below |
| `sample_rate` | Raw PCM only | Sample rate in Hz |
| `num_channels` | Raw PCM only | Number of channels (1–8) |

## Supported audio formats

**Container formats** — `sample_rate` and `num_channels` are ignored if supplied (the headers already carry this metadata):

`mp3`, `wav`, `flac`, `m4a`, `mp4`, `ogg`, `opus`, `webm`, `aac`, `aiff`, `wma`, `amr`, and `au`, plus 66 others

<Accordion title="All 79 accepted container values">
  `3g2`, `3ga`, `3gp`, `3gpp`, `8svx`, `aa3`, `aac`, `ac3`, `act`, `adts`, `aif`, `aifc`, `aiff`,
  `amb`, `amr`, `asf`, `at3`, `au`, `avr`, `awb`, `bwf`, `c2`, `caf`, `dss`, `dts`, `dtshd`,
  `eac3`, `ec3`, `f4a`, `f4b`, `flac`, `gsm`, `iff`, `m2a`, `m2ts`, `m4a`, `m4b`, `m4r`, `m4v`,
  `mka`, `mkv`, `mlp`, `mp+`, `mp1`, `mp2`, `mp3`, `mp4`, `mpa`, `mpc`, `mpga`, `mpp`, `mts`,
  `oga`, `ogg`, `ogx`, `oma`, `omg`, `opus`, `paf`, `pvf`, `qcp`, `ra`, `rf64`, `rm`, `rmvb`,
  `snd`, `svx`, `thd`, `ts`, `tta`, `voc`, `vqf`, `w64`, `wav`, `wave`, `weba`, `webm`, `wma`,
  `wmv`
</Accordion>

The MP4-family values (`mp4`, `m4a`, `m4b`, `m4r`, `m4v`, `3gp`, `3gpp`, `3ga`, `3g2`, `f4a`, `f4b`) must be sent in a streamable layout; otherwise the connection ends with an audio-processing error.

**Raw PCM formats** — `sample_rate` and `num_channels` are required:

`s8`, `s16le`, `s16be`, `s24le`, `s24be`, `s32le`, `s32be`, `u8`, `u16le`, `u16be`, `u24le`, `u24be`, `u32le`, `u32be`, `f32le`, `f32be`, `f64le`, `f64be`, `mulaw`, `alaw`, `g722`, `vox`

`g722` and `vox` are mono-only: `num_channels` must be `1`.

**Valid sample rates:** 8000, 11025, 16000, 22050, 32000, 44100, 48000, 96000

## Protocol

### Client → server

| Message | Description |
| - | - |
| Binary frame | Chunk of audio bytes in the declared format (any size) |
| Empty text frame `""` | Signals end of stream |

### Server → client

| Message | Description |
| - | - |
| `{"type": "frame", "frame": {...}}` | Frame result — emitted after each 192ms chunk |
| `{"type": "done", "duration_ms": ..., ...}` | Stream complete — includes overall summary across all frames |
| `{"type": "error", "error": "..."}` | An error occurred — connection will close |

### Frame object

| Field | Type | Description |
| - | - | - |
| `start_time_ms` | integer | Frame start time in milliseconds |
| `end_time_ms` | integer | Frame end time in milliseconds |
| `music_prob` | float | Music probability (0.0–1.0) |
| `speech_prob` | float | Speech probability (0.0–1.0) |

### Done object

| Field | Type | Description |
| - | - | - |
| `duration_ms` | integer | Total audio duration processed in milliseconds |
| `frame_count` | integer | Total number of frames returned |
| `music_pct` | float | Percentage of the analysed duration classified as music, rounded to one decimal place. `0.0` when no audio was analysed |
| `speech_pct` | float | Percentage of the analysed duration classified as speech, rounded to one decimal place. `0.0` when no audio was analysed |
| `primary_label` | string | Dominant classification: `"music"`, `"speech"`, `"neither"`, or `"unknown"` (no frames produced from the audio) |

## WebSocket close codes

| Code | Meaning |
| - | - |
| `1000` | Normal closure after a successful `done` message |
| `1003` | Invalid query parameters (unknown format, bad sample rate, missing `audio_format`) |
| `1011` | Internal server error |
| `4001` | The `api_key` query parameter is missing or invalid |
| `4002` | Audio could not be decoded or does not match the declared format |
| `4003` | The request is not permitted |
| `4004` | The API key does not have access to this model |
| `4029` | Insufficient credits |
| `4030` | Concurrent request limit reached |
| `4031` | Monthly usage limit reached |

## Chunking behaviour

Audio is buffered in 192ms chunks (one output frame each). Frames are emitted as soon as each chunk is ready, so results begin arriving within 192ms of the first audio being received. At end-of-stream, any remaining audio ≥ 192ms is processed and its frames are emitted before the `done` message.

## Examples

<CodeGroup>
  ```python Python (raw PCM) theme={null}
  import asyncio
  import websockets
  import json

  WS_URL = "wss://platform.modulate.ai/api/velma-2-music-detection-streaming"
  API_KEY = "YOUR_API_KEY"

  async def stream_audio(file_path: str) -> None:
      url = (
          f"{WS_URL}?api_key={API_KEY}"
          f"&audio_format=s16le&sample_rate=16000&num_channels=1"
      )
      async with websockets.connect(url) as ws:
          # Send audio in chunks
          with open(file_path, "rb") as f:
              while chunk := f.read(16000):  # 0.5s of s16le/16kHz mono
                  await ws.send(chunk)

          # Signal end of stream
          await ws.send("")

          # Receive results
          async for message in ws:
              msg = json.loads(message)
              if msg["type"] == "frame":
                  frame = msg["frame"]
                  print(
                      f"{frame['start_time_ms']}ms – {frame['end_time_ms']}ms  "
                      f"music={frame['music_prob']:.4f}  speech={frame['speech_prob']:.4f}"
                  )
              elif msg["type"] == "done":
                  print(f"\nDone — {msg['duration_ms']}ms, {msg['frame_count']} frames")
                  print(f"music={msg['music_pct']}%  speech={msg['speech_pct']}%  label={msg['primary_label']}")
                  break
              elif msg["type"] == "error":
                  raise RuntimeError(f"Server error: {msg['error']}")

  asyncio.run(stream_audio("/path/to/audio.raw"))
  ```

  ```python Python (container format) theme={null}
  import asyncio
  import websockets
  import json

  WS_URL = "wss://platform.modulate.ai/api/velma-2-music-detection-streaming"
  API_KEY = "YOUR_API_KEY"

  async def stream_audio_file(file_path: str, audio_format: str) -> None:
      url = f"{WS_URL}?api_key={API_KEY}&audio_format={audio_format}"
      async with websockets.connect(url) as ws:
          with open(file_path, "rb") as f:
              while chunk := f.read(65536):
                  await ws.send(chunk)
          await ws.send("")

          async for message in ws:
              msg = json.loads(message)
              if msg["type"] == "frame":
                  frame = msg["frame"]
                  print(
                      f"{frame['start_time_ms']}ms – {frame['end_time_ms']}ms  "
                      f"music={frame['music_prob']:.4f}  speech={frame['speech_prob']:.4f}"
                  )
              elif msg["type"] == "done":
                  print(f"\nDone — {msg['duration_ms']}ms, {msg['frame_count']} frames")
                  print(f"music={msg['music_pct']}%  speech={msg['speech_pct']}%  label={msg['primary_label']}")
                  break
              elif msg["type"] == "error":
                  raise RuntimeError(f"Server error: {msg['error']}")

  asyncio.run(stream_audio_file("/path/to/audio.mp3", "mp3"))
  ```

  ```javascript JavaScript (Node.js) theme={null}
  import { WebSocket } from "ws";
  import { createReadStream } from "fs";

  const WS_URL = "wss://platform.modulate.ai/api/velma-2-music-detection-streaming";
  const API_KEY = "YOUR_API_KEY";

  async function streamAudio(filePath, audioFormat) {
    const url = `${WS_URL}?api_key=${API_KEY}&audio_format=${audioFormat}`;
    const ws = new WebSocket(url);

    await new Promise((resolve, reject) => {
      ws.on("open", () => {
        const stream = createReadStream(filePath, { highWaterMark: 65536 });
        stream.on("data", (chunk) => ws.send(chunk));
        stream.on("end", () => ws.send(""));
        stream.on("error", reject);
      });

      ws.on("message", (data) => {
        const msg = JSON.parse(data);
        if (msg.type === "frame") {
          const { start_time_ms, end_time_ms, music_prob, speech_prob } = msg.frame;
          console.log(
            `${start_time_ms}ms – ${end_time_ms}ms  ` +
            `music=${music_prob.toFixed(4)}  speech=${speech_prob.toFixed(4)}`
          );
        } else if (msg.type === "done") {
          console.log(`\nDone — ${msg.duration_ms}ms, ${msg.frame_count} frames`);
          console.log(`music=${msg.music_pct}%  speech=${msg.speech_pct}%  label=${msg.primary_label}`);
          ws.close();
          resolve();
        } else if (msg.type === "error") {
          reject(new Error(`Server error: ${msg.error}`));
        }
      });

      ws.on("error", reject);
    });
  }

  await streamAudio("/path/to/audio.mp3", "mp3");
  ```
</CodeGroup>

## Rate limits

* Concurrent connection limits apply per model


## AsyncAPI

````yaml api/velma_2_music_detection_streaming.yaml streaming-detection
id: streaming-detection
title: Streaming-detection
description: |
  ## WebSocket Close Codes
  - `1000`: Normal closure after the `done` message, or after the
    connection completes.
  - `1003`: Invalid or missing query parameters (unknown `audio_format`,
    invalid `sample_rate` or `num_channels`, a raw format missing
    `sample_rate`/`num_channels`, `sample_rate`/`num_channels` supplied
    without `audio_format`, or `audio_format` omitted). An `error` message
    is sent before the close.
  - `4002`: The audio could not be decoded, or does not match the
    declared raw `audio_format`. An `error` message is sent before the
    close.
  - `1011`: Internal server error. An `error` message is sent before the
    close.
  - `4001`: The `api_key` query parameter is missing or invalid. An
    `error` message is sent before the close.
  - `4003`: The request is not permitted. An `error` message is sent
    before the close.
  - `4004`: The API key does not have access to this model. An `error`
    message is sent before the close.
  - `4029`: The request could not be completed due to insufficient credits.
    An `error` message is sent before the close.
  - `4030`: Concurrent request limit reached. An `error` message is sent
    before the close.
  - `4031`: Monthly usage limit reached. An `error` message is sent before
    the close.
servers:
  - id: production
    protocol: wss
    host: platform.modulate.ai
    bindings: []
    variables: []
address: /api/velma-2-music-detection-streaming
parameters: []
bindings:
  - protocol: ws
    version: latest
    value:
      query:
        type: object
        required:
          - api_key
          - audio_format
        properties:
          api_key:
            type: string
            description: API key used for authentication and usage tracking.
          audio_format:
            type: string
            description: >
              Audio encoding format. Required for all connections. Raw formats

              (headerless samples) also require `sample_rate` and

              `num_channels`; container formats carry that metadata in the

              stream, so those parameters are ignored if supplied. The
              MP4-family

              values (`mp4`, `m4a`, `m4b`, `m4r`, `m4v`, `3gp`, `3gpp`,

              `3ga`, `3g2`, `f4a`, `f4b`) must be sent in a streamable

              layout; otherwise the connection ends with an

              audio-processing error.

              The raw `g722` and `vox` formats are mono-only (`num_channels`

              must be `1`). A larger value passes parameter validation and

              the connection ends with close code `4002` after the first

              audio frame.
            enum:
              - s8
              - s16le
              - s16be
              - s24le
              - s24be
              - s32le
              - s32be
              - u8
              - u16le
              - u16be
              - u24le
              - u24be
              - u32le
              - u32be
              - f32le
              - f32be
              - f64le
              - f64be
              - mulaw
              - alaw
              - g722
              - vox
              - 3g2
              - 3ga
              - 3gp
              - 3gpp
              - 8svx
              - aa3
              - aac
              - ac3
              - act
              - adts
              - aif
              - aifc
              - aiff
              - amb
              - amr
              - asf
              - at3
              - au
              - avr
              - awb
              - bwf
              - c2
              - caf
              - dss
              - dts
              - dtshd
              - eac3
              - ec3
              - f4a
              - f4b
              - flac
              - gsm
              - iff
              - m2a
              - m2ts
              - m4a
              - m4b
              - m4r
              - m4v
              - mka
              - mkv
              - mlp
              - mp+
              - mp1
              - mp2
              - mp3
              - mp4
              - mpa
              - mpc
              - mpga
              - mpp
              - mts
              - oga
              - ogg
              - ogx
              - oma
              - omg
              - opus
              - paf
              - pvf
              - qcp
              - ra
              - rf64
              - rm
              - rmvb
              - snd
              - svx
              - thd
              - ts
              - tta
              - voc
              - vqf
              - w64
              - wav
              - wave
              - weba
              - webm
              - wma
              - wmv
            examples: &ref_0
              - s16le
              - f32le
              - mulaw
              - webm
          sample_rate:
            type: integer
            description: |
              Sample rate in Hz. **Required** when `audio_format` is a raw
              format. Not required and ignored for container formats. Must
              not be specified without `audio_format`.
            enum:
              - 8000
              - 11025
              - 16000
              - 22050
              - 32000
              - 44100
              - 48000
              - 96000
            examples: &ref_1
              - 16000
              - 48000
          num_channels:
            type: integer
            description: |
              Number of audio channels. **Required** when `audio_format` is
              a raw format. Must be between 1 and 8. Not required and
              ignored for container formats. Must not be specified without
              `audio_format`.
            minimum: 1
            maximum: 8
            examples: &ref_2
              - 1
              - 2
    schemaProperties:
      - name: query
        type: object
        required: false
        properties:
          - name: api_key
            type: string
            description: API key used for authentication and usage tracking.
            required: true
          - name: audio_format
            type: string
            description: >
              Audio encoding format. Required for all connections. Raw formats

              (headerless samples) also require `sample_rate` and

              `num_channels`; container formats carry that metadata in the

              stream, so those parameters are ignored if supplied. The
              MP4-family

              values (`mp4`, `m4a`, `m4b`, `m4r`, `m4v`, `3gp`, `3gpp`,

              `3ga`, `3g2`, `f4a`, `f4b`) must be sent in a streamable

              layout; otherwise the connection ends with an

              audio-processing error.

              The raw `g722` and `vox` formats are mono-only (`num_channels`

              must be `1`). A larger value passes parameter validation and

              the connection ends with close code `4002` after the first

              audio frame.
            enumValues:
              - s8
              - s16le
              - s16be
              - s24le
              - s24be
              - s32le
              - s32be
              - u8
              - u16le
              - u16be
              - u24le
              - u24be
              - u32le
              - u32be
              - f32le
              - f32be
              - f64le
              - f64be
              - mulaw
              - alaw
              - g722
              - vox
              - 3g2
              - 3ga
              - 3gp
              - 3gpp
              - 8svx
              - aa3
              - aac
              - ac3
              - act
              - adts
              - aif
              - aifc
              - aiff
              - amb
              - amr
              - asf
              - at3
              - au
              - avr
              - awb
              - bwf
              - c2
              - caf
              - dss
              - dts
              - dtshd
              - eac3
              - ec3
              - f4a
              - f4b
              - flac
              - gsm
              - iff
              - m2a
              - m2ts
              - m4a
              - m4b
              - m4r
              - m4v
              - mka
              - mkv
              - mlp
              - mp+
              - mp1
              - mp2
              - mp3
              - mp4
              - mpa
              - mpc
              - mpga
              - mpp
              - mts
              - oga
              - ogg
              - ogx
              - oma
              - omg
              - opus
              - paf
              - pvf
              - qcp
              - ra
              - rf64
              - rm
              - rmvb
              - snd
              - svx
              - thd
              - ts
              - tta
              - voc
              - vqf
              - w64
              - wav
              - wave
              - weba
              - webm
              - wma
              - wmv
            examples: *ref_0
            required: true
          - name: sample_rate
            type: integer
            description: |
              Sample rate in Hz. **Required** when `audio_format` is a raw
              format. Not required and ignored for container formats. Must
              not be specified without `audio_format`.
            enumValues:
              - 8000
              - 11025
              - 16000
              - 22050
              - 32000
              - 44100
              - 48000
              - 96000
            examples: *ref_1
            required: false
          - name: num_channels
            type: integer
            description: |
              Number of audio channels. **Required** when `audio_format` is
              a raw format. Must be between 1 and 8. Not required and
              ignored for container formats. Must not be specified without
              `audio_format`.
            examples: *ref_2
            required: false
operations:
  - &ref_17
    id: sendAudio
    title: Send audio
    description: Stream audio data to the server.
    type: send
    messages:
      - &ref_22
        id: audioData
        contentType: application/octet-stream
        payload:
          - type: string
            format: binary
            description: A binary frame containing audio samples.
            x-parser-schema-id: <anonymous-schema-1>
            name: Audio Data
        headers: []
        jsonPayloadSchema:
          type: string
          format: binary
          description: |
            Audio data in the format declared by the `audio_format` query
            parameter. For raw formats this is headerless samples; for
            container formats this is the encoded byte stream. Frames may
            be any size; the server buffers internally.
          x-parser-schema-id: <anonymous-schema-1>
        title: Audio Data
        description: A binary frame containing audio samples.
        example: '{}'
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: audioData
    bindings: []
    extensions: &ref_3
      - id: x-parser-unique-object-id
        value: streaming-detection
  - &ref_18
    id: signalEnd
    title: Signal end
    description: Signal end of audio stream.
    type: send
    messages:
      - &ref_23
        id: endOfStream
        contentType: text/plain
        payload:
          - type: string
            const: ''
            x-parser-schema-id: <anonymous-schema-2>
            name: End of Stream
            description: >-
              An empty text frame signalling that the client has finished
              sending audio.
        headers: []
        jsonPayloadSchema:
          type: string
          const: ''
          x-parser-schema-id: <anonymous-schema-2>
        title: End of Stream
        description: >-
          An empty text frame signalling that the client has finished sending
          audio.
        example: '{}'
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: endOfStream
    bindings: []
    extensions: *ref_3
  - &ref_14
    id: receiveFrameResult
    title: Receive frame result
    description: Receive a per-frame detection result.
    type: receive
    messages:
      - &ref_19
        id: frameResult
        contentType: application/json
        payload:
          - name: Frame Result
            description: A single 192-millisecond frame detection result.
            type: object
            properties:
              - name: type
                type: string
                description: frame
                required: true
              - name: frame
                type: object
                required: true
                properties:
                  - name: start_time_ms
                    type: integer
                    description: >-
                      Frame start time in the original audio stream, in
                      milliseconds.
                    examples: &ref_4
                      - 0
                    required: true
                  - name: end_time_ms
                    type: integer
                    description: >
                      Frame end time in the original audio stream, in
                      milliseconds.

                      Each frame spans 192 milliseconds.
                    examples: &ref_5
                      - 192
                    required: true
                  - name: music_prob
                    type: number
                    description: >
                      Probability that this frame contains music, rounded to
                      four

                      decimal places. Independent of `speech_prob`.
                    examples: &ref_6
                      - 0.0213
                    required: true
                  - name: speech_prob
                    type: number
                    description: >
                      Probability that this frame contains speech, rounded to
                      four

                      decimal places. Independent of `music_prob`; music and
                      speech

                      are not mutually exclusive (e.g. music with vocals).
                    examples: &ref_7
                      - 0.9888
                    required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
            - frame
          properties:
            type:
              type: string
              const: frame
              x-parser-schema-id: <anonymous-schema-3>
            frame:
              type: object
              required:
                - start_time_ms
                - end_time_ms
                - music_prob
                - speech_prob
              properties:
                start_time_ms:
                  type: integer
                  minimum: 0
                  description: >-
                    Frame start time in the original audio stream, in
                    milliseconds.
                  examples: *ref_4
                  x-parser-schema-id: <anonymous-schema-4>
                end_time_ms:
                  type: integer
                  minimum: 0
                  description: >
                    Frame end time in the original audio stream, in
                    milliseconds.

                    Each frame spans 192 milliseconds.
                  examples: *ref_5
                  x-parser-schema-id: <anonymous-schema-5>
                music_prob:
                  type: number
                  format: double
                  minimum: 0
                  maximum: 1
                  description: |
                    Probability that this frame contains music, rounded to four
                    decimal places. Independent of `speech_prob`.
                  examples: *ref_6
                  x-parser-schema-id: <anonymous-schema-6>
                speech_prob:
                  type: number
                  format: double
                  minimum: 0
                  maximum: 1
                  description: >
                    Probability that this frame contains speech, rounded to four

                    decimal places. Independent of `music_prob`; music and
                    speech

                    are not mutually exclusive (e.g. music with vocals).
                  examples: *ref_7
                  x-parser-schema-id: <anonymous-schema-7>
              x-parser-schema-id: Frame
          x-parser-schema-id: FrameResultMessage
        title: Frame Result
        description: A single 192-millisecond frame detection result.
        example: |-
          {
            "type": "<string>",
            "frame": {
              "start_time_ms": 123,
              "end_time_ms": 123,
              "music_prob": 123,
              "speech_prob": 123
            }
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: frameResult
    bindings: []
    extensions: *ref_3
  - &ref_15
    id: receiveDone
    title: Receive done
    description: Receive the final summary when analysis is complete.
    type: receive
    messages:
      - &ref_20
        id: done
        contentType: application/json
        payload:
          - name: Done
            description: Analysis complete - all frame results have been delivered.
            type: object
            properties:
              - name: type
                type: string
                description: done
                required: true
              - name: duration_ms
                type: integer
                description: >-
                  Total duration of the streamed audio processed, in
                  milliseconds.
                examples: &ref_8
                  - 15360
                required: true
              - name: frame_count
                type: integer
                description: Total number of frame results delivered during the session.
                examples: &ref_9
                  - 80
                required: true
              - name: music_pct
                type: number
                description: >
                  Percentage of the analysed duration classified as music,

                  rounded to one decimal place. `0.0` when no audio was
                  analysed.
                examples: &ref_10
                  - 42.5
                required: true
              - name: speech_pct
                type: number
                description: >
                  Percentage of the analysed duration classified as speech,

                  rounded to one decimal place. `0.0` when no audio was
                  analysed.
                examples: &ref_11
                  - 57.5
                required: true
              - name: primary_label
                type: string
                description: |
                  The dominant classification across the analysed audio.

                  - `music` - music covers at least as much of the audio as
                    speech, and more than zero.
                  - `speech` - speech covers more of the audio than music, and
                    more than zero.
                  - `neither` - neither music nor speech reached the dominant
                    threshold for any portion of the audio.
                  - `unknown` - no audio was analysed (no frames produced).
                enumValues:
                  - music
                  - speech
                  - neither
                  - unknown
                examples: &ref_12
                  - speech
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
            - duration_ms
            - frame_count
            - music_pct
            - speech_pct
            - primary_label
          properties:
            type:
              type: string
              const: done
              x-parser-schema-id: <anonymous-schema-8>
            duration_ms:
              type: integer
              minimum: 0
              description: Total duration of the streamed audio processed, in milliseconds.
              examples: *ref_8
              x-parser-schema-id: <anonymous-schema-9>
            frame_count:
              type: integer
              minimum: 0
              description: Total number of frame results delivered during the session.
              examples: *ref_9
              x-parser-schema-id: <anonymous-schema-10>
            music_pct:
              type: number
              format: double
              minimum: 0
              maximum: 100
              description: |
                Percentage of the analysed duration classified as music,
                rounded to one decimal place. `0.0` when no audio was analysed.
              examples: *ref_10
              x-parser-schema-id: <anonymous-schema-11>
            speech_pct:
              type: number
              format: double
              minimum: 0
              maximum: 100
              description: |
                Percentage of the analysed duration classified as speech,
                rounded to one decimal place. `0.0` when no audio was analysed.
              examples: *ref_11
              x-parser-schema-id: <anonymous-schema-12>
            primary_label:
              type: string
              enum:
                - music
                - speech
                - neither
                - unknown
              description: |
                The dominant classification across the analysed audio.

                - `music` - music covers at least as much of the audio as
                  speech, and more than zero.
                - `speech` - speech covers more of the audio than music, and
                  more than zero.
                - `neither` - neither music nor speech reached the dominant
                  threshold for any portion of the audio.
                - `unknown` - no audio was analysed (no frames produced).
              examples: *ref_12
              x-parser-schema-id: <anonymous-schema-13>
          x-parser-schema-id: DoneMessage
        title: Done
        description: Analysis complete - all frame results have been delivered.
        example: |-
          {
            "type": "<string>",
            "duration_ms": 123,
            "frame_count": 123,
            "music_pct": 123,
            "speech_pct": 123,
            "primary_label": "<string>"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: done
    bindings: []
    extensions: *ref_3
  - &ref_16
    id: receiveError
    title: Receive error
    description: Receive an error notification.
    type: receive
    messages:
      - &ref_21
        id: error
        contentType: application/json
        payload:
          - name: Error
            description: An error occurred during connection or analysis.
            type: object
            properties:
              - name: type
                type: string
                description: error
                required: true
              - name: error
                type: string
                description: Human-readable error description.
                examples: &ref_13
                  - Internal server error
                  - This request is not permitted.
                  - Invalid API key.
                  - This API key does not have access to this model.
                  - Insufficient credits.
                  - >-
                    Concurrent request limit reached. Please retry after your
                    in-flight requests complete.
                  - Monthly usage limit reached.
                  - Audio data does not match the declared audio_format.
                  - >-
                    Invalid audio_format='mid'. Valid values: ['3g2', '3ga',
                    '3gp', '3gpp', '8svx', 'aa3', 'aac', 'ac3', 'act', 'adts',
                    'aif', 'aifc', 'aiff', 'alaw', 'amb', 'amr', 'asf', 'at3',
                    'au', 'avr', 'awb', 'bwf', 'c2', 'caf', 'dss', 'dts',
                    'dtshd', 'eac3', 'ec3', 'f32be', 'f32le', 'f4a', 'f4b',
                    'f64be', 'f64le', 'flac', 'g722', 'gsm', 'iff', 'm2a',
                    'm2ts', 'm4a', 'm4b', 'm4r', 'm4v', 'mka', 'mkv', 'mlp',
                    'mp+', 'mp1', 'mp2', 'mp3', 'mp4', 'mpa', 'mpc', 'mpga',
                    'mpp', 'mts', 'mulaw', 'oga', 'ogg', 'ogx', 'oma', 'omg',
                    'opus', 'paf', 'pvf', 'qcp', 'ra', 'rf64', 'rm', 'rmvb',
                    's16be', 's16le', 's24be', 's24le', 's32be', 's32le', 's8',
                    'snd', 'svx', 'thd', 'ts', 'tta', 'u16be', 'u16le', 'u24be',
                    'u24le', 'u32be', 'u32le', 'u8', 'voc', 'vox', 'vqf', 'w64',
                    'wav', 'wave', 'weba', 'webm', 'wma', 'wmv']
                  - >-
                    Raw audio formats require sample_rate and num_channels query
                    parameters
                  - >-
                    sample_rate and num_channels require audio_format to be
                    specified
                  - >-
                    Invalid sample_rate=50000. Valid values: [8000, 11025,
                    16000, 22050, 32000, 44100, 48000, 96000]
                  - Invalid num_channels=12. Must be between 1 and 8
                  - >-
                    audio_format query parameter is required. Example:
                    audio_format=s16le&sample_rate=16000&num_channels=1
                required: true
        headers: []
        jsonPayloadSchema:
          type: object
          required:
            - type
            - error
          properties:
            type:
              type: string
              const: error
              x-parser-schema-id: <anonymous-schema-14>
            error:
              type: string
              description: Human-readable error description.
              examples: *ref_13
              x-parser-schema-id: <anonymous-schema-15>
          x-parser-schema-id: ErrorResponse
        title: Error
        description: An error occurred during connection or analysis.
        example: |-
          {
            "type": "<string>",
            "error": "<string>"
          }
        bindings: []
        extensions:
          - id: x-parser-unique-object-id
            value: error
    bindings: []
    extensions: *ref_3
sendOperations:
  - *ref_14
  - *ref_15
  - *ref_16
receiveOperations:
  - *ref_17
  - *ref_18
sendMessages:
  - *ref_19
  - *ref_20
  - *ref_21
receiveMessages:
  - *ref_22
  - *ref_23
extensions:
  - id: x-parser-unique-object-id
    value: streaming-detection
securitySchemes: []

````

This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.