Emotion Detection

Emotion Detection classifies emotional tone from the voice signal. Classification is acoustic: it reflects how something was said, not the words used. Two recordings of identical text can receive different labels if the delivery differs. The endpoint produces no transcript, diarization, or enrichment data.

Emotion Detection (batch)

Returns application/json.

Field	Type	Contents
`emotion`	string	The label for the whole file. Always present, never null, at any file length.
`time_series`	array	Consecutive fixed-length windows over the file, each with its own label. Empty when the file is shorter than one window.
`time_series[].start_ms`	integer	Window start, in milliseconds from the beginning of the file.
`time_series[].duration_ms`	integer	Window length.
`time_series[].emotion`	string	The label for that window.

There is no top-level duration field. Derive audio length from the last window if it is needed.

Try it

curl -sS -X POST https://platform.modulate.ai/api/velma-2-emotion-batch \
  -H "X-API-Key: $MODULATE_API_KEY" \
  -F "upload_file=@audio.mp3" \
| jq -r 'if .detail then "error: \(.detail)" else
    "Overall: \(.emotion)",
    (.time_series[] | "  \(.start_ms)-\(.start_ms + .duration_ms) ms: \(.emotion)")
  end'

import os, requests

response = requests.post(
    "https://platform.modulate.ai/api/velma-2-emotion-batch",
    headers={"X-API-Key": os.environ["MODULATE_API_KEY"]},
    files={"upload_file": open("audio.mp3", "rb")},
)
response.raise_for_status()
result = response.json()

print(f"Overall: {result['emotion']}")
for window in result["time_series"]:
    end = window["start_ms"] + window["duration_ms"]
    print(f"  {window['start_ms']}-{end} ms: {window['emotion']}")

import fs from "fs";
import FormData from "form-data";

const form = new FormData();
form.append("upload_file", fs.createReadStream("audio.mp3"), { filename: "audio.mp3" });

const response = await fetch(
  "https://platform.modulate.ai/api/velma-2-emotion-batch",
  {
    method: "POST",
    headers: { "X-API-Key": process.env.MODULATE_API_KEY, ...form.getHeaders() },
    body: form,
  }
);

const result = await response.json();

if (!response.ok) {
  console.error(`error: ${result.detail}`);
  process.exit(1);
}
console.log(`Overall: ${result.emotion}`);
for (const w of result.time_series) {
  console.log(`  ${w.start_ms}-${w.start_ms + w.duration_ms} ms: ${w.emotion}`);
}

Output

Overall: Happy
  0-15000 ms: Happy
  15000-30000 ms: Neutral

Response

{
  "emotion": "Happy",
  "time_series": [
    { "start_ms": 0, "duration_ms": 15000, "emotion": "Happy" },
    { "start_ms": 15000, "duration_ms": 15000, "emotion": "Neutral" }
  ]
}

What you can configure

Form field	Default	Effect
`upload_file`	required	The audio to analyze.

Audio formats

Accepted extensions: .aac, .aiff, .flac, .mov, .mp3, .mp4, .ogg, .opus, .wav, .webm. Maximum file size is 100 MB. Empty files are rejected with 400.

Labels

Angry, Disgusted, Afraid, Happy, Neutral, Other, Sad, Surprised, Unknown.

Working with the time series

Windows are consecutive and cover the file from the start. A trailing remainder shorter than one full window is dropped. A file shorter than a single window returns an empty time_series, but emotion is still present, so the whole-file label is safe to rely on at any length.

result = response.json()

overall = result["emotion"]

if result["time_series"]:
    shifts = [w["emotion"] for w in result["time_series"]]
    print(f"{overall} overall; window by window: {shifts}")
else:
    print(f"{overall} (file shorter than one window)")

Emotion alongside a transcript

Multilingual Transcription accepts emotion_signal=true, which attaches an emotion label to every utterance in the same call as the transcript. That label set is separate from the one above. Use that when a transcript is also needed. This endpoint exists for whole-file classification with no transcript. See Transcription.

API reference

Emotion Detection Batch

Get started

By capability

Build end to end

Guides

Emotion Detection