Screen a call for voice fraud

Deepfake Detection reports whether a voice is synthetic. Velma Triage reports what the caller did. Fraud screening needs both, because neither answer predicts the other. An AI scheduling assistant calling on a customer’s behalf is synthetic and legitimate. A human social engineer running an account-takeover script is natural speech and fraudulent. A cloned voice running that same script is both.

Call	Deepfake Detection verdict	Velma Triage detections	Screening outcome
Ordinary customer	`non-synthetic`	none	Proceed
AI assistant acting for a customer	`synthetic`	none	Proceed
Human social engineer	`non-synthetic`	vishing, coercion, account impersonation	Review
Cloned voice running the same script	`synthetic`	vishing, coercion, account impersonation	Review

A synthetic verdict on its own separates rows one and two, which need the same handling. Behavior detections on their own separate rows three and four from the rest, but lose the escalation signal that a cloned voice carries. This page runs both models on one recording and combines the outputs into a single decision.

Before starting

Set your API key

export MODULATE_API_KEY=your_api_key_here

See Authentication for key handling and limits.

Get a call recording in a shared format

Velma Triage batch accepts .aac, .aiff, .flac, .mov, .mp3, .mp4, .ogg, .opus, .wav, and .webm. Deepfake Detection batch accepts those plus .3gp, .3gpp, .amr, .au, .m4a, and .wma. Velma’s list is the binding constraint for a file sent to both.Phone and voicemail recordings are often .m4a, which Velma Triage rejects. Convert first:

ffmpeg -i call.m4a -ac 1 -ar 16000 -c:a pcm_s16le call.wav

Maximum file size is 100 MB on both endpoints.

Install the Python dependency

The curl examples need nothing else.

pip install requests

Step 1: Classify the voice

Which endpoint classifies the voice depends on how the call was recorded.

Recording	Endpoint	Attribution
Caller-only audio, or one channel of a two-channel recording	Deepfake Detection (batch)	A verdict per 4-second frame across the clip
Mixed mono, both parties in one track	Velma Triage (batch) with `stt.deepfake_signal`	A `deepfake_score` per transcript clip, carrying a `speaker_label`

Worth knowing: Deepfake Detection batch classifies single-speaker audio. On a mixed recording of a two-party call, frames cover whichever voice is speaking and no field says which. Split the channels before calling it, or take the second path and read deepfake_score off the clips, where Velma’s diarization has already attributed each clip to a speaker.

The rest of this step covers the dedicated endpoint. For the transcription signal instead, skip to Step 2 and enable stt.deepfake_signal there.

curl -X POST https://platform.modulate.ai/api/velma-2-synthetic-voice-detection-batch \
  -H "X-API-Key: $MODULATE_API_KEY" \
  -F "upload_file=@caller.wav"

import os, requests

response = requests.post(
    "https://platform.modulate.ai/api/velma-2-synthetic-voice-detection-batch",
    headers={"X-API-Key": os.environ["MODULATE_API_KEY"]},
    files={"upload_file": open("caller.wav", "rb")},
)
response.raise_for_status()
frames = response.json()["frames"]

See the Deepfake Detection page for the full response shape.

Aggregate the frames into one verdict

The response is a timeline, not a clip-level verdict. Screening needs one value, so compute it:

Drop no-content frames. They are silence, classified before inference rather than by the model, and counting them dilutes the result on a call with hold time.
Weight by frame duration rather than counting frames, so a trimmed final frame does not carry the same weight as a full 4-second one.
Divide synthetic speech time by total speech time.
Compare against a threshold.

def clip_verdict(frames, threshold=0.5):
    """Collapse per-frame verdicts into one clip-level verdict and a share."""
    speech_ms = synthetic_ms = 0
    for frame in frames:
        if frame["verdict"] == "no-content":
            continue
        duration = frame["end_time_ms"] - frame["start_time_ms"]
        speech_ms += duration
        if frame["verdict"] == "synthetic":
            synthetic_ms += duration

    if speech_ms == 0:
        return "inconclusive", 0.0

    share = synthetic_ms / speech_ms
    return ("synthetic" if share >= threshold else "non-synthetic"), share


verdict, share = clip_verdict(frames)
print(f"{verdict}  ({share:.0%} of speech)")

Worth knowing: 0.5 is a starting point, not a value the API defines. Real recordings carry IVR prompts, hold music, and transfers, so a partial share is common and a clip is rarely 0% or 100%. Set the threshold against your own labelled audio and your tolerance for false positives.

Recommended clip length for this endpoint is 4 to 60 seconds. On a long call, frames still covers the full duration, and the per-frame timestamps locate a synthetic segment rather than only reporting that one exists.

Step 2: Analyze the behaviors

Velma Triage evaluates a configured set of behaviors against the transcript. Nothing is evaluated unless the behaviors array names it, so fraud screening starts by choosing which signals to turn on. These seven presets from the Fraud Detection and Prevention package are the fraud tactics themselves:

Preset	Tactic
`preset:vishing`	Eliciting sensitive information through deception
`preset:account_impersonation`	Claiming to be another account holder
`preset:coercion_manipulation`	Pressure through intimidation or threats
`preset:bargaining_manipulation`	Pressure through cajoling and persuasion
`preset:feigned_ignorance`	Feigned confusion to draw sympathy
`preset:return_fraud_attempt`	Fraudulent product return
`preset:ai_agent_manipulation`	Pushing an AI agent into unintended behavior

curl -X POST https://platform.modulate.ai/api/velma-2-batch \
  -H "X-API-Key: $MODULATE_API_KEY" \
  -F "upload_file=@call.wav" \
  -F 'config={"behaviors":["preset:vishing","preset:account_impersonation","preset:coercion_manipulation","preset:bargaining_manipulation","preset:feigned_ignorance","preset:return_fraud_attempt","preset:ai_agent_manipulation"],"stt":{"deepfake_signal":true}}'

import os, json, requests

FRAUD_TACTICS = [
    "preset:vishing",
    "preset:account_impersonation",
    "preset:coercion_manipulation",
    "preset:bargaining_manipulation",
    "preset:feigned_ignorance",
    "preset:return_fraud_attempt",
    "preset:ai_agent_manipulation",
]

config = {
    "behaviors": FRAUD_TACTICS,
    "stt": {"speaker_diarization": True, "deepfake_signal": True},
}

response = requests.post(
    "https://platform.modulate.ai/api/velma-2-batch",
    headers={"X-API-Key": os.environ["MODULATE_API_KEY"]},
    data={"config": json.dumps(config)},
    files={"upload_file": open("call.wav", "rb")},
)
response.raise_for_status()
result = response.json()

clips = {c["clip_uuid"]: c for c in result["clips"]}

for behavior in result["behaviors"]:
    if not behavior["detected"]:
        continue
    print(f"{behavior['behavior_name']}  {behavior['speaker_label']}  {behavior.get('confidence')}")
    print(f"  {behavior.get('reasoning')}")
    for clip_uuid in behavior.get("evidence_clip_uuids", []):
        print(f"  [{clips[clip_uuid]['start_ms']}ms] {clips[clip_uuid]['text']}")

Every configured behavior comes back whether or not it fired, so detected: false means Velma checked and found nothing, not that the check was skipped. evidence_clip_uuids and definitive_clip_uuid index into clips, which is how the snippet above prints the words that triggered each detection. See Velma Triage for the full response shape.

Worth knowing: the fraud package holds 18 presets, and 11 of them are context and outcome signals rather than fraud tactics. preset:complaints, preset:issue_resolved, and preset:refund_or_credit_issued describe how a call went, and firing on them is not evidence of fraud. Split the list by what each signal means before wiring detections to an action.

Move to the full package

The seven presets above run without any other configuration. The full package adds 25 conversation types and 14 participant roles, which constrain what Velma infers about the call and improve role attribution on support calls. Download fraud-detection-and-prevention.json from the package page and send it as the config value:

curl -X POST https://platform.modulate.ai/api/velma-2-batch \
  -H "X-API-Key: $MODULATE_API_KEY" \
  -F "upload_file=@call.wav" \
  -F "config=<fraud-detection-and-prevention.json"

Preset identifiers resolve server-side against the catalog your organization has access to. An unknown identifier is rejected with 422, so confirm the set available to you before pinning a list:

curl https://platform.modulate.ai/api/velma-2-batch/list-presets \
  -H "X-API-Key: $MODULATE_API_KEY"

Step 3: Combine the two outputs

Each model contributes one axis of the decision.

Deepfake Detection verdict	Fraud-tactic detections	Screening outcome
`non-synthetic`	none	Proceed
`synthetic`	none	Proceed, and record the synthetic verdict
`non-synthetic`	one or more	Route to review
`synthetic`	one or more	Route to review, ahead of the natural-speech cases

def screen(frames, velma_result):
    """Combine both outputs. Assumes the Velma config enabled fraud tactics only."""
    voice, share = clip_verdict(frames)
    tactics = [b["behavior_name"] for b in velma_result["behaviors"] if b["detected"]]

    return {
        "voice": voice,
        "synthetic_share": round(share, 3),
        "tactics": tactics,
        "action": "review" if tactics else "proceed",
        "priority": "high" if tactics and voice == "synthetic" else "normal",
        "summary": velma_result.get("summary"),
    }

Every detection counts as a tactic here because the config enabled nothing else. Running the full package mixes tactics with context and outcome signals, so keep the subset you treat as a tactic and filter on behavior_uuid, which is stable across catalog updates. Filtering on behavior_name breaks when a display name is reworded.

Screening output for a cloned voice running an account-takeover script

{
  "voice": "synthetic",
  "synthetic_share": 0.94,
  "tactics": ["Vishing", "Account Impersonation", "Coercion Manipulation"],
  "action": "review",
  "priority": "high",
  "summary": "A caller reported being locked out, declined to complete verification, asked the agent to read back a one-time code, and threatened a complaint when refused."
}

Behavior detections are signals, not verdicts. A single detection at moderate confidence is weaker evidence than three tactics firing together with the evidence clips agreeing. Weigh several detections against your own rules before acting on any of them.

Build a test matrix

Two synthetic recordings are enough to see that behavior analysis separates them, and not enough to set a threshold. Cover all four quadrants, because each one fails differently.

	Benign intent	Fraudulent intent
Natural speech	Ordinary customer	Human social engineer
Synthetic speech	AI assistant acting for a customer	Cloned voice running a script

Record a natural-speech control

Ten seconds of your own voice, converted with the ffmpeg command above. Deepfake Detection should return non-synthetic across the speech frames. A control that comes back synthetic means the format or the pipeline needs checking before any other result is worth reading.

Generate synthetic samples matching your threat model

Text-to-speech output exercises the detection path but is not what a motivated attacker uses. Test against the voice-cloning tools relevant to the accounts you protect.

Use real call recordings

Consented, compliant recordings from your own environment carry the codecs, channel counts, hold music, and transfers that synthetic test clips do not. Thresholds tuned on clean studio audio move once real calls arrive.

Include benign fraud-adjacent calls

A frustrated customer who cannot remember which email they signed up with produces some of the same acoustic markers as feigned ignorance. These calls set your false-positive rate.

Before trusting the result

Concurrency is capped per model, and the default is 3 in flight against one endpoint. Screening a backlog runs two endpoints, so bound each with its own semaphore rather than retrying into a full queue. A rejected request returns 429 on both, and the detail field distinguishes a full queue from exhausted credits. Match on detail rather than on the status alone. See Limits. Deepfake Detection classifies acoustically and reports no reason for its verdict. Velma Triage returns reasoning and evidence clips for every detection, which is what an analyst reviewing a flagged call can act on. Route synthetic-voice verdicts to a queue where a human sees the transcript, not to an automatic block.

Deepfake Detection, both endpoints and how the frame verdicts compare with the transcription signal
Velma Triage, the full configuration reference for batch and streaming
Fraud Detection and Prevention, all 18 presets with detection criteria
Using behaviors, listing the catalog and writing custom definitions
Which API should I use?, the choice between endpoints that return overlapping signals

Get started

By capability

Build end to end

Guides

Screen a call for voice fraud

Before starting

Step 1: Classify the voice

Aggregate the frames into one verdict

Step 2: Analyze the behaviors

Move to the full package

Step 3: Combine the two outputs

Build a test matrix

Before trusting the result

​Before starting

​Step 1: Classify the voice

​Aggregate the frames into one verdict

​Step 2: Analyze the behaviors

​Move to the full package

​Step 3: Combine the two outputs

​Build a test matrix

​Before trusting the result

​Related

Before starting

Step 1: Classify the voice

Aggregate the frames into one verdict

Step 2: Analyze the behaviors

Move to the full package

Step 3: Combine the two outputs

Build a test matrix

Before trusting the result

Related