PII/PHI Redaction

PII/PHI Redaction transcribes audio and returns a version safe to retain: sensitive spans replaced in the text, and the corresponding ranges silenced in the audio. Redaction transcribes as part of its work, so the transcript comes back either way. A separate transcription call on the same audio is redundant. Each detected span is replaced with an empty marker tag, with the surrounding words preserved. <pii:CATEGORY></pii:CATEGORY> marks personal information, where CATEGORY identifies the entity type. <phi></phi> marks health information.

	Tagging on Transcription	Redaction
Request	`pii_phi_tagging=true` on a Transcription endpoint	The endpoints on this page
Transcript	Sensitive spans wrapped in tags, content preserved	Sensitive spans replaced with empty marker tags
Audio	Unchanged	Sensitive ranges silenced
Best fit	The transcript needs marking up but the original audio is kept	Both transcript and audio must be sanitized

PII/PHI Redaction (batch)

Returns multipart/form-data, not JSON. Two parts:

Part	Content type	Contents
`metadata`	`application/json`	The redacted transcript, utterances, and the ranges silenced.
`audio`	`audio/mpeg`	The redacted MP3.

The metadata part:

Field	Type	Contents
`text`	string	The complete redacted transcript.
`duration_ms`	integer	Total audio duration.
`utterances`	array	Speaker turns, ordered by start time.
`utterances[].utterance_uuid`	string	Identifier for the utterance.
`utterances[].text`	string	Redacted text for this utterance.
`utterances[].start_ms`	integer	Start relative to the beginning of the audio.
`utterances[].duration_ms`	integer	Utterance length.
`utterances[].speaker`	integer	Speaker number, 1-indexed.
`utterances[].language`	string	Detected language code for this utterance.
`redaction_ranges`	array	`[start_ms, end_ms]` pairs silenced in the returned audio.

redaction_ranges reflects the final merged and padded ranges actually applied, not the raw detections, so a range can be wider than the words it covers.

Try it

curl -X POST https://platform.modulate.ai/api/velma-2-pii-phi-redaction-batch \
  -H "X-API-Key: $MODULATE_API_KEY" \
  -F "upload_file=@audio.mp3" \
  -F "speaker_diarization=true" \
  -o response.multipart \
  -D response_headers.txt

import os, json, requests
from requests_toolbelt.multipart.decoder import MultipartDecoder

response = requests.post(
    "https://platform.modulate.ai/api/velma-2-pii-phi-redaction-batch",
    headers={"X-API-Key": os.environ["MODULATE_API_KEY"]},
    data={"speaker_diarization": "true"},
    files={"upload_file": open("audio.mp3", "rb")},
)
response.raise_for_status()

for part in MultipartDecoder.from_response(response).parts:
    disposition = part.headers.get(b"Content-Disposition", b"").decode()
    if 'name="metadata"' in disposition:
        metadata = json.loads(part.content)
        print(metadata["text"])
    elif 'name="audio"' in disposition:
        with open("redacted.mp3", "wb") as f:
            f.write(part.content)

Decoding the multipart response in Python needs requests-toolbelt:

pip install requests-toolbelt

Response (the metadata part)

{
  "text": "My name is <pii:name></pii:name> and my SSN is <pii:ssn></pii:ssn>.",
  "duration_ms": 5600,
  "utterances": [
    {
      "utterance_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890",
      "text": "My name is <pii:name></pii:name> and my SSN is <pii:ssn></pii:ssn>.",
      "start_ms": 0,
      "duration_ms": 5600,
      "speaker": 1,
      "language": "en"
    }
  ],
  "redaction_ranges": [[1100, 1800], [3200, 4400]]
}

What you can configure

Form field	Default	Effect
`upload_file`	required	The audio to transcribe and redact.
`speaker_diarization`	`true`	Adds a 1-indexed `speaker` to each utterance.
`start_redaction_padding_ms`	`100`	Silence prepended to each redacted range, as a buffer before the sensitive words.
`end_redaction_padding_ms`	`0`	Silence appended to each redacted range. Raise this when trailing syllables survive.
`language`	auto-detect	ISO 639-1 hint such as `en` or `fr`. BCP 47 subtags like `en-US` are accepted; only the primary subtag is used. Omit to detect per utterance.

An invalid language is rejected with 400.

Audio formats

Accepted extensions: .aac, .aiff, .flac, .mov, .mp3, .mp4, .ogg, .opus, .wav, .webm. Maximum file size is 100 MB. Empty files are rejected with 400.

PII/PHI Redaction (streaming)

Redacted results arrive utterance by utterance while the call is still running. The stream interleaves JSON text frames and binary MP3 frames, and the pairing rule is the part a client has to get right.

Message	Payload	Binary frame
`utterance`	`utterance` object plus `redacted_audio`	Only when `redacted_audio` is not null
`done`	`duration_ms` plus `trailing_redacted_audio`	Only when `trailing_redacted_audio` is not null
`error`	`error` string	Never

When redacted_audio is not null it carries start_ms and duration_ms describing the window the next binary frame covers. It is null for an out-of-order utterance whose audio was already emitted in a previous window, and in that case no binary frame follows. trailing_redacted_audio works the same way for audio after the final utterance. It is null when the last utterance already reached the end. The utterance object carries utterance_uuid, text, start_ms, duration_ms, speaker, and language.

Try it

websocat "wss://platform.modulate.ai/api/velma-2-pii-phi-redaction-streaming?api_key=$MODULATE_API_KEY&speaker_diarization=true" \
  --binary - < audio.mp3

import os, asyncio, json, websockets

API_KEY = os.environ["MODULATE_API_KEY"]
AUDIO_FILE = "audio.mp3"
CHUNK_SIZE = 4096

async def redact():
    url = (
        f"wss://platform.modulate.ai/api/velma-2-pii-phi-redaction-streaming"
        f"?api_key={API_KEY}&speaker_diarization=true"
    )
    audio_clips = []

    async with websockets.connect(url) as ws:
        async def send():
            with open(AUDIO_FILE, "rb") as f:
                while chunk := f.read(CHUNK_SIZE):
                    await ws.send(chunk)
            await ws.send("")  # end of stream

        async def receive():
            is_done = False
            async for message in ws:
                if isinstance(message, bytes):
                    audio_clips.append(message)
                    if is_done:          # the trailing clip after done
                        break
                    continue
                msg = json.loads(message)
                if msg["type"] == "utterance":
                    u = msg["utterance"]
                    print(f"[{u['start_ms']}ms] Speaker {u['speaker']}: {u['text']}")
                elif msg["type"] == "done":
                    is_done = True
                    if not msg.get("trailing_redacted_audio"):
                        break            # no binary frame follows
                elif msg["type"] == "error":
                    raise RuntimeError(msg["error"])

        await asyncio.gather(send(), receive())

    if audio_clips:
        with open("redacted.mp3", "wb") as f:
            for clip in audio_clips:
                f.write(clip)

asyncio.run(redact())

Messages received

{ "type": "utterance", "utterance": { "utterance_uuid": "a1b2c3d4-e5f6-7890-abcd-ef1234567890", "text": "My name is <pii:name></pii:name> and my SSN is <pii:ssn></pii:ssn>.", "start_ms": 0, "duration_ms": 5600, "speaker": 1, "language": "en" }, "redacted_audio": { "start_ms": 0, "duration_ms": 5600 } }
<binary MP3 frame>
{ "type": "done", "duration_ms": 5600, "trailing_redacted_audio": null }

WebSocket endpoints cannot be exercised with cURL. For command-line testing use websocat.

What you can configure

Query parameter	Default	Effect
`api_key`	required	The API key. WebSocket connections authenticate through the query string, not a header.
`speaker_diarization`	`true`	Adds a 1-indexed `speaker` to each utterance.
`language`	auto-detect	ISO 639-1 hint. Omit to detect per utterance. An invalid code closes the connection with `1003`.
`start_redaction_padding_ms`	`100`	Silence prepended to each redacted range.
`end_redaction_padding_ms`	`0`	Silence appended to each redacted range.
`audio_format`	auto-detect	Optional for self-describing containers. Required for raw formats.
`sample_rate`	required for raw	Sample rate in Hz. Must not be sent for self-describing formats.
`num_channels`	required for raw	1 to 8. Must not be sent for self-describing formats.

Neither redaction endpoint accepts custom_terms. language is the only vocabulary control.

Audio formats

Self-describing formats. wav, mp3, ogg, flac, webm, aac, aiff. These carry sample rate and channel count in the stream, so audio_format is optional and is auto-detected from the headers when omitted. sample_rate and num_channels must not be sent for these. Unlike the detection streaming endpoints, this one auto-detects containers. audio_format is only required for raw audio. Raw formats. s8, s16le, s16be, s24le, s24be, s32le, s32be, u8, u16le, u16be, u24le, u24be, u32le, u32be, f32le, f32be, f64le, f64be, mulaw, alaw. Headerless, so audio_format, sample_rate, and num_channels are all required. Omitting either of the latter two closes the connection with 1003. sample_rate accepts 8000, 11025, 16000, 22050, 32000, 44100, 48000, 96000. Common raw configurations:

Source	`audio_format`	`sample_rate`	`num_channels`
Native app default	`s16le`	`16000`	`1`
Web Audio API (`AudioWorklet`)	`f32le`	`48000`	`1`
Telephony (mu-law)	`mulaw`	`8000`	`1`
Telephony (A-law)	`alaw`	`8000`	`1`

To convert a file to raw PCM:

ffmpeg -i audio.mp3 -ar 16000 -ac 1 -f s16le audio.raw

Get started

By capability

Guides

PII/PHI Redaction

PII/PHI Redaction (batch)

Try it

What you can configure

Audio formats

PII/PHI Redaction (streaming)

Try it

What you can configure

Audio formats

API reference

​PII/PHI Redaction (batch)

​Try it

​What you can configure

​Audio formats

​PII/PHI Redaction (streaming)

​Try it

​What you can configure

​Audio formats

​API reference

PII/PHI Redaction (batch)

Try it

What you can configure

Audio formats

PII/PHI Redaction (streaming)

Try it

What you can configure

Audio formats

API reference