Skip to navigation

Audio formats

Supported input and output encodings.

The ASR, STS, and TTS APIs all speak JSON over WebSocket, and audio payloads are Base64-encoded inside event bodies. The Python SDK always sends pcm16 on WebSocket sessions; client.asr.transcribe() uploads files as-is over REST and the server decodes them.

Input audio (ASR)

FormatDescription
pcm1616-bit signed little-endian PCM
float3232-bit IEEE float PCM
twilioμ-law 8 kHz mono (Twilio Media Streams)
mulawμ-law 8 kHz mono
opusOgg-wrapped Opus

Input audio (STS)

FormatDescription
pcm1616-bit signed little-endian PCM
float3232-bit IEEE float PCM
twilioμ-law 8 kHz mono (Twilio Media Streams)

Sample rate defaults to 24,000 Hz and mono (1 channel). You can change both via the session update event.

Output audio (STS)

FormatDescription
pcm1616-bit signed little-endian PCM
float3232-bit IEEE float PCM
twilioμ-law 8 kHz mono (Twilio Media Streams)

The Python SDK’s translate() returns pcm16 @ 24 kHz.

Output audio (TTS)

FormatSample rateDescription
pcm_f328 / 16 / 24 kHz (default 24)32-bit IEEE float PCM (default)
pcm168 / 16 / 24 kHz (default 24)16-bit signed little-endian PCM
mulawfixed 8 kHzG.711 μ-law (Twilio Media Streams)
opusfixed 24 kHzSelf-contained Ogg/Opus stream

See Choosing an audio format for the SDK kwargs.

Chunk size

Each input_audio_buffer.append event can carry up to 1 MiB (approximately 10 seconds at the default sample rate). For most real-time applications, 20–40 ms chunks work well.