Skip to navigation

Python SDK — s2t

Speech-to-text from Python, REST file transcription + WebSocket streaming.

Install

pip install kotoba-sdk

Requires Python 3.10 or later. The package is imported as kotoba. For live-microphone examples, install with the optional mic extra (pulls in sounddevice, which needs PortAudio on the system):

pip install 'kotoba-sdk[mic]'

Configure

KotobaClient() reads its credentials and per-route URLs from environment variables. For ASR you need:

VariablePurpose
KOTOBA_API_KEYAPI key, sent as Authorization: Bearer … on REST and WebSocket requests. The SDK picks the auth scheme from the endpoint host automatically.
KOTOBA_ASR_REST_URLREST base URL including the version prefix, e.g. https://.../v1 — used by transcribe()
KOTOBA_ASR_URLWebSocket URL for live ASR, e.g. wss://.../asr — used by stream() / transcribe_stream()

Pass them in code instead if you’d rather not rely on the environment:

import kotoba
client = kotoba.KotobaClient(
api_key="kotoba-...",
url="https://.../v1", # REST base
asr_ws_url="wss://.../asr", # WS
)

KotobaClient(timeout=30.0, max_retries=3) also sets the per-request HTTP timeout used by transcribe(); raise it for long files or a cold deployment. Only GET requests are retried — an upload is sent exactly once.

See Authentication for the full handshake details.

Using the Kotoba apps on fal

The same SDK talks to the Kotoba apps published on fal. Use your fal key as KOTOBA_API_KEY and point each service at its fal app; the Authorization: Key … scheme is selected automatically for fal.run hosts:

export KOTOBA_API_KEY=<fal key>
export KOTOBA_ASR_REST_URL=https://fal.run/Kotoba-Technologies/kotoba-stt/v1
export KOTOBA_ASR_URL=wss://fal.run/Kotoba-Technologies/kotoba-streaming-stt/v1/realtime

An app that has been idle boots a GPU runner on the first request, which can take a few minutes; raise KotobaClient(timeout=...) or retry the first request.

Pick a transport

TransportMethodBest for
RESTclient.asr.transcribe(path, ...) — one POST, transcript in the responseFiles, short or long (long audio is transcribed in pieces)
REST job APIclient.asr.transcribe(path, batch=True, ...) — submit a job, poll until doneAsynchronous jobs on private endpoints (not available on fal)
WebSocketclient.asr.stream(...) / transcribe_stream(...)Live mic, latency-sensitive pipelines, live speaker labels

One-shot transcription (transcribe)

transcribe() uploads the file in a single synchronous POST /speech-to-text relative to KOTOBA_ASR_REST_URL and returns the transcript in the response — no job, no polling:

import kotoba
client = kotoba.KotobaClient()
result = client.asr.transcribe("clip.mp3", language="ja")
print(result.text)

Ask for per-segment timestamps with with_timestamps=True:

result = client.asr.transcribe("clip.mp3", language="ja", with_timestamps=True)
print(result.text)
for seg in result.segments or []:
print(f"{seg.start:6.2f} - {seg.end:6.2f} {seg.text}")

Ask for speaker attribution with diarization=True. result.utterances then lists who said what, in reading order:

result = client.asr.transcribe(
"meeting.wav", language="ja", diarization=True, with_timestamps=True
)
for u in result.utterances or []:
print(f"{u.start:6.2f} {u.speaker_id}: {u.text}")

Options:

  • language — "ja", "en", "ko", or "zh"; omit to use the deployment’s configured language.
  • with_timestamps=True — populates result.segments with Segment(text, start, end, speaker_id) entries, or None when the server returned no timed words. A deployment with timestamps disabled rejects the request with ProtocolError (HTTP 400).
  • diarization=True — populates result.utterances with Utterance(speaker_id, text, start, end) entries: runs of consecutive words by speaker (speaker_1, speaker_2, …). start / end are set only together with with_timestamps=True; a run the model did not attribute has speaker_id=None. Each timed segment also carries its speaker_id. Only deployments whose model carries speaker tokens support it (on fal, kotoba-stt does); a deployment without them rejects the request with ProtocolError (HTTP 400) when it declares so, or APIError (HTTP 503) on older deployments. Requires kotoba-sdk >= 0.2.3.
  • keywords=[...] — terms such as names or product words whose spelling the transcript should prefer. The server biases decoding toward them; on fal kotoba-stt a request takes up to 1,000 terms of up to 64 characters each. Blank terms are dropped. A longer list or a longer term comes back as ProtocolError (HTTP 400, invalid_parameters); the limits are the deployment’s, so the SDK does not check them.
  • style_preference={"human_name": "kana"} — transcribe personal names in katakana. Also accepts kotoba.models.TranscriptionStylePreference(human_name="kana").
  • file_format="pcm_s16le_16" — upload raw 16 kHz mono int16 PCM. The default "other" is any encoded audio the server can decode (WAV / FLAC / OGG / MP3 / …); the SDK uploads the file as-is.
  • result.metadata carries language_code, language_probability, and audio_duration_secs. result.job_id is None in this mode.

An option the deployment has not enabled comes back as a ProtocolError (HTTP 400).

Long audio

One transcribe() call also takes a long file. The server cuts audio longer than its max_audio_seconds limit (120 s by default) into pieces of that length, transcribes them, and joins the text. Three limits apply:

  • Speaker attribution. diarization=True takes audio up to max_audio_seconds only; longer audio comes back as a ProtocolError (HTTP 400).
  • Upload size. A file over the server’s upload limit (500 MiB by default) comes back as a ProtocolError (HTTP 413). Compress long or multichannel WAV first, for example to FLAC or MP3.
  • Time. The whole transcription runs inside one HTTP request, under KotobaClient(timeout=...) (30 s by default). Raise the timeout to suit the length of the file.
client = kotoba.KotobaClient(timeout=600.0)
result = client.asr.transcribe("meeting.mp3", language="ja")
print(result.metadata["audio_duration_secs"], result.text)

Private endpoints also offer the job API below, which runs the transcription as an asynchronous job.

Batch job API (transcribe(batch=True))

The job API (POST /transcription_jobs) is available only on private endpoints — dedicated streaming_stt deployments that Kotoba provisions for specific customers. It is not available on fal: use transcribe() for files and stream() / transcribe_stream() for live audio there.

batch=True submits the file as a job, polls GET /transcription_jobs/{id} with exponential backoff, and returns the final transcript with result.job_id set. language defaults to "ja" in this mode:

client = kotoba.KotobaClient()
result = client.asr.transcribe(
"clip.mp3", batch=True, language="ja", with_timestamps=True
)
print(result.text)
for seg in result.segments or []:
print(f"{seg.start:6.2f} - {seg.end:6.2f} {seg.text}")

Polling knobs:

result = client.asr.transcribe(
"clip.mp3",
batch=True,
language="ja",
with_timestamps=True,
poll_interval=1.0, # initial GET polling interval (s)
poll_backoff=1.5, # multiplied each poll
max_poll_interval=10.0,
timeout=1200.0, # overall deadline for job completion (s)
)

TranscriptionError is raised on a server-reported failure, TimeoutError if the deadline elapses. Passing a knob that belongs to the other mode (poll_* / timeout without batch=True, or keywords / file_format with it) raises ValueError before any request is sent.

Low-level: submit_job / get_job

If you’d rather poll yourself — for example to drive a job queue or surface progress in a UI — call the two REST endpoints directly:

import time
job = client.asr.submit_job("clip.mp3", language="ja")
print("submitted:", job.job_id)
while True:
status = client.asr.get_job(job.job_id)
if status.state == "done":
print(status.transcription)
break
if status.state == "error":
raise RuntimeError(status.error_message)
time.sleep(2)

JobStatus.state is one of processing | done | error. For done, read status.transcription (and status.segments when the job was submitted with with_timestamps=True); for error, read status.error_message.

WebSocket streaming (stream / transcribe_stream)

For the realtime / mic case — where transcript deltas should surface while audio is still being captured — pass a generator of PCM16 LE mono bytes to transcribe_stream(...). The feeder and receiver run concurrently, so the first delta can fire before your source is exhausted:

for delta in client.asr.transcribe_stream(mic_chunks(), language="ja"):
print(delta, end="", flush=True)

mic_chunks() is any iterable of PCM16 little-endian mono bytes — it is not provided by the SDK. For a runnable end-to-end example see examples/asr_stream_async.py in the SDK repo.

Optional knobs on both stream(...) and transcribe_stream(...):

  • language — "ja" (the only language live transcription supports today)
  • sample_rate — defaults to 24 kHz; the session resamples internally
  • style_preference — {"human_name": "kana"} to transcribe personal names in katakana
  • turn_detection — how the server ends a turn; see Turn detection below. Omit it to use the deployment’s default.
  • timestamps=True — every transcript event carries start_ms / end_ms in event.metadata, in milliseconds from the start of the session’s audio: a delta’s own span on partials, the whole turn’s span on finals. audio_processed_ms in the same metadata is the safe resume point after a disconnect: audio before it is fully transcribed, so a client resends from there. Requires kotoba-sdk >= 0.2.3 for stream() and kotoba-sdk >= 0.3.1 for transcribe_stream(). transcribe_stream() yields plain strings unless diarization=True is set, so the spans are readable there only on a diarized session.
  • keywords — terms such as names or product words whose spelling the transcript should prefer, applied to the whole session. On fal kotoba-streaming-stt a session takes up to 1,000 terms of up to 64 characters each: the first 50 also go into the model’s prompt, and every term biases decoding. Blank terms are dropped. A longer list or a longer term fails the session with ProtocolError (invalid_parameters). Keywords combine with diarization=True.
  • diarization=True — attribute the transcript to speakers; see Speaker diarization below. Requires kotoba-sdk >= 0.3.2.

Turn detection

With turn detection on, the server ends a turn and emits a completed event that carries the full transcript of that turn. Every delta and the completion of one turn share the same server-generated item_id. turn_detection selects how a turn ends:

ValueA turn ends…
kotoba.SemanticVAD(eagerness="medium")shortly after the speaker finishes a sentence; the server waits longer in the middle of one
kotoba.ServerVAD(silence_duration_ms=800)after a fixed run of blank model outputs (it counts model outputs, not wall-clock silence)
Falsenever: one uninterrupted delta stream and no completed events

SemanticVAD reads the model’s text output, not the audio energy. Its eagerness is "low", "medium", "high" or "auto" (the default, equal to "medium"): "high" ends a turn soonest, "low" waits longest for the speaker to continue.

When you omit turn_detection, the SDK sends none and the deployment’s default applies:

  • ServerVAD(silence_duration_ms=800), unless the deployment sets another default.
  • On fal kotoba-streaming-stt, SemanticVAD (eagerness medium).
  • On a session with diarization=True, always SemanticVAD (eagerness medium). Such a session rejects turn_detection=False.

Pass a value to get the same behavior on every deployment:

for delta in client.asr.transcribe_stream(
mic_chunks(), language="ja", turn_detection=kotoba.SemanticVAD(eagerness="high")
):
print(delta, end="", flush=True)

ServerVAD requires kotoba-sdk >= 0.2.1, SemanticVAD kotoba-sdk >= 0.3.0.

Driving the session yourself

stream(...) gives you the session object so you can feed audio and read every event type, including completed turns:

with client.asr.stream(language="ja") as session:
for chunk in pcm16_chunks():
session.send_audio(chunk)
session.commit()
for event in session:
if event.type == "partial_transcript":
print(event.text, end="", flush=True)
elif event.type == "final_transcript":
print(f"\n[turn {event.metadata['item_id']}] {event.text}")
elif event.type == "done":
break
event.typeMeaning
session_readyThe server accepted the session configuration
partial_transcriptA transcript delta in event.text; with diarization, event.speaker_id is the provisional speaker
final_transcriptFull transcript of one completed turn (turn detection on); event.metadata["item_id"] identifies the turn; with diarization, event.speaker_id is the settled speaker
committedThe server acknowledged commit()
doneThe session has finished; stop iterating

transcribe_stream() is the same loop with the feeder running concurrently, yielding only the partial_transcript text (or, with diarization=True, the partial_transcript and final_transcript events).

Speaker diarization

diarization=True on stream() or transcribe_stream() attributes the live transcript to speakers (speaker_1 … speaker_8, in order of first appearance). On fal, kotoba-streaming-stt supports it. Requires kotoba-sdk >= 0.3.2.

With diarization=True, transcribe_stream() yields StreamEvent objects instead of strings: the partial_transcript and final_transcript events. Each event carries event.speaker_id, and event.metadata["item_id"] names the transcript item it belongs to.

  • Partials are provisional, in text and in speaker. A partial is sent as soon as the text is ready, and its speaker_id is the speaker who talks at that moment.
  • A final is the settled result. Each final_transcript is one speaker’s turn, also inside overlapping speech. It replaces the partials of its item_id: the text and the speaker can both change. Do not append a final to its partials.
  • An empty final retires its item. A final with event.text == "" means that the provisional item is gone: remove it.
  • A final can arrive late or alone. The server sends a final after the speaker attribution of the turn’s audio has settled. The first partials of the next turn can arrive before it, and a final can arrive without any partial.
  • A whitespace-only partial has no speaker. Its speaker_id and item_id are None; it belongs to the previous item.

So keep the transcript keyed by item_id:

items: dict[str, tuple[str | None, str]] = {} # item_id -> (speaker, text)
last = None # item_id of the last non-blank partial
for event in client.asr.transcribe_stream(
mic_chunks(), language="ja", diarization=True, timestamps=True
):
item_id = event.metadata["item_id"]
if event.type == "partial_transcript":
if not event.text.strip():
# A space between words: it joins the previous partial's item.
if last in items:
items[last] = (items[last][0], items[last][1] + event.text)
continue
last = item_id
items[item_id] = (event.speaker_id, items.get(item_id, (None, ""))[1] + event.text)
elif event.text: # final_transcript: the settled turn replaces its partials
items[item_id] = (event.speaker_id, event.text)
span = f"{event.metadata.get('start_ms')}-{event.metadata.get('end_ms')} ms"
print(f"{event.speaker_id} {span}: {event.text.strip()}")
else: # an empty final retires a provisional item
items.pop(item_id, None)
  • Order. Keep every settled item. The time spans of different speakers can overlap, and finals can arrive out of start-time order; sort by start_ms when you need a timeline. An empty final carries no start_ms / end_ms.
  • Long sessions. The dict above keeps every settled item (about 1,600 per hour of conversation), so drop the settled items you no longer display.
  • Speaker ids. The ids are stable within one connection only: a new connection starts again at speaker_1. Speakers with very similar voices can share an id.
  • Turns. Unless you pass turn_detection, a diarized session uses SemanticVAD (eagerness medium), on every deployment. turn_detection=False is rejected, because only the finals carry the settled speaker.
  • Keywords and timestamps. keywords and timestamps=True combine with diarization.

The session fails with a ProtocolError when the deployment cannot serve the request. error.code and error.param name the cause:

codeparamCauseWhat to do
invalid_parameterssession.diarizationThe deployment does not diarize streaming sessionsConnect without diarization=True
too_many_concurrent_requestssession.diarizationThe deployment already diarizes its maximum number of sessionsRetry later, or connect without diarization
invalid_parameterssession.turn_detectionturn_detection=False together with diarization=TrueOmit turn_detection, or pass a VAD
try:
events = list(client.asr.transcribe_stream(chunks, language="ja", diarization=True))
except kotoba.ProtocolError as error:
if error.param != "session.diarization":
raise
texts = list(client.asr.transcribe_stream(chunks, language="ja"))

Async

Every ASR entry point has an async equivalent via AsyncKotobaClient:

import asyncio
import kotoba
async def main() -> None:
async with kotoba.AsyncKotobaClient() as client:
result = await client.asr.transcribe("clip.mp3", language="ja")
print(result.text)
asyncio.run(main())

The sync wrapper runs an asyncio loop on a background daemon thread — underlying transport is identical, only the call style differs.

What’s in the box (ASR)

SymbolWhat
client.asr.transcribe(path, ...)REST, one POST /speech-to-text; transcript in the response
client.asr.transcribe(path, batch=True, ...)Job API: submit, poll, return (private endpoints only, not on fal)
client.asr.submit_job(...) / get_job(id)Low-level job API helpers
client.asr.stream(...)WebSocket session you drive manually
client.asr.transcribe_stream(iter, ...)Generator in → transcript deltas out
kotoba.ServerVAD(silence_duration_ms=...)Turn-detection config for stream() / transcribe_stream()
kotoba.SemanticVAD(eagerness=...)Turn detection at sentence boundaries (the default with diarization)
StreamEvent.speaker_idSpeaker of a streaming event, from stream(..., diarization=True)
ProtocolError.code / ProtocolError.paramThe server’s error code and the setting it refers to
kotoba.Utterance / result.utterancesWho said what, from transcribe(..., diarization=True)

See the API reference for the on-the-wire protocol that this SDK wraps: Live (WebSocket) · File (REST) · Job API (REST).