Python SDK — s2t
Speech-to-text from Python, REST file transcription + WebSocket streaming.
Install
Requires Python 3.10 or later. The package is imported as kotoba.
For live-microphone examples, install with the optional mic extra
(pulls in sounddevice, which needs PortAudio on the system):
Configure
KotobaClient() reads its credentials and per-route URLs from environment
variables. For ASR you need:
Pass them in code instead if you’d rather not rely on the environment:
KotobaClient(timeout=30.0, max_retries=3) also sets the per-request HTTP
timeout used by transcribe(); raise it for long files or a cold deployment.
Only GET requests are retried — an upload is sent exactly once.
See Authentication for the full handshake details.
Using the Kotoba apps on fal
The same SDK talks to the Kotoba apps published on fal. Use
your fal key as KOTOBA_API_KEY and point each service at its fal app; the
Authorization: Key … scheme is selected automatically for fal.run hosts:
An app that has been idle boots a GPU runner on the first request, which can
take a few minutes; raise KotobaClient(timeout=...) or retry the first
request.
Pick a transport
One-shot transcription (transcribe)
transcribe() uploads the file in a single synchronous POST /speech-to-text
relative to KOTOBA_ASR_REST_URL and returns the transcript in the response —
no job, no polling:
Ask for per-segment timestamps with with_timestamps=True:
Ask for speaker attribution with diarization=True. result.utterances
then lists who said what, in reading order:
Options:
language—"ja","en","ko", or"zh"; omit to use the deployment’s configured language.with_timestamps=True— populatesresult.segmentswithSegment(text, start, end, speaker_id)entries, orNonewhen the server returned no timed words. A deployment with timestamps disabled rejects the request withProtocolError(HTTP 400).diarization=True— populatesresult.utteranceswithUtterance(speaker_id, text, start, end)entries: runs of consecutive words by speaker (speaker_1,speaker_2, …).start/endare set only together withwith_timestamps=True; a run the model did not attribute hasspeaker_id=None. Each timed segment also carries itsspeaker_id. Only deployments whose model carries speaker tokens support it (on fal,kotoba-sttdoes); a deployment without them rejects the request withProtocolError(HTTP 400) when it declares so, orAPIError(HTTP 503) on older deployments. Requireskotoba-sdk >= 0.2.3.keywords=[...]— terms such as names or product words whose spelling the transcript should prefer. The server biases decoding toward them; on falkotoba-stta request takes up to 1,000 terms of up to 64 characters each. Blank terms are dropped. A longer list or a longer term comes back asProtocolError(HTTP 400,invalid_parameters); the limits are the deployment’s, so the SDK does not check them.style_preference={"human_name": "kana"}— transcribe personal names in katakana. Also acceptskotoba.models.TranscriptionStylePreference(human_name="kana").file_format="pcm_s16le_16"— upload raw 16 kHz mono int16 PCM. The default"other"is any encoded audio the server can decode (WAV / FLAC / OGG / MP3 / …); the SDK uploads the file as-is.result.metadatacarrieslanguage_code,language_probability, andaudio_duration_secs.result.job_idisNonein this mode.
An option the deployment has not enabled comes back as a ProtocolError
(HTTP 400).
Long audio
One transcribe() call also takes a long file. The server cuts audio longer
than its max_audio_seconds limit (120 s by default) into pieces of that
length, transcribes them, and joins the text. Three limits apply:
- Speaker attribution.
diarization=Truetakes audio up tomax_audio_secondsonly; longer audio comes back as aProtocolError(HTTP 400). - Upload size. A file over the server’s upload limit (500 MiB by default)
comes back as a
ProtocolError(HTTP 413). Compress long or multichannel WAV first, for example to FLAC or MP3. - Time. The whole transcription runs inside one HTTP request, under
KotobaClient(timeout=...)(30 s by default). Raise the timeout to suit the length of the file.
Private endpoints also offer the job API below, which runs the transcription as an asynchronous job.
Batch job API (transcribe(batch=True))
The job API (POST /transcription_jobs) is available only on private
endpoints — dedicated streaming_stt deployments that Kotoba provisions for
specific customers. It is not available on fal: use transcribe() for files
and stream() / transcribe_stream() for live audio there.
batch=True submits the file as a job, polls GET /transcription_jobs/{id}
with exponential backoff, and returns the final transcript with
result.job_id set. language defaults to "ja" in this mode:
Polling knobs:
TranscriptionError is raised on a server-reported failure, TimeoutError
if the deadline elapses. Passing a knob that belongs to the other mode
(poll_* / timeout without batch=True, or keywords / file_format with
it) raises ValueError before any request is sent.
Low-level: submit_job / get_job
If you’d rather poll yourself — for example to drive a job queue or surface progress in a UI — call the two REST endpoints directly:
JobStatus.state is one of processing | done | error. For done, read
status.transcription (and status.segments when the job was submitted with
with_timestamps=True); for error, read status.error_message.
WebSocket streaming (stream / transcribe_stream)
For the realtime / mic case — where transcript deltas should surface
while audio is still being captured — pass a generator of PCM16 LE
mono bytes to transcribe_stream(...). The feeder and receiver run
concurrently, so the first delta can fire before your source is
exhausted:
mic_chunks() is any iterable of PCM16 little-endian mono bytes — it is
not provided by the SDK. For a runnable end-to-end example see
examples/asr_stream_async.py in the SDK repo.
Optional knobs on both stream(...) and transcribe_stream(...):
language—"ja"(the only language live transcription supports today)sample_rate— defaults to 24 kHz; the session resamples internallystyle_preference—{"human_name": "kana"}to transcribe personal names in katakanaturn_detection— how the server ends a turn; see Turn detection below. Omit it to use the deployment’s default.timestamps=True— every transcript event carriesstart_ms/end_msinevent.metadata, in milliseconds from the start of the session’s audio: a delta’s own span on partials, the whole turn’s span on finals.audio_processed_msin the same metadata is the safe resume point after a disconnect: audio before it is fully transcribed, so a client resends from there. Requireskotoba-sdk >= 0.2.3forstream()andkotoba-sdk >= 0.3.1fortranscribe_stream().transcribe_stream()yields plain strings unlessdiarization=Trueis set, so the spans are readable there only on a diarized session.keywords— terms such as names or product words whose spelling the transcript should prefer, applied to the whole session. On falkotoba-streaming-stta session takes up to 1,000 terms of up to 64 characters each: the first 50 also go into the model’s prompt, and every term biases decoding. Blank terms are dropped. A longer list or a longer term fails the session withProtocolError(invalid_parameters). Keywords combine withdiarization=True.diarization=True— attribute the transcript to speakers; see Speaker diarization below. Requireskotoba-sdk >= 0.3.2.
Turn detection
With turn detection on, the server ends a turn and emits a completed event
that carries the full transcript of that turn. Every delta and the completion
of one turn share the same server-generated item_id. turn_detection
selects how a turn ends:
SemanticVAD reads the model’s text output, not the audio energy. Its
eagerness is "low", "medium", "high" or "auto" (the default, equal
to "medium"): "high" ends a turn soonest, "low" waits longest for the
speaker to continue.
When you omit turn_detection, the SDK sends none and the deployment’s
default applies:
ServerVAD(silence_duration_ms=800), unless the deployment sets another default.- On fal
kotoba-streaming-stt,SemanticVAD(eagerness medium). - On a session with
diarization=True, alwaysSemanticVAD(eagerness medium). Such a session rejectsturn_detection=False.
Pass a value to get the same behavior on every deployment:
ServerVAD requires kotoba-sdk >= 0.2.1, SemanticVAD
kotoba-sdk >= 0.3.0.
Driving the session yourself
stream(...) gives you the session object so you can feed audio and read
every event type, including completed turns:
transcribe_stream() is the same loop with the feeder running concurrently,
yielding only the partial_transcript text (or, with diarization=True, the
partial_transcript and final_transcript events).
Speaker diarization
diarization=True on stream() or transcribe_stream() attributes the live
transcript to speakers (speaker_1 … speaker_8, in order of first
appearance). On fal, kotoba-streaming-stt supports it. Requires
kotoba-sdk >= 0.3.2.
With diarization=True, transcribe_stream() yields StreamEvent objects
instead of strings: the partial_transcript and final_transcript events.
Each event carries event.speaker_id, and event.metadata["item_id"] names
the transcript item it belongs to.
- Partials are provisional, in text and in speaker. A partial is sent as
soon as the text is ready, and its
speaker_idis the speaker who talks at that moment. - A final is the settled result. Each
final_transcriptis one speaker’s turn, also inside overlapping speech. It replaces the partials of itsitem_id: the text and the speaker can both change. Do not append a final to its partials. - An empty final retires its item. A final with
event.text == ""means that the provisional item is gone: remove it. - A final can arrive late or alone. The server sends a final after the speaker attribution of the turn’s audio has settled. The first partials of the next turn can arrive before it, and a final can arrive without any partial.
- A whitespace-only partial has no speaker. Its
speaker_idanditem_idareNone; it belongs to the previous item.
So keep the transcript keyed by item_id:
- Order. Keep every settled item. The time spans of different speakers
can overlap, and finals can arrive out of start-time order; sort by
start_mswhen you need a timeline. An empty final carries nostart_ms/end_ms. - Long sessions. The dict above keeps every settled item (about 1,600 per hour of conversation), so drop the settled items you no longer display.
- Speaker ids. The ids are stable within one connection only: a new
connection starts again at
speaker_1. Speakers with very similar voices can share an id. - Turns. Unless you pass
turn_detection, a diarized session usesSemanticVAD(eagerness medium), on every deployment.turn_detection=Falseis rejected, because only the finals carry the settled speaker. - Keywords and timestamps.
keywordsandtimestamps=Truecombine with diarization.
The session fails with a ProtocolError when the deployment cannot serve the
request. error.code and error.param name the cause:
Async
Every ASR entry point has an async equivalent via AsyncKotobaClient:
The sync wrapper runs an asyncio loop on a background daemon thread —
underlying transport is identical, only the call style differs.
What’s in the box (ASR)
See the API reference for the on-the-wire protocol that this SDK wraps: Live (WebSocket) · File (REST) · Job API (REST).