A local OpenAI-compatible transcription endpoint in 150 lines

openwhispr can use a self-hosted speech-to-text backend, as long as it speaks the OpenAI /v1/audio/transcriptions shape. Handy already has a good local model working well for me, so rather than adopting a second model just for meeting transcription, I wrapped the same engine in a small FastAPI server.

It’s a single server.py, about 150 lines. The parts worth explaining:

• The model loads once, at startup, into a persistent session — not per request. That alone saves roughly 1.8 seconds a request.
• Silero VAD gates the audio before it reaches the model. Both Whisper and Cohere Transcribe hallucinate text on silence and background noise if you don’t — things like a phantom “Thank you.” on an empty clip. The VAD pass returns an empty string instead of asking the transcription model to explain nothing.
• Speech segments get merged and chunked, not transcribed one at a time — adjacent segments under the model’s 400-second cap are transcribed together so pauses in natural speech don’t fragment the output, and anything longer gets hard-split.
• One transcription at a time on the GPU, via an async lock — this runs on the same card Handy uses, so the two are never fighting over VRAM.

the endpoint

POST /v1/audio/transcriptions, 127.0.0.1:8756 — the same shape OpenAI’s own transcription API uses.

The API surface is deliberately small: a health check, and one POST endpoint that accepts a multipart file plus the OpenAI-shaped optional fields — language, response_format, and model/prompt/temperature accepted and ignored, so anything already speaking the OpenAI transcription API works against it unmodified.

@app.post("/v1/audio/transcriptions")
async def transcriptions(
    file: UploadFile,
    language: str | None = Form(default=None),
    response_format: str = Form(default="json"),
    model: str | None = Form(default=None),  # accepted for API compat, ignored
    prompt: str | None = Form(default=None),  # ignored
    temperature: float | None = Form(default=None),  # ignored
):

Runs as a systemd --user service, so it comes up on login and openwhispr just points at http://127.0.0.1:8756/v1.