Jul 29, 2026 · 5 min read

Whisper + Home Assistant — Local Speech Recognition Setup (2026)

Cloud-based voice assistants hear everything you say and send it to someone else’s server. If that bothers you — and it should — there’s a better way. OpenAI’s Whisper runs entirely on your own hardware, turns speech into text with impressive accuracy, and never phones home. This guide walks you through setting it up with Home Assistant so your smart home listens locally.

What Is Whisper?

Whisper is an open-source automatic speech recognition (ASR) model released by OpenAI. Trained on 680,000 hours of multilingual audio, it handles English and dozens of other languages with accuracy that rivals commercial cloud APIs. The model weights are freely available, which means you can run it on your own machine — a desktop, a NAS, or even a Raspberry Pi with the right model size.

Unlike cloud STT services, Whisper:

Processes audio entirely on-device
Requires no API key, subscription, or internet connection
Supports over 90 languages out of the box
Can be fine-tuned for domain-specific vocabulary

Why Local Speech-to-Text Matters

Every time you talk to Alexa or Google Assistant, your audio gets uploaded, processed, and stored on remote servers. Local STT eliminates that entirely.

Privacy. Your voice data stays on your network. No recordings sitting on a corporate server. No humans reviewing transcripts for “quality improvement.”

Reliability. Cloud outages don’t affect your smart home. Your voice pipeline works during internet downtime, which is exactly when you need home automation the most.

No subscription fees. Services like Google Cloud STT or Amazon Transcribe charge per minute of audio. Whisper costs you nothing beyond the electricity to run it.

If you’re already running Ollama for local AI, adding Whisper is a natural next step toward a fully local smart home stack.

How It Works: The Wyoming Protocol

Home Assistant uses the Wyoming protocol as a standard interface for voice services. Instead of baking STT/TTS directly into the core, Home Assistant communicates with external services over Wyoming. This means you can swap Whisper in (or out) without touching your HA configuration.

The architecture looks like this:

Microphone → Home Assistant → Wyoming Whisper (STT) → Intent Recognition → Action
                            → Wyoming Piper (TTS)  → Speaker

Each component runs as a separate service, typically in Docker containers. Home Assistant discovers them automatically.

Choosing a Whisper Model

Whisper comes in several sizes. Picking the right one depends on your hardware and how much accuracy you need for home automation commands.

Model	Parameters	English Accuracy	Relative Speed	RAM Usage
`tiny`	39M	Good	~10x realtime	~1 GB
`base`	74M	Better	~7x realtime	~1 GB
`small`	244M	Very Good	~4x realtime	~2 GB
`medium`	769M	Excellent	~2x realtime	~5 GB

For smart home commands — short phrases like “turn off the kitchen lights” — tiny or base work surprisingly well. They’re fast and light enough for a Raspberry Pi 4 or an older NUC. If you need better accuracy with longer sentences or non-English languages, step up to small or medium.

Check our hardware guide for specific recommendations on what to run these models on.

Tip: Start with base. If recognition errors bother you, move to small. Most people never need medium for home automation.

Docker Setup

The easiest way to run Whisper for Home Assistant is through the official wyoming-whisper Docker image. Here’s a complete docker-compose.yml that includes both Whisper (STT) and Piper (TTS) for a full local voice pipeline:

version: "3.8"

services:
  wyoming-whisper:
    image: rhasspy/wyoming-whisper
    container_name: wyoming-whisper
    restart: unless-stopped
    ports:
      - "10300:10300"
    volumes:
      - whisper-data:/data
    command: --model base --language en
    environment:
      - TZ=America/New_York

  wyoming-piper:
    image: rhasspy/wyoming-piper
    container_name: wyoming-piper
    restart: unless-stopped
    ports:
      - "10200:10200"
    volumes:
      - piper-data:/data
    command: --voice en_US-lessac-medium
    environment:
      - TZ=America/New_York

volumes:
  whisper-data:
  piper-data:

Start both services:

docker compose up -d

The first run downloads the model files, which takes a minute or two depending on your connection. After that, everything runs offline.

To switch models, change the --model flag to tiny, small, or medium and restart the container. To use a different language, change --language to the appropriate code (e.g., de for German, fr for French).

Integrating with Home Assistant

Once the containers are running, Home Assistant can discover them automatically or you can add them manually.

Automatic Discovery

If Home Assistant and the Wyoming containers are on the same network, HA should discover them within a few minutes. Check Settings → Devices & Services for new Wyoming entries.

Manual Setup

Go to Settings → Devices & Services → Add Integration
Search for Wyoming Protocol
Enter the host IP and port 10300 for Whisper
Repeat with port 10200 for Piper

Configure the Voice Pipeline

Navigate to Settings → Voice Assistants
Create a new assistant or edit the default one
Set Speech-to-Text to your Wyoming Whisper instance
Set Text-to-Speech to your Wyoming Piper instance
Choose a conversation agent (the built-in one works, or use Ollama for smarter responses)

Testing the Pipeline

Open Developer Tools → Assist in Home Assistant and click the microphone icon. Say a command like “What’s the temperature in the living room?” You should see:

The audio being sent to Whisper for transcription
The transcribed text appearing in the Assist panel
Home Assistant processing the intent and responding through Piper

If transcription seems slow, check your hardware. On a Raspberry Pi 4, tiny processes a 3-second clip in under a second. base takes about 1.5 seconds. If you’re running on anything with an x86 CPU and 4+ GB of RAM, even small should feel instant.

Troubleshooting

No audio detected: Make sure your microphone is configured in HA and the correct audio input is selected in the voice pipeline settings.
Slow transcription: Drop down a model size or check that no other heavy processes are competing for CPU.
Wrong language: Verify the --language flag matches what you’re speaking. Whisper can auto-detect, but explicitly setting the language is faster and more reliable.
Container won’t start: Check logs with docker logs wyoming-whisper. The most common issue is insufficient RAM for the chosen model.

Pairing with Piper for Full Local Voice

With Whisper handling STT and Piper handling TTS, you have a complete voice loop that never leaves your network. Piper is a fast, lightweight neural TTS engine that produces natural-sounding speech. The en_US-lessac-medium voice in the compose file above is a solid default — clear and not robotic.

For an even more capable setup, you can wire the conversation agent to a local LLM through Ollama. That gives you natural language understanding without cloud dependencies. We cover that in detail in our local voice assistant guide.

What’s Next

You now have a fully local speech-to-text pipeline for Home Assistant. No cloud, no subscriptions, no audio leaving your network. From here you can:

Add a local LLM for smarter voice control
Build a complete local voice assistant with Whisper + Ollama
Explore hardware options for running AI at home

The local AI ecosystem for smart homes has matured significantly. Whisper, Piper, and the Wyoming protocol make it straightforward to cut the cloud out of your voice pipeline entirely. Your home, your hardware, your data.