Complete local voice server

Home Assistant Local Voice Server Hardware Guide

A full local voice server combines wake-word processing, speech-to-text, Home Assistant intent handling and text-to-speech. The best hardware depends on whether you use focused Speech-to-Phrase or open-ended Whisper, plus how many rooms can speak at once.

See live hardware Size my workload

Quick answer

What this voice workload needs

For one or two rooms using Speech-to-Phrase, modest hardware can be excellent. For multiple Whisper users, start at N100-class or better and increase CPU/GPU headroom as language/model size and concurrency rise.

Live Amazon hardware

Current products that fit this decision

These listings come from this sprint's dedicated Amazon catalogue. Barebones PCs, laptops, GPU enclosures, accessories, ambiguous voice-device listings and incomplete systems are excluded by the normalizer.

Checking the dedicated Home Assistant Voice & Local AI catalogue…

Buying decision

Match compute to speech and LLM load

Separate room endpoints from server compute. Buy good microphones/satellites for capture quality, then size the central server around transcription and optional LLM latency.

Interactive decision tool

Local Voice Server Planner

Size the voice and local-AI workload before comparing products. Results are planning tiers, not benchmark guarantees. Verify current Home Assistant voice support and Ollama hardware support for the exact system.

Compatibility checklist

Four checks before you purchase

Choose focused or open-ended STT first

Speech-to-Phrase and Whisper have very different compute needs.

Treat LLM memory separately

Model size, quantization, context and concurrency drive RAM/VRAM.

Keep Piper in perspective

Text-to-speech is usually lighter than Whisper or Ollama.

Measure end-to-end latency

Microphone, network, STT, LLM and TTS all affect response time.

Define the central server boundary first

A local voice system can place speech services on the Home Assistant box or distribute them through Wyoming to another machine. Decide what must remain available when an AI server is rebooted or powered off. Keeping core Home Assistant intent handling and Piper on the automation host while moving Whisper to a stronger speech server can preserve basic local control even during maintenance.

One room and six rooms are not six times the compute

Satellite count mainly affects endpoint cost until users speak concurrently. A six-room home with sequential requests can behave like a one-room system from the server’s perspective. Ask how many people may trigger Assist at the same moment. That concurrency estimate is a better input for CPU and GPU sizing than simply multiplying hardware by the number of microphones.

Speech-to-Phrase is ideal for an appliance-like server

If the objective is reliable local home control with predictable phrases, Speech-to-Phrase lets the server stay small, cool and quiet. It can run on hardware that would be unsatisfying for open-ended Whisper. This is valuable for users who prefer deterministic behavior and low maintenance. Open-ended features can later be added as a separate service instead of forcing them into the original automation appliance.

Whisper moves the baseline to stronger x86 hardware

When the local server must understand arbitrary speech, Home Assistant’s N100-equivalent recommendation for Whisper Base becomes the starting point. N150 or N305 offers more margin; Core/Ryzen systems are safer when larger models or several simultaneous utterances are expected. Choose based on the tested language and transcription target instead of assuming Base works equally well for every household.

Wyoming makes service separation a supported pattern

Home Assistant’s Wyoming integration exists specifically to connect external speech-to-text, text-to-speech and wake-word services. That means a dedicated Whisper machine is not a workaround—it fits the modular voice architecture. Network separation also lets you upgrade speech compute independently while leaving Home Assistant OS, radios and automation history untouched.

A local LLM is optional, not a prerequisite for natural control

Assist can operate without an LLM, and many home-control requests are better served by deterministic intents. Add Ollama when conversational flexibility, richer questions or fallback behavior has a clear use case. This keeps the voice server responsive and reduces memory requirements. If Ollama is enabled, expose only the entities it actually needs and treat control as experimental as Home Assistant currently documents.

Voice endpoint quality can dominate recognition success

A central server only receives the audio captured by each room device. Echo, distance, television noise and poor microphone arrays can undermine even a large Whisper model. Test satellites in their final locations before attributing errors to CPU speed. A modest server fed clean audio often outperforms an expensive one fed unreliable audio, especially for short device and room names.

Use wired networking for the server where possible

Interactive voice does not need huge bandwidth, but it benefits from consistent latency. Connect the central Home Assistant or speech server by Ethernet when practical, then optimize wireless coverage for satellites. If Ollama runs on another local host, keep that path on the same dependable LAN. The architecture should not send private local voice traffic through unnecessary external hops.

Design failure behavior, not just peak speed

Ask what happens if Whisper stops, the GPU driver crashes or the Ollama server is unavailable. A resilient design can fall back to focused local commands or normal Home Assistant controls instead of taking voice offline entirely. Separating optional AI from core automation reduces the blast radius of experimentation and makes updates less stressful.

Memory should reflect all co-hosted services

A voice-only x86 server can be comfortable with 16GB, but the number changes if Home Assistant, databases, cameras, media servers or Ollama share the machine. Allocate memory for the full peak, not average idle usage. Thirty-two gigabytes is a sensible consolidation tier when several services coexist; larger local models can justify 64GB or a separate AI system.

Plan the physical placement of endpoints

Voice satellites belong where people speak, not where the server rack is convenient. Power, Wi-Fi signal, acoustic reflections and speaker audibility should all influence endpoint placement. The central compute can live elsewhere as long as the network is stable. This separation helps preserve good room audio without compromising server cooling or cable management.

A staged multi-room rollout is safer

Deploy one or two representative rooms first, measure transcription and response time, then expand. This reveals language, microphone and network issues before purchasing many endpoints. If the server remains mostly idle, keep it. If queues appear, upgrade compute before the full rollout. A staged project turns hardware sizing from prediction into evidence and prevents expensive overbuilding.

Questions people ask

Home Assistant Local Voice Server Hardware Guide questions

Can Home Assistant voice run fully locally?

Yes. Home Assistant documents fully local pipelines using local speech-to-text and Piper text-to-speech, with no requirement to send spoken commands to a cloud service.

What is the lightest local speech-to-text option?

Speech-to-Phrase is designed for focused home-control commands and can run very quickly on low-powered Home Assistant hardware.

What CPU does Home Assistant recommend for Whisper?

For fully local Voice Preview Edition use with Whisper Base, Home Assistant currently recommends at least an Intel N100 or equivalent processor.

Does Piper need a GPU?

Usually no. Piper is optimized for modest hardware and is much lighter than open-ended Whisper or a local LLM.

Does Home Assistant run Ollama itself?

The Home Assistant Ollama integration connects to an external Ollama server, which may be on the same machine or another computer reachable over the network.

How much context does Home Assistant use with Ollama?

Home Assistant currently defaults its Ollama integration to an 8K context window. Increasing context can improve capacity for larger homes but raises memory use.

How many entities should I expose to a local LLM?

Home Assistant currently recommends exposing fewer than 25 entities when experimenting with local LLM control, especially because smaller models can make more mistakes.

Is NVIDIA required for Ollama?

No. Ollama documents NVIDIA, supported AMD/ROCm, Apple Metal and experimental Vulkan paths. Compatibility depends on the exact hardware and operating system.

Should I put Ollama on a separate server?

Separate it when GPU power, memory pressure, experimentation or noise would compromise the always-on Home Assistant host. Small models can stay consolidated when simplicity matters more.

What should I measure after installation?

Measure end-to-end voice latency, Whisper transcription time, LLM first-token time, RAM/VRAM use, network delay and behavior when more than one request arrives.

Official references and methodology

Verify current Home Assistant and Ollama support

Cloudzat separates speech capture, speech-to-text, intent/LLM processing and text-to-speech because each stage has different hardware requirements. Current Home Assistant voice/Ollama documentation and current Ollama hardware/context documentation are the technical baseline. Live products come from this sprint’s dedicated Amazon catalogue; calculator results are planning tiers rather than guaranteed latency benchmarks.

As an Amazon Associate, Cloudzat may earn from qualifying purchases. Prices, seller terms, exact configurations and Home Assistant or Ollama hardware support can change.