Home Assistant Ollama Hardware Requirements

Ollama hardware sizing

Home Assistant connects to an external Ollama server, so the LLM host can be a tiny CPU box, a high-memory mini PC or a dedicated GPU server. Hardware requirements depend on the chosen model and context—not Home Assistant itself.

See live hardware Size my workload

Quick answer

What this voice workload needs

For small 3B-class quantized models, 16GB system memory can be workable. 7B/8B-class use is more comfortable with 16–32GB and suitable VRAM if GPU acceleration is desired. Larger models, larger context and concurrency increase memory needs significantly.

Live Amazon hardware

Current products that fit this decision

Compare current Amazon listings relevant to this guide. Product availability and prices can change.

Checking the dedicated Home Assistant Voice & Local AI catalogue…

Buying decision

Match compute to speech and LLM load

Choose a model that reliably supports the Home Assistant job first. Then size RAM/VRAM with headroom for context and concurrent requests. Home Assistant’s integration defaults to 8K context, while Ollama notes that larger context consumes more memory.

Interactive decision tool

Ollama Hardware Requirements Calculator

Size the voice and local-AI workload before comparing products. Results are planning tiers, not benchmark guarantees. Verify current Home Assistant voice support and Ollama hardware support for the exact system.

Compatibility checklist

Four checks before you purchase

Choose focused or open-ended STT first

Speech-to-Phrase and Whisper have very different compute needs.

Treat LLM memory separately

Model size, quantization, context and concurrency drive RAM/VRAM.

Keep Piper in perspective

Text-to-speech is usually lighter than Whisper or Ollama.

Measure end-to-end latency

Microphone, network, STT, LLM and TTS all affect response time.

Home Assistant and Ollama are separate services

The Home Assistant integration connects to an external Ollama server over a URL. That architecture means the automation system itself does not define the LLM hardware requirement. Ollama may run on the same physical machine, in another container or on a dedicated GPU host. Size the Ollama side from the model, memory and latency target, then decide whether consolidation is operationally sensible.

Parameter count is only the first memory clue

A 3B, 8B or 14B label does not map to one exact RAM or VRAM number because quantization changes model size. Runtime overhead, context and concurrent requests add more memory. Use the calculator as a tiering tool, then check the actual model artifact and `ollama ps` after deployment. This avoids publishing false precision such as claiming every 8B model requires exactly the same GPU.

Three-billion-parameter class models are the easiest entry point

Small quantized models can run on 16GB system-memory hosts and modest GPUs, making them attractive for low-power Home Assistant experiments. Their limitation is often capability rather than hardware. Home Assistant warns that smaller local models are more likely to make mistakes, especially when controlling devices. Test tool use and reliability before treating low resource consumption as proof that the model is appropriate.

Seven- and eight-billion-class models favor 32GB hosts

A 7B/8B-class quantized model can still run on CPU, but 32GB system RAM provides more comfortable headroom for Home Assistant, the OS and context. GPU acceleration commonly becomes attractive at this tier because response time matters in spoken interaction. Eight gigabytes of VRAM may fit some quantizations, while 12GB provides more flexibility. Always verify the exact model and context rather than relying on the class name.

Fourteen-billion-class models make headroom more important

At 14B, system memory and GPU capacity start to dominate the server design. CPU-only inference is possible with enough RAM, but voice users may find the latency unsatisfying. Twelve to sixteen gigabytes of VRAM can be a more realistic GPU planning tier depending on quantization and context. A dedicated AI server also becomes easier to justify because Home Assistant itself does not need this much compute.

Context length has a direct memory cost

Home Assistant currently defaults its Ollama integration to an 8K context window. Ollama’s own documentation states that increasing context length increases memory requirements. Larger homes may benefit from more context for entities and conversation history, but it is not free. If a model barely fits at 8K, a jump to 32K can change whether it remains fully GPU-resident.

Parallel requests multiply allocation

Ollama documents that parallel request handling can increase context memory allocation. A household may rarely hit this, but automations, dashboards or several family members can generate overlapping use. Do not size a GPU to the exact memory footprint of one idle conversation if concurrency matters. Leave headroom or limit parallelism so a second request does not push the model into slower CPU offload.

CPU-only inference is a valid architecture

A supported GPU is not mandatory. For occasional questions, small models and relaxed latency expectations, modern CPUs with 32–64GB system memory can offer a simple local server with fewer driver dependencies. This is particularly attractive on mini PCs that are already running. GPU acceleration earns its cost when spoken interaction needs faster generation or when larger models make CPU latency impractical.

GPU support is specific, not generic

Ollama documents NVIDIA, selected AMD/ROCm, Apple Metal and experimental Vulkan hardware paths. A product page that simply says “AI GPU” does not establish compatibility. Verify the exact GPU family, operating system and driver stack. On Linux, container access also matters. A technically supported card that is not exposed correctly to Docker will fall back or fail regardless of its benchmark score.

System RAM still matters with a GPU

Even a GPU-backed server needs enough system memory for the OS, Home Assistant-side services, model loading and any CPU offload. Do not pair a 16GB GPU with an 8GB host and assume VRAM replaces RAM. For a combined local-AI box, 32GB system RAM is a sensible modern floor; 64GB is more comfortable for larger models, experimentation and multiple services.

Model storage grows faster than expected

Testing local AI encourages downloads. Several quantizations and model families can consume hundreds of gigabytes even if only one is loaded at a time. A 1TB NVMe drive is a practical starting point; 2TB or 4TB becomes useful for enthusiasts. Storage capacity should be planned independently from inference memory, because a 40GB model on disk does not mean it uses exactly 40GB of VRAM.

Hardware sizing ends with real telemetry

After setup, use Ollama’s status tools to see whether the model is on GPU, CPU or split, then measure response time and memory under the Home Assistant context you actually use. If the model fits with margin and responds quickly, the server is correctly sized. If it spills to CPU or queues under concurrency, upgrade the specific constraint instead of replacing unrelated components.

Questions people ask

Home Assistant Ollama Hardware Requirements questions

Can Home Assistant voice run fully locally?

Yes. Home Assistant documents fully local pipelines using local speech-to-text and Piper text-to-speech, with no requirement to send spoken commands to a cloud service.

What is the lightest local speech-to-text option?

Speech-to-Phrase is designed for focused home-control commands and can run very quickly on low-powered Home Assistant hardware.

What CPU does Home Assistant recommend for Whisper?

For fully local Voice Preview Edition use with Whisper Base, Home Assistant currently recommends at least an Intel N100 or equivalent processor.

Does Piper need a GPU?

Usually no. Piper is optimized for modest hardware and is much lighter than open-ended Whisper or a local LLM.

Does Home Assistant run Ollama itself?

The Home Assistant Ollama integration connects to an external Ollama server, which may be on the same machine or another computer reachable over the network.

How much context does Home Assistant use with Ollama?

Home Assistant currently defaults its Ollama integration to an 8K context window. Increasing context can improve capacity for larger homes but raises memory use.

How many entities should I expose to a local LLM?

Home Assistant currently recommends exposing fewer than 25 entities when experimenting with local LLM control, especially because smaller models can make more mistakes.

Is NVIDIA required for Ollama?

No. Ollama documents NVIDIA, supported AMD/ROCm, Apple Metal and experimental Vulkan paths. Compatibility depends on the exact hardware and operating system.

Should I put Ollama on a separate server?

Separate it when GPU power, memory pressure, experimentation or noise would compromise the always-on Home Assistant host. Small models can stay consolidated when simplicity matters more.

What should I measure after installation?

Measure end-to-end voice latency, Whisper transcription time, LLM first-token time, RAM/VRAM use, network delay and behavior when more than one request arrives.

Official references and methodology

Verify current Home Assistant and Ollama support