2.4 KiB
2.4 KiB
Local LiveKit Voice Chat
Talk to a locally-running LLM through your browser. Everything runs in Docker — one command to start.
Stack
| Layer | Tool |
|---|---|
| Signalling / WebRTC | LiveKit Server |
| LLM | Ollama (Heretic-Dolphin3.0-Qwen2.5-3b by default) |
| VAD | Silero (in agent container) |
| STT | faster-whisper (in agent container, fully local) |
| TTS | Kokoro (local, OpenAI-compat API) |
| UI + token server | FastAPI + plain HTML |
Prerequisites
- Docker + Docker Compose (that's it)
Quick start
cp .env.example .env # edit if needed
./start.sh
Open http://localhost:7731 — pick a model, click Connect, start talking.
First run will:
- Pull Docker images (~2-3 GB total)
- Download the default LLM via Ollama (~2 GB)
- Download the Kokoro TTS model (~350 MB)
Subsequent starts are fast.
Configuration (.env)
# LLM
DEFAULT_MODEL=hf.co/mradermacher/Heretic-Dolphin3.0-Qwen2.5-3b-i1-GGUF:Q4_K_M
# TTS: kokoro | piper | elevenlabs | openai
TTS_BACKEND=kokoro
KOKORO_VOICE=af_heart # af_heart af_sky am_adam bm_lewis bf_emma ...
# STT (faster-whisper model size)
WHISPER_MODEL=base # tiny base small medium large-v3
# ElevenLabs (only if TTS_BACKEND=elevenlabs)
# ELEVENLABS_API_KEY=sk-...
# ELEVENLABS_VOICE_ID=21m00Tcm4TlvDq8ikWAM
Kokoro voices
Browse samples at: https://huggingface.co/hexgrad/Kokoro-82M
Popular voices:
| Voice | Style |
|---|---|
af_heart |
American female, warm (default) |
af_sky |
American female, airy |
af_bella |
American female, expressive |
am_adam |
American male |
am_michael |
American male, deep |
bm_lewis |
British male |
bf_emma |
British female |
LLM models
The default is Heretic-Dolphin3.0-Qwen2.5-3b (fine-tuned, conversational, ~2 GB GGUF).
To switch, edit DEFAULT_MODEL in .env — any ollama pull-compatible model name works.
# Browse and pull models manually
docker compose exec ollama ollama list
docker compose exec ollama ollama pull llama3.2
Useful commands
./start.sh # start everything (foreground, Ctrl-C to stop)
./start.sh -d # start in background
docker compose down # stop all services
docker compose logs -f agent # watch agent logs
docker compose logs -f kokoro # watch TTS logs
docker compose exec ollama ollama pull hf.co/mradermacher/heretic_magnum-v2-4b-GGUF:Q4_K_M