91 lines
2.4 KiB
Markdown
91 lines
2.4 KiB
Markdown
|
# Local LiveKit Voice Chat
|
||
|
|
|
||
|
|
Talk to a locally-running LLM through your browser. Everything runs in Docker — one command to start.
|
||
|
|
|
||
|
|
## Stack
|
||
|
|
|
||
|
|
| Layer | Tool |
|
||
|
|
|---|---|
|
||
|
|
| Signalling / WebRTC | LiveKit Server |
|
||
|
|
| LLM | Ollama (Heretic-Dolphin3.0-Qwen2.5-3b by default) |
|
||
|
|
| VAD | Silero (in agent container) |
|
||
|
|
| STT | faster-whisper (in agent container, fully local) |
|
||
|
|
| TTS | Kokoro (local, OpenAI-compat API) |
|
||
|
|
| UI + token server | FastAPI + plain HTML |
|
||
|
|
|
||
|
|
## Prerequisites
|
||
|
|
|
||
|
|
- Docker + Docker Compose (that's it)
|
||
|
|
|
||
|
|
## Quick start
|
||
|
|
|
||
|
|
```bash
|
||
|
|
cp .env.example .env # edit if needed
|
||
|
|
./start.sh
|
||
|
|
```
|
||
|
|
|
||
|
|
Open **http://localhost:7731** — pick a model, click Connect, start talking.
|
||
|
|
|
||
|
|
First run will:
|
||
|
|
1. Pull Docker images (~2-3 GB total)
|
||
|
|
2. Download the default LLM via Ollama (~2 GB)
|
||
|
|
3. Download the Kokoro TTS model (~350 MB)
|
||
|
|
|
||
|
|
Subsequent starts are fast.
|
||
|
|
|
||
|
|
## Configuration (`.env`)
|
||
|
|
|
||
|
|
```ini
|
||
|
|
# LLM
|
||
|
|
DEFAULT_MODEL=hf.co/mradermacher/Heretic-Dolphin3.0-Qwen2.5-3b-i1-GGUF:Q4_K_M
|
||
|
|
|
||
|
|
# TTS: kokoro | piper | elevenlabs | openai
|
||
|
|
TTS_BACKEND=kokoro
|
||
|
|
KOKORO_VOICE=af_heart # af_heart af_sky am_adam bm_lewis bf_emma ...
|
||
|
|
|
||
|
|
# STT (faster-whisper model size)
|
||
|
|
WHISPER_MODEL=base # tiny base small medium large-v3
|
||
|
|
|
||
|
|
# ElevenLabs (only if TTS_BACKEND=elevenlabs)
|
||
|
|
# ELEVENLABS_API_KEY=sk-...
|
||
|
|
# ELEVENLABS_VOICE_ID=21m00Tcm4TlvDq8ikWAM
|
||
|
|
```
|
||
|
|
|
||
|
|
## Kokoro voices
|
||
|
|
|
||
|
|
Browse samples at: https://huggingface.co/hexgrad/Kokoro-82M
|
||
|
|
|
||
|
|
Popular voices:
|
||
|
|
|
||
|
|
| Voice | Style |
|
||
|
|
|---|---|
|
||
|
|
| `af_heart` | American female, warm (default) |
|
||
|
|
| `af_sky` | American female, airy |
|
||
|
|
| `af_bella` | American female, expressive |
|
||
|
|
| `am_adam` | American male |
|
||
|
|
| `am_michael` | American male, deep |
|
||
|
|
| `bm_lewis` | British male |
|
||
|
|
| `bf_emma` | British female |
|
||
|
|
|
||
|
|
## LLM models
|
||
|
|
|
||
|
|
The default is `Heretic-Dolphin3.0-Qwen2.5-3b` (fine-tuned, conversational, ~2 GB GGUF).
|
||
|
|
To switch, edit `DEFAULT_MODEL` in `.env` — any `ollama pull`-compatible model name works.
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# Browse and pull models manually
|
||
|
|
docker compose exec ollama ollama list
|
||
|
|
docker compose exec ollama ollama pull llama3.2
|
||
|
|
```
|
||
|
|
|
||
|
|
## Useful commands
|
||
|
|
|
||
|
|
```bash
|
||
|
|
./start.sh # start everything (foreground, Ctrl-C to stop)
|
||
|
|
./start.sh -d # start in background
|
||
|
|
docker compose down # stop all services
|
||
|
|
docker compose logs -f agent # watch agent logs
|
||
|
|
docker compose logs -f kokoro # watch TTS logs
|
||
|
|
```
|
||
|
|
docker compose exec ollama ollama pull hf.co/mradermacher/heretic_magnum-v2-4b-GGUF:Q4_K_M
|