File: README.md
[!WARNING]
Active beta. Use the latest release for stable work ormainfor current fixes. Report problems through GitHub Issues.
At a glance
| VoiceStudio | |
|---|---|
| Workflows | Voice cloning and design, video dubbing, dictation, stories, audiobooks, batch generation |
| Language catalogue | 646 TTS languages; actual coverage and quality depend on the selected engine |
| Engines | 16 TTS ยท 11 ASR ยท switch in Model Catalogue or with Ctrl/Cmd+E |
| Platforms | macOS 13.3+ on Apple Silicon ยท Windows 10/11 x64 ยท Linux x86_64 with glibc 2.39+ |
| Compute | CUDA ยท Apple Silicon MPS/MLX ยท ROCm on Linux ยท CPU ยท optional remote workers |
| Interfaces | Desktop app ยท local REST/SSE/WebSocket API ยท OpenAI-compatible audio API ยท MCP Server |
| Storage | Voices, projects, settings, and outputs stay on the machine by default |
| License | AGPL-3.0; optional engines keep their own model licenses |
Install
| Platform | Package | Guide |
|---|---|---|
| macOS 13.3+ | DMG, Apple Silicon | Install on macOS |
| Windows 10/11 | MSI, x64 | Install on Windows |
| Linux | AppImage, x86_64 with glibc 2.39+ | Install on Linux |
| Docker | CUDA, ROCm, or CPU | Run with Docker |
Download packages from the latest release. First launch creates a managed Python environment and downloads the default model. Later launches reuse both.
[!NOTE]
On macOS, first launch needs a one-time right-click โ Open approval. Intel Macs cannot run the local Python backend; use a remote backend instead.
First voice
- Launch VoiceStudio and open Voice Cloning.
- Add a clean voice sample. Three seconds works; 5โ15 seconds usually gives a better prompt.
- Enter text, choose a language, then select Generate.
Run from source
Install the development prerequisites, then:
git clone https://github.com/debpalash/VoiceStudio.git
cd VoiceStudio
bun install
bun run desktopUse bun run dev for the browser UI. See Contributing for services, tests, and platform packages.
If setup fails
- Run Settings โ About โ Run self-check or
uv run python backend/main.py --diagnose --deep. - Check install troubleshooting.
- Save a scrubbed diagnostic bundle from the app when opening an issue.
- For slow generation, compare measured benchmarks and performance settings.
Features
| Area | Included |
|---|---|
| Voice Cloning | Zero-shot synthesis from a short reference clip |
| Voice Design | Create a voice from age, accent, pitch, style, and delivery instructions |
| Video Dubbing | Transcribe, translate, preserve speakers, synthesize, and export video |
| Stories and audiobooks | Multi-voice scripts ยท EPUB/PDF import ยท chapter rendering ยท .m4b export |
| Dictation Widget | System-wide shortcut, live transcription, optional local-LLM cleanup |
| Vocal Isolation | Demucs speech/background separation |
| Speaker Diarization | Pyannote and WhisperX speaker assignment |
| Batch Queue | Queue large sets of audio and video jobs with per-job progress |
| Model Catalogue | Install, remove, select, and route TTS, ASR, and LLM models |
| Remote Model Downloads | Install models on enrolled remote workers with live progress |
| GPU Auto-Detect | CUDA, MPS, ROCm, and CPU routing with per-engine checks |
| AI Watermark | AudioSeal embedding and detection |
| MCP Server | Synthesis and transcription tools for MCP clients |
| Diagnostics | Self-checks, error journal, logs, and scrubbed support bundles |
| Local-first | Core creation stays local; network-backed features are explicit opt-ins |
| Extensible | Registry-based TTS, ASR, and plugin interfaces |
| Model Catalogue: engine, device, and install state | Gallery: save a shared voice as a local profile |
Comparison
VoiceStudio trades managed cloud compute for local control. This is the practical difference:
| VoiceStudio | Typical hosted voice service | |
|---|---|---|
| Best fit | Private, offline, self-hosted, or high-volume work | Fast setup without local model management |
| Data path | Local by default; remote features are opt-in | Audio and text are processed by the provider |
| Cost model | Free software; you supply the hardware | Subscription, credits, or metered API use |
| Setup | Install the app and model weights | Create an account and use the web app or API |
| Performance | Depends on your engine and hardware | Provider manages compute and scaling |
| Offline use | Yes, after required models are installed | Usually requires a network connection |
| Customization | Source, engines, models, API, and routing are open | Limited to provider options |
| Maintenance | You manage updates, disk, and compute | Provider manages infrastructure |
Requirements
Requirements vary by engine. These values cover the default local workflow.
| Minimum | Recommended | |
|---|---|---|
| OS | Windows 10 x64 ยท macOS 13.3 Apple Silicon ยท Linux x86_64 with glibc 2.39+ | Current supported OS release |
| RAM | 8 GB | 16 GB+ |
| Disk | 10 GB free | 20 GB+ SSD |
| GPU | Optional; CPU mode is supported | NVIDIA CUDA or Apple Silicon |
| VRAM | 4 GB when using a GPU | 8 GB+; large optional engines need more |
| Python from source | 3.11+ | 3.11โ3.12 |
ROCm is Linux-only and opt-in. Windows AMD/Ryzen AI uses CPU. Systems with limited VRAM offload work to CPU when required. See performance, benchmarks, and engine disk usage.
Engines
Engine support is capability-specific. Check cloning, language, platform, memory, and license before choosing one. Full setup guides: docs/engines.
Text to speech
| Engine | Languages | Clone | Instruct | Linux | macOS ARM | Windows | License |
|---|---|---|---|---|---|---|---|
| VoiceStudio (default, powered by k2-fsa/OmniVoice) | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | AGPL-3.0 app ยท Apache-2.0 model |
| CosyVoice 3 | 9 + 18 dialects | Yes | Yes | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| GPT-SoVITS | 5 | Yes | โ | CUDA/CPU | โ | CUDA/CPU | MIT |
| VoxCPM2 | 30 | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | Apache-2.0 |
| MOSS-TTS-Nano | 20 | Yes | โ | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| KittenTTS | English | โ | โ | CPU | CPU | CPU | MIT |
| MLX-Audio | Model-dependent | Varies | Varies | โ | MLX | โ | Varies |
| Sherpa-ONNX | 20+ | โ | โ | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| IndexTTS 2.5 โก | ZH ยท EN ยท JA ยท ES ยท AR | Yes | โ | CUDA/CPU | CPU | CUDA/CPU | Bilibili model licenseยน |
| OmniVoice GGUF โก | 600+ | Yes | Yes | CUDA/CPU | MPS/CPU | CUDA/CPU | AGPL-3.0 app ยท Apache-2.0 model |
| OmniVoice (subprocess) โก | 600+ | Yes | Yes | CUDA/CPU | MPS | CUDA/CPU | AGPL-3.0 app ยท Apache-2.0 model |
| PocketTTS โก | EN ยท FR ยท DE ยท PT ยท IT ยท ES | Yes | โ | CPU | CPU | CPU | CC-BY-4.0, gatedยฒ |
| Supertonic 3 โก | 31 | โ | โ | CPU | CPU | CPU | OpenRAIL-M |
| MOSS-TTS-v1.5 โก | 31 | Yes | โ | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
| dots.tts โก | 24 | Yes | โ | CUDA/CPU | CPU | โ | Apache-2.0 |
| Confucius4-TTS โก | 14 | Yes | โ | CUDA/CPU | CPU | CUDA/CPU | Apache-2.0 |
โก Installed or registered on demand.
ยน IndexTTS 2.5 requires a separate written Bilibili license above 100 million monthly active users or RMB 1 billion annual revenue. Review the model license.
ยฒ PocketTTS shows its gated-access and CC-BY-4.0 terms before first use.
Clone-less engines cannot preserve a reference speaker in dubbing or pinned-voice batch jobs. VoiceStudio rejects those jobs instead of silently changing engines. Heavy engines have separate memory and platform limits; check their engine guide first.
Speech to text
| Engine | ID | Languages | Best fit |
|---|---|---|---|
| WhisperX (default) | whisperx |
~100 | Dubbing, subtitles, word-level timing |
| Faster-Whisper | faster-whisper |
~100 | General cross-platform transcription |
| Faster-Whisper (isolated) | faster-whisper-isolated |
~100 | Crash-isolated batch transcription |
| MLX Whisper | mlx-whisper |
~100 | Apple Silicon |
| PyTorch Whisper | pytorch-whisper |
~100 | CUDA, MPS, and CPU fallback |
| Parakeet TDT | nemo-parakeet |
English + 25 EU | Fast CPU/CUDA transcription |
| Parakeet TDT v3 (MLX) | parakeet-mlx |
25 EU | Apple Silicon dictation and word timestamps |
| Moonshine | moonshine |
English | Low-power, low-latency ONNX |
| FunASR | funasr |
50+ | VAD and inline diarization |
| sherpa-onnx (live dictation) | sherpa-onnx-asr |
Model-dependent | Streaming CPU dictation |
| OpenAI-compatible โ ๏ธ remote | openai-compat-asr |
Server-dependent | Qwen3-ASR or another compatible endpoint; audio leaves the machine |
WhisperX and Faster-Whisper retry with int8 when efficient float16 is unavailable. Pin ASR_COMPUTE_TYPE=int8 or float32 only if automatic selection still fails.
Architecture
Tauri v2 desktop shell (Rust)
โ IPC
React + Vite UI
โ HTTP ยท SSE ยท WebSocket on localhost:3900
FastAPI backend
โโโ TTS / ASR engine registries
โโโ dubbing / audio / long-form pipelines
โโโ OpenAI-compatible API and MCP server
โโโ SQLite + Alembic โ omnivoice_data/| Layer | Path | Responsibility |
|---|---|---|
| Desktop shell | frontend/src-tauri/ |
Window lifecycle, tray, shortcuts, updater, sidecar bootstrap |
| Frontend | frontend/src/ |
React UI, Zustand state, API and event clients, i18n |
| API | backend/api/ |
REST routes, schemas, auth boundaries, streaming |
| Core services | backend/services/ |
Generation, dubbing, audio processing, persistence |
| Engines | backend/engines/ |
Isolated and optional engine adapters |
| Worker system | backend/worker/ |
Authenticated remote compute and job transport |
| Data | omnivoice_data/ |
Projects, voices, settings, logs, and SQLite state |
| Delivery | scripts/, deploy/, .github/workflows/ |
Development, packaging, containers, releases, CI |
Network boundary
- The desktop talks to a loopback-only backend on
localhost:3900. - Loopback API calls need no server key. Remote access requires a share PIN or API key.
- Remote workers and OpenAI-compatible ASR are opt-in. The UI identifies when audio leaves the machine.
- Analytics is off until consent. If enabled, it sends allowlisted, content-free usage metadataโnot text, audio, file names, or projects.
OpenAI-compatible API
Point an OpenAI-compatible audio client at the local backend:
- base_url="https://api.openai.com/v1"
+ base_url="http://localhost:3900/v1"| Endpoint | Purpose |
|---|---|
POST /v1/audio/speech |
TTS to mp3, opus, aac, flac, wav, or pcm; select a profile with voice and an engine with model |
POST /v1/audio/transcriptions |
STT to json, text, verbose_json, srt, or vtt |
GET /v1/audio/voices |
List local voice profiles and engines |
from openai import OpenAI
client = OpenAI(base_url="http://localhost:3900/v1", api_key="local")
with client.audio.speech.with_streaming_response.create(
model="tts-1",
voice="",
input="Made on my own hardware.",
response_format="wav",
) as response:
response.stream_to_file("speech.wav")The full API reference is in Settings โ OpenAPI Reference. For LAN, Tailscale, or proxy access, read API authentication before exposing the backend.
Agent skills
Install the VoiceStudio skills for Claude Code, Codex, Cursor, and other skills.sh-compatible agents:
npx skills add debpalash/omnivoice-studioomnivoice: synthesize speech and transcribe audio through local VoiceStudio.oss-maintainer: the repository's open-source maintenance workflow.
Google Colab
The notebook runs the app and web UI on a Colab GPU. Colab is remote compute, so uploaded audio and project data do not remain local to your machine.
Documentation
| Need | Read |
|---|---|
| Install | macOS ยท Windows ยท Linux ยท Docker |
| Fix setup | Troubleshooting ยท model downloads ยท Hugging Face token |
| Choose an engine | Engine guides ยท benchmarks ยท expressive speech |
| Tune hardware | Performance ยท remote workers |
| Build integrations | API auth ยท MCP ยท examples |
| Build VoiceStudio | Contributing ยท engine acceptance |
| Track changes | Changelog ยท roadmap ยท latest release |
| Remove everything | Uninstall guide |
FAQ
Does it work on Apple Silicon and Intel Macs?
Apple Silicon is supported with MPS and MLX options. Intel Macs cannot run the local backend because current PyTorch wheels are unavailable; they can connect to a remote backend. See macOS installation.
How much VRAM do I need?
A GPU is optional. Use 4 GB VRAM as the minimum for accelerated work and 8 GB+ for the default multi-stage workflow. Large optional engines can require 12โ16 GB or more. Check the benchmarks and engine guide.
Why does a longer reference clip not always improve the clone?
Cloning is zero-shot: the clip is a prompt, not training data. Use 5โ15 seconds of one speaker, close to the microphone, without music, noise, or reverb. Match the tone and pace you want in the output. For training, see data preparation and training.
Can I use generated audio commercially?
Yes under VoiceStudio's AGPL-3.0 terms. Optional engines and model weights may use different licenses; review the selected engine's license before commercial use.
Does VoiceStudio collect data?
Not unless you opt in. Analytics is off by default and skipping consent keeps it off. When enabled, the app sends allowlisted, content-free usage metadata. Text, audio, file names, voices, and projects are excluded. Change this at Settings โ Privacy.
How do I remove VoiceStudio and its data?
Use scripts/uninstall.sh on macOS/Linux or scripts\uninstall.ps1 on Windows. Both show a dry run before deletion. See the uninstall guide for every path.
Community and contributing
- GitHub Issues for reproducible bugs and feature requests.
- Discord for setup help and project discussion.
- Good first issues for a scoped starting point.
- Contributing guide for setup, tests, and pull requests.
Support development
VoiceStudio is free and has no paid tier. Donations fund development and infrastructure.
Ko-fi ยท PayPal ยท Sponsorship details
License
VoiceStudio is licensed under AGPL-3.0. You may run it, modify it, use it internally, and sell generated audio. If you modify VoiceStudio and provide that modified version as a network service, AGPL requires you to offer the corresponding source under the same license. A commercial license is available for proprietary embedding; contact [email protected]. See LICENSE-NOTICE.md for the plain-language scope.
Optional engines and downloaded models retain their own licenses. The bundled omnivoice/ model remains Apache-2.0 upstream.
Acknowledgments
VoiceStudio builds on OmniVoice, WhisperX, Demucs, Pyannote, CTranslate2, AudioSeal, Tauri, Supertonic, Sherpa-ONNX, GPT-SoVITS, and PocketTTS.
File: backend/engines/omnivoice_gguf/README.md
OmniVoice GGUF Engine
Hardware-adaptive quantized variant of the upstream k2-fsa/OmniVoice
voice-cloning model. Runs the C++17/GGML omnivoice.cpp runtime as a
subprocess (via Phase 2's SubprocessBackend) so the runtime is fully
isolated from the parent's Python process.
License
- Model weights (
Serveurperso/OmniVoice-GGUF): Apache-2.0. Same
license as the upstreamk2-fsa/OmniVoiceit derives from. - Runtime (
omnivoice.cpp, the C++ inference binary inbin/): MIT. - Audio codec (Higgs Audio v2, embedded in the tokenizer GGUF): Apache-2.0.
The license chain is identical to the engine VoiceStudio already
ships in v0.2.7 (VoiceStudioBackend); this engine is the same model
under a different runtime.
Sources
- Model card: https://huggingface.co/Serveurperso/OmniVoice-GGUF
- Runtime source: https://github.com/ServeurpersoCom/omnivoice.cpp
- Decision doc:
docs/adr/SPIKE-01-gguf.md - Research:
docs/adr/SPIKE-01-gguf-research.md
Pinned SHAs
The _meta block of quant_map.json carries the pins:
source_commit_shaโ HuggingFace revision ofServeurperso/OmniVoice-GGUF
used to download quant files. Currently361609388ae572a820d085185bbbe2a2aac4b30e.runtime_commit_shaโomnivoice.cppmaster HEAD that the bundledbin/omnivoice-tts-binaries were built from. Currently886fc079838ca7400cb2b42b36e2a65aa1daabe8.
Both SHAs are mirrored in the SPIKE-01 ADR so the engine code and the
decision doc cannot drift.
Quant variants
| Compute class | Quant | Base size | Tokenizer size | Total VRAM | When auto-selected |
|---|---|---|---|---|---|
high-vram |
BF16 | 1.23 GB | 373 MB | ~1.60 GB | 12 GB+ VRAM |
mid-vram |
Q8_0 | 656 MB | 289 MB | ~945 MB | 4-12 GB VRAM |
low-vram |
Q4_K_M | 407 MB | 252 MB | ~659 MB | 1-4 GB VRAM |
cpu |
Q4_K_M | 407 MB | 252 MB | ~659 MB | CPU-only fallback |
F32 is published by the model but not auto-selected by the probe โ it's
~3.2 GB total and offers no quality gain over BF16. Available as a
Settings override for users on machines where bit-exact reference output
is needed.
Hardware probe
hardware_probe.detect_capabilities() walks:
torch.cuda.is_available()โ CUDA backend, total VRAM fromtorch.cuda.mem_get_info().torch.backends.mps.is_available()โ MPS backend, effective VRAM =
half of system RAM (unified-memory bookkeeping).- Otherwise โ CPU backend, VRAM = 0.
Thresholds (in _bucket()): 12 GB โ high-vram, 4 GB โ mid-vram,
1 GB โ low-vram, else cpu.
Subprocess invocation
The engine spawns bin/omnivoice-tts- once per generate()
call. The CLI matches the omnivoice.cpp README:
echo "Hello world." | ./build/omnivoice-tts \
--model models/omnivoice-base-Q8_0.gguf \
--codec models/omnivoice-tokenizer-Q8_0.gguf \
--lang English -o hello.wavThe Python wrapper composes argv from typed pathlib.Path objects rooted
in $HF_HUB_CACHE, never passes shell=True, and routes stdin/stderr
through SubprocessBackend.run() so HF token leak redaction (AUTH-05)
and the 120-second timeout (T-04-06) apply uniformly.
Settings override
backend/services/settings_store.py exposes get_quant_override() andset_quant_override(value). Allowed values:
Noneโ clear override; auto-select perquant_map.json."auto"โ explicit auto-select (same asNonebut writes a marker
row so the UI can show "user explicitly chose auto").- One of the
basefilenames listed inquant_map.jsonโ forces that
quant regardless of hardware bucket.
Freeform path input is rejected with ValueError (T-04-05 in the
plan's threat model).
macOS Gatekeeper
The bundled bin/omnivoice-tts-darwin-* binaries are not code-signed in
v0.3.x (REL-05 tracks signing as a separate work item). macOS Sequoia
Gatekeeper will quarantine them on first launch, surfacing the same UX
as the .app quarantine (issue #54).
The workaround is identical:
xattr -cr "/Applications/VoiceStudio.app"This clears the quarantine xattr recursively โ including on the bundledbin/omnivoice-tts-darwin-* binary inside the .app. Theis_available() probe in backend.py detects the quarantine xattr and
returns a clear error message pointing at this command rather than
silently hanging on a Gatekeeper-killed spawn.
If the macOS Apple Silicon Metal build fails to materialize in Wave 1
(no published buildmetal.sh in omnivoice.cpp per Pitfall 1), the
GGUF engine is unavailable on Apple Silicon and the existing in-processVoiceStudioBackend remains the cloning default on that platform โ no
hard block, no error toast on launch.
Smoke test
scripts/smoke-gguf.sh --hardware-class {cpu,mid,high} runs the
GGUF-06 cross-hardware acceptance test. See the script's --help for
details.
File: backend/migrations/README.md
Alembic migrations
Schema evolution lives here going forward. The legacy hand-rolled_migrate() function in backend/core/db.py is kept for the next release
as a fallback, and can be retired once Alembic has run in production.
Workflow
From the repo root:
# Create a new migration
uv run alembic revision -m "add glossary table"
# Apply pending migrations
uv run alembic upgrade head
# Show current schema version
uv run alembic current
# Downgrade one step
uv run alembic downgrade -1Conventions
- SQLite-safe:
env.pysetsrender_as_batch=True, soALTER TABLEemits
a table-rewrite strategy that works on SQLite. - No autogeneration: this repo has no SQLAlchemy models; every migration is
written by hand usingop.execute(...)or typed helpers likeop.add_column,op.create_table, etc. - No destructive migrations without review: if a migration deletes a column
or drops a table, the PR must be explicit about it.
Bootstrap note
On first run against an existing DB already at legacy PRAGMA user_version = 2,
stamp Alembic to a baseline before applying new migrations:
uv run alembic stamp headThis tells Alembic that the schema is up-to-date as of the baseline version.
File: bin/README.md
Bundled binaries
This directory holds platform-specific binaries that ship inside the
VoiceStudio installer. Today:
| File | Built from | Purpose |
|---|---|---|
omnivoice-tts-darwin-arm64 |
ServeurpersoCom/omnivoice.cpp @ pinned SHA |
GGUF inference runtime โ Apple Silicon |
omnivoice-tts-darwin-x86_64 |
same | Intel Mac |
omnivoice-tts-linux-x86_64 |
same | Linux |
omnivoice-tts-windows-x86_64.exe |
same | Windows |
checksums.sha256 |
computed by scripts/build-omnivoice-tts.sh |
SHA-256 manifest โ verified by VoiceStudioGGUFBackend.is_available() |
The pinned commit SHA for omnivoice.cpp lives inbackend/engines/omnivoice_gguf/quant_map.json _meta.runtime_commit_sha.
Building locally
scripts/build-omnivoice-tts.sh --platform <slug> --commit-sha <40hex>See .github/workflows/ci.yml build-omnivoice-tts job for the CI
matrix that produces these artifacts on every PR. The Apple Silicon
slot (macos-14) is marked continue-on-error: true because the
upstream omnivoice.cpp README does not publish a buildmetal.sh
(Pitfall 1 in 04-RESEARCH.md); a failed Metal build is documented
and the macOS Apple Silicon cloning default falls back to the
in-process VoiceStudioBackend.
Placeholder note
Until the CI matrix produces real binaries, this directory contains
zero-byte placeholders โ a plain git clone always gets those (real
binaries ship via the installers / CI artifacts, and are never committed
here). VoiceStudioGGUFBackend.is_available() validates the file before
trusting it (services/binary_preflight.py: non-empty + a real
Mach-O/ELF/PE magic, #1172) and returns (False, "...not a usable executable...") for a placeholder, so the engine reports honestly
through the Engine Compatibility Matrix and the default selection falls
back to the in-process VoiceStudioBackend. Selecting the engine anyway
(e.g. model: "omnivoice-gguf" on /v1/audio/speech) yields an
actionable 400/503 naming scripts/build-omnivoice-tts.sh, never a raw
"Exec format error".
File: docs/engines/cosyvoice.md
VoiceStudio โ CosyVoice Engine
CosyVoice is one of the multilingual TTS engines VoiceStudio can drive. It does
zero-shot voice cloning across 9+ languages with separate models for "base",
"instruct", and "SFT" use cases.
Install
CosyVoice is installed per-engine from the in-app Model Catalogue โ Engines tab:
- Open Model Catalogue โ Engines.
- Click Install next to "CosyVoice".
- The app fetches the engine source, creates a dedicated venv, syncs deps,
and downloads model weights (~2 GB). - Once installed, the engine appears in the Voice Cloning and
Voice Design engine picker dropdowns.
The dedicated venv keeps CosyVoice's transformer pins from clashing with
IndexTTS / ChatterboxTTS / SonicTranslate (see
troubleshooting.md).
Common errors
`Model not found: CosyVoice-300M-Instruct`
The first synthesis call downloads the weights from HuggingFace. If the
download was interrupted, the manifest can be inconsistent. Fix: delete~/.cache/huggingface/hub/models--FunAudioLLM--CosyVoice-300M* and retry โ
the engine re-downloads cleanly.
`HfHubHTTPError: 401 Client Error`
CosyVoice models are not gated as of v1.x, but the underlying download
goes through huggingface_hub which still wants a token for rate-limit
buckets. Set one โ see
docs/setup/huggingface-token.md.
`RuntimeError: CUDA out of memory` on first synthesise
The CosyVoice-300M-Instruct path peaks at ~4.5 GB VRAM. If you're on an 8 GB
GPU and also have a browser open, that's tight. Fix: close other CUDA
apps, or pick a smaller variant (CosyVoice-300M without instruct).
Troubleshooting
- Issue #55:
CosyVoice install clashing with IndexTTS โ fixed in v0.3+ via per-engine
venvs. - For other errors, capture the splash-screen log (Settings โ Logs โ Backend)
and open a bug report with Settings โ Help โ Report a bug (Phase 5
ships the auto-report path).
File: docs/engines/disk-usage.md
Engine venvs & disk usage
Most engines run in-process in VoiceStudio's main environment. A few
(IndexTTS2, MOSS-TTS-v1.5, dots.tts, and any engine whose
dependencies conflict with the parent's torch/transformers pins) run in a
dedicated sidecar venv so their pins can't break the rest of the app. Those
sidecars are where disk adds up โ this page explains why, and how the on-disk
cost is kept down.
Why a sidecar needs its own venv
IndexTTS2 pins transformers<5, but VoiceStudio requires transformers>=5.3.
You can't have both in one environment, so IndexTTS2 gets its own venv created
on first use (uv venv + uv pip install, seebackend/engines/indextts/bootstrap.py). The cost is a second copy of the
heavy ML stack โ most of which is torch + the bundled CUDA libraries.
How big is "a second torch"?
Measured (2026-06):
| Platform | torch CUDA wheel | + bundled CUDA libs |
|---|---|---|
| Linux (cu128) | ~0.83 GiB | several GiB of nvidia-* packages on top |
| Windows (cu128) | ~3.2 GiB (DLLs bundled in the wheel) | โ |
So a sidecar that pins a different torch version than the parent is a
multi-GB add. A sidecar that pins the same torch + CUDA build shares almost
all of it (see below).
uv dedupes identical wheels โ for free, with one condition
uv installs packages by linking from a global wheel cache into each venv'ssite-packages. The link mode:
- macOS + Linux โ
clone(copy-on-write reflink). N venvs that install the
same wheel share the bytes until one is modified โ effectively one copy on
disk. - Windows โ
hardlink. Same effect on a single volume.
The one condition: the cache and the venv must be on the same
filesystem. If UV_CACHE_DIR lives on a different drive than the engine
venvs, uv falls back to a full copy (no dedup, slower). Keep them together.
Dedup is per identical wheel. torch==2.6.0+cu124 and torch==2.8.0+cu128
are different wheels โ zero sharing โ a full extra multi-GB copy. The single
biggest disk decision for a sidecar is therefore: pin the same torch build as
the parent whenever the engine allows it. When it doesn't (IndexTTS2'stransformers<5 forces an older torch line), the second copy is the
unavoidable price of isolation โ not a bug.
The opt-in #498 engines illustrate both sides: dots.tts pinstorch==2.8.0 โ the same build the parent constrains to โ so it shares
almost all of torch with the main venv and only its transformers==4.57 +
model deps are new. MOSS-TTS-v1.5 pins torch==2.9.1+cu128, a different
build, so it pays a full extra multi-GB torch copy on CUDA hosts (the price of
running an 8B model whose stack pins transformers==5.0).
On Linux, the
nvidia-*CUDA packages are separate wheels, so even across
different torch versions anynvidia-*whose pinned version happens to
match is still shared. On Windows the CUDA DLLs live inside the one torch
wheel, so nothing is shared across torch versions.
Practical guidance
- Keep
UV_CACHE_DIRand the engine venvs on one filesystem (the default โ
both under your home dir โ already satisfies this). The app enforces this
automatically: when the app env or the engine venvs live on a different
volume than uv's default cache (D:-drive install, portable mode), every
manageduvinvocation getsUV_CACHE_DIRpointed at a cache next to the
venvs (<env root>/uv-cache,<data dir>/engines/.uv-cache). An explicitUV_CACHE_DIRyou set yourself always wins. - On Linux ext4 (no reflink),
export UV_LINK_MODE=hardlinkguarantees dedup
on any single filesystem; the defaultcloneonly dedupes on
reflink-capable filesystems (XFS-with-reflink, btrfs, APFS). - Reclaiming space: deleting a sidecar venv (
backend/engines//.venv/)
frees its unique files; shared cache bytes stay untiluv cache prune. - Forward-looking: PyTorch's experimental wheel variants (shipped in 2.8)
will eventually letuv install torchauto-pick the right CUDA build, and
uv already exposes--torch-backend=auto.
File: docs/engines/dots-tts.md
VoiceStudio โ dots.tts Engine
dots.tts (rednote-hilab) is a 2B fully-continuous autoregressive TTS,
widely cited as one of the strongest open zero-shot voice-cloning models. It
covers 24 languages, emits 48 kHz audio, and is released under
Apache-2.0 (code + checkpoints).
It runs in its own subprocess and its own Python venv withtransformers==4.57.0, isolated from the VoiceStudio parent process which
pins transformers>=5.3 โ the same isolation primitive used by
IndexTTS-2 and MOSS-TTS-v1.5.
Opt-in, and never a default. dots.tts is selected explicitly in
Model Catalogue โ Engines (orOMNIVOICE_TTS_BACKEND=dots-tts). It is not
part of the default install.
Platform support
- Linux / macOS only. dots.tts's upstream package declares Linux and
macOS classifiers and has no Windows install path. On Windows the
engine reports itself unavailable in Model Catalogue โ Engines with a clear
reason โ run VoiceStudio under WSL2 or use a Linux/macOS host. - No MPS. Upstream device selection is CUDA-or-CPU with no Metal branch,
so on Apple Silicon the official package runs on CPU (slow but
correct). A faster Apple-Silicon path exists only via community MLX ports,
which VoiceStudio does not auto-wire. - VRAM: ~9 GB checkpoint; a 12โ16 GB CUDA GPU is the realistic target.
Install
dots.tts is not bundled (large checkpoint + conflicting transformers).
Clone the dots.tts repo on disk:
bashgit clone https://github.com/rednote-hilab/dots.tts.gitInstall the editable package into a fresh venv with the upstream
constraints. Useuv pip install -e . -c constraints/recommended.txtโ
neveruv sync --all-extras, which would overwrite VoiceStudio's lock
file withtransformers==4.57and break the parent process:bashcd dots.tts uv venv .venv uv pip install -e . -c constraints/recommended.txtThe ~9 GB checkpoint downloads from HuggingFace on first synthesize. The
parent forwardsHF_HOME/HF_HUB_CACHEto the sidecar so the cache is
shared with the rest of VoiceStudio's downloads.Set
OMNIVOICE_DOTS_TTS_DIRto the repo root (the directory that
containspyproject.tomlandconstraints/):bash# macOS / Linux echo 'export OMNIVOICE_DOTS_TTS_DIR=$HOME/code/dots.tts' >> ~/.zshrc source ~/.zshrcRestart VoiceStudio. dots.tts appears in Model Catalogue โ Engines with
available: trueandisolation_mode: subprocess.
Venv resolution order
VoiceStudio probes for a usable dots.tts Python interpreter in this priority
order (see backend/engines/dots_tts/bootstrap.py):
${OMNIVOICE_DOTS_TTS_DIR}/.venv/โ your existing clone's venv.backend/engines/dots_tts/.venv/โ VoiceStudio's own venv, created on
demand by step 3.- Lazy bootstrap โ
uv venvthenuv pip install -e <clone> -c <clone>/constraints/recommended.txt. RequiresOMNIVOICE_DOTS_TTS_DIR.
Voice cloning
For best fidelity ("continuation cloning"), pass both a reference clip
(ref_audio) and its exact transcript (ref_text). A reference clip alone
does x-vector-only cloning. Keep the reference ~10 s. Upstream requires the
reference audio whenever a transcript is given, so VoiceStudio drops a strayref_text that arrives without ref_audio.
Optional env knobs
| Variable | Default | Purpose |
|---|---|---|
OMNIVOICE_DOTS_TTS_DIR |
โ | Path to the dots.tts clone (required). |
OMNIVOICE_DOTS_TTS_MODEL |
rednote-hilab/dots.tts-soar |
Checkpoint override (-base, -soar, -mf). |
OMNIVOICE_DOTS_TTS_PRECISION |
bfloat16 (CUDA) / float32 (CPU) |
Inference precision. |
OMNIVOICE_DOTS_TTS_OPTIMIZE |
0 |
1 enables torch.compile (slower first call, faster after). |
Using the
dots.tts-mf(MeanFlow-distilled) checkpoint? It's tuned for
4 flow-matching steps โ passnum_step=4.
Common errors
`dots.tts is not supported on Windows ...`
Upstream is Linux/macOS only. Use WSL2 or a Linux/macOS host.
`dots.tts venv not found. Set OMNIVOICE_DOTS_TTS_DIR ...`
You haven't pointed VoiceStudio at a dots.tts clone yet. Follow Install.
License
Apache-2.0 (code and checkpoints). See the upstream
README.
dots.tts runs in a dedicated sidecar venv (it pins transformers==4.57,
which conflicts with the parent's transformers>=5.3). For why that adds
disk and how uv keeps the cost down, see
Engine venvs & disk usage.
File: docs/engines/indextts.md
VoiceStudio โ IndexTTS 2.5
IndexTTS 2.5 is an optional, multilingual voice-cloning engine for dubbing
and expressive speech. It supports Chinese, English, Japanese, Spanish, and
Arabic, with reference-audio cloning, emotion references, emotion vectors,
and text-directed emotion.
VoiceStudio runs IndexTTS in a dedicated subprocess and Python environment.
This keeps its transformers<5 dependency isolated from VoiceStudio's
runtime. Existing user-managed IndexTTS-2 environments remain supported.
Install
IndexTTS 2.5 is not bundled because its source environment and model weights
require substantial disk space.
- Open Model Catalogue โ Engines.
- Expand IndexTTS 2.5 and select Install.
- Keep VoiceStudio open while source, dependencies, and weights download.
The installer:
- checks for
uvand at least 12 GB of free space; - installs the reviewed
indextts-2.5source revision in an isolated venv; - downloads the reviewed
IndexTeam/IndexTTS-2.5model revision; - resumes partial model downloads;
- saves
OMNIVOICE_INDEXTTS_DIRand activates the engine without a restart.
An app-managed IndexTTS-2 checkout remains intact while 2.5 installs into a
separate directory. VoiceStudio switches to 2.5 only after the new source,
environment, and weights pass verification. User-managed clones are never
modified or removed; their legacyindextts.infer_v2 entry point remains supported.
Manual install
Use a separate checkout and venv. Do not install IndexTTS into VoiceStudio's
root environment.
git clone --branch indextts-2.5 https://github.com/index-tts/index-tts.git
cd index-tts
uv venv .venv
uv pip install --python .venv/bin/python -e .
hf download IndexTeam/IndexTTS-2.5 --local-dir=checkpointsOn Windows, replace .venv/bin/python with .venv\Scripts\python.exe.
Then set OMNIVOICE_INDEXTTS_DIR to the checkout root:
export OMNIVOICE_INDEXTTS_DIR=/path/to/index-tts[Environment]::SetEnvironmentVariable(
"OMNIVOICE_INDEXTTS_DIR",
"$env:USERPROFILE\code\index-tts",
"User"
)Restart VoiceStudio after setting a persistent environment variable outside
the app.
Compatibility
VoiceStudio probes these locations in order:
${OMNIVOICE_INDEXTTS_DIR}/.venv/;backend/engines/indextts/.venv/;- a venv bootstrapped from
OMNIVOICE_INDEXTTS_DIR.
The probe prefers indextts.infer_v2_5 and falls back toindextts.infer_v2. A timed-out import is treated as unproven rather than
missing, preventing slow disks or antivirus scans from hiding a valid venv.
Set OMNIVOICE_INDEXTTS_IMPORT_PROBE_TIMEOUT_S to raise the default 60-second
probe limit.
IndexTTS 2.5 requires a language token. VoiceStudio maps locale codes and
language names to the five supported languages and detects Chinese, Japanese,
or Arabic script for Auto requests. Ambiguous Latin text defaults to English.
IndexTTS 2.5 uses duration_factor for native duration guidance. VoiceStudio's
dubbing fit stage remains responsible for exact segment timing. Legacy
IndexTTS-2 installations continue receiving their target_tokens control.
Troubleshooting
Engine unavailable
Use Model Catalogue โ Engines โ IndexTTS 2.5 โ Install. For a manual install,
confirm that the configured directory contains:
pyproject.toml
indextts/infer_v2_5.py
checkpoints/config_v2_5.yaml`uv` not found
Install uv from https://docs.astral.sh/uv/ or configure the bundled binary
through OMNIVOICE_BUNDLED_UV.
Import fails after installation
For an app-managed install, retry Install to repair the source and venv.
For a manual install, run:
uv pip install --python .venv/bin/python -e .Insufficient disk space
Free the amount reported by the installer, then retry. Completed model files
are reused.
License
IndexTTS 2.5 uses the bilibili Model Use License. It grants a limited,
worldwide, non-exclusive, royalty-free license subject to its restrictions.
A separate license is required when the user or an affiliate exceeded 100
million monthly active users in the preceding month or RMB 1 billion in annual
revenue in the preceding year. The agreement also includes downstream,
derivative-work, prohibited-use, attribution, and compliance obligations.
Review the official license
before installing or using the model.
This model is not covered by VoiceStudio's blanket commercial-use statement.
Organizations above either threshold must obtain Bilibili's separate written
license before installing or using IndexTTS 2.5. Other engines remain available
without enabling this optional sidecar.
See Engine venvs and disk usage for storage details.
File: docs/engines/moss-tts-v15.md
VoiceStudio โ MOSS-TTS-v1.5 Engine
MOSS-TTS-v1.5 (OpenMOSS) is an 8B flagship zero-shot TTS โ a Qwen3-8B
language backbone plus a 1.6B audio codec. It covers 31 languages, does
zero-shot voice cloning, token-level duration control and inline[pause Ns] markers. Released under Apache-2.0 (code + weights).
It runs in its own subprocess and its own Python venv withtransformers==5.0.0, isolated from the VoiceStudio parent process which
pins transformers>=5.3. This is the same isolation primitive used by
IndexTTS-2: the two transformers pins cannot share one
interpreter, so MOSS runs behindbackend/services/subprocess_backend.py::SubprocessBackend.
Opt-in, and never a default. MOSS-TTS-v1.5 is selected explicitly in
Model Catalogue โ Engines (orOMNIVOICE_TTS_BACKEND=moss-tts-v15). It is
not part of the default install and does not change VoiceStudio's
out-of-the-box behaviour on any platform.
Hardware
- VRAM/RAM: an 8B model. The upstream llama.cpp pipeline fits the 8B on
8 GB GPUs when quantized; the bf16 Transformers path used here is ~16 GB
of weights, so a 16 GB+ GPU is the realistic CUDA target. It also runs on
CPU (fp32) โ correct but slow. - Device: CUDA when present, else CPU. There is no MPS path โ
upstream documents only CUDA/CPU and the custom modelling code is
untested on Apple Silicon, so VoiceStudio never routes MOSS to MPS. On a
Mac it runs on CPU.
Install
MOSS-TTS-v1.5 is not bundled (the model is large and the package pins a
conflicting transformers). VoiceStudio ships a sidecar runner that loads it
into an isolated venv on demand.
Clone the MOSS-TTS repo on disk:
bashgit clone https://github.com/OpenMOSS/MOSS-TTS.gitInstall the editable package into a fresh venv. Use
uv pip install -e ".[torch-runtime]"โ neveruv sync --all-extras,
which would overwrite VoiceStudio's lock file withtransformers==5.0and
break the parent process. Thetorch-runtimeextra is CUDA (+cu128):bashcd MOSS-TTS uv venv .venv uv pip install -e ".[torch-runtime]"On a non-CUDA / CPU host (e.g. Apple Silicon), install plain
torch/torchaudio/transformers==5.0.0into the venv instead of the+cu128extra (the auto-bootstrap below only targets CUDA hosts).The ~16 GB weights download from HuggingFace on first synthesize. The
parent forwardsHF_HOME/HF_HUB_CACHEto the sidecar so the cache is
shared with the rest of VoiceStudio's downloads.Set
OMNIVOICE_MOSS_TTS_V15_DIRto the repo root (the directory that
containspyproject.toml):bash# macOS / Linux echo 'export OMNIVOICE_MOSS_TTS_V15_DIR=$HOME/code/MOSS-TTS' >> ~/.zshrc source ~/.zshrcpowershell# Windows PowerShell [Environment]::SetEnvironmentVariable("OMNIVOICE_MOSS_TTS_V15_DIR","$env:USERPROFILE\code\MOSS-TTS","User")Restart VoiceStudio. MOSS-TTS-v1.5 appears in Model Catalogue โ Engines with
available: trueandisolation_mode: subprocess.
Venv resolution order
VoiceStudio probes for a usable MOSS Python interpreter in this priority
order (see backend/engines/moss_tts_v15/bootstrap.py):
${OMNIVOICE_MOSS_TTS_V15_DIR}/.venv/โ your existing clone's venv.
Highest priority, so a power user who already set MOSS up gets zero
re-install.backend/engines/moss_tts_v15/.venv/โ VoiceStudio's own venv,
created on demand by step 3.- Lazy bootstrap โ if neither venv exists, VoiceStudio runs
uv venv
thenuv pip install --python -e "${DIR}[torch-runtime]".
RequiresOMNIVOICE_MOSS_TTS_V15_DIR; raises a clear error otherwise.
On a non-CUDA host the+cu128extra cannot resolve โ set the venv up
manually per step 2.
Voice cloning
Pass a reference clip as ref_audio. MOSS's zero-shot clone mode needs only
the audio (no transcript). Without a reference, MOSS synthesizes in its own
default voice. duration (seconds) maps to MOSS's tokens argument at
~12.5 tokens/second.
Optional env knobs
| Variable | Default | Purpose |
|---|---|---|
OMNIVOICE_MOSS_TTS_V15_DIR |
โ | Path to the MOSS-TTS clone (required). |
OMNIVOICE_MOSS_TTS_V15_MODEL |
OpenMOSS-Team/MOSS-TTS-v1.5 |
Advanced HF repo override. Custom repositories execute their modelling code and are rejected unless both safeguards below are set. Configure HF_ENDPOINT for a mirror instead. |
OMNIVOICE_MOSS_TTS_V15_REVISION |
โ | Immutable 40-character commit SHA required with a custom model repository. |
OMNIVOICE_MOSS_TTS_V15_TRUST_REMOTE_CODE |
โ | Set to 1 only after auditing a custom repository's Python code. The reviewed built-in repository needs no opt-in. |
OMNIVOICE_MOSS_TTS_V15_ATTN |
sdpa |
Attention impl; set flash_attention_2 on Ampere+ CUDA with flash-attn installed. |
Common errors
`MOSS-TTS-v1.5 venv not found. Set OMNIVOICE_MOSS_TTS_V15_DIR ...`
You haven't pointed VoiceStudio at a MOSS-TTS clone yet. Follow Install.
`uv pip install -e failed ... '[torch-runtime]' extra (cu128) cannot resolve`
You're on a non-CUDA host. The upstream torch-runtime extra is CUDA-only;
set up the venv manually with plain torch/transformers==5.0.0 (step 2).
License
Apache-2.0 (code and weights) โ no acceptance gate. See the upstream
README.
MOSS-TTS-v1.5 runs in a dedicated sidecar venv (it pins transformers==5.0,
which conflicts with the parent's transformers>=5.3). For why that adds
disk and how uv keeps the cost down, see
Engine venvs & disk usage.