VoxCPM Documentation

File: README.md Copy Chapter

VoxCPM2: Tokenizer-Free TTS for Multilingual Speech Generation, Creative Voice Design, and True-to-Life Cloning

VoxCPM is a tokenizer-free Text-to-Speech system that directly generates continuous speech representations via an end-to-end diffusion autoregressive architecture, bypassing discrete tokenization to achieve highly natural and expressive synthesis.

VoxCPM2 is the latest major release — a 2B parameter model trained on over 2 million hours of multilingual speech data, now supporting 30 languages, Voice Design, Controllable Voice Cloning, and 48kHz studio-quality audio output. Built on a MiniCPM-4 backbone.

✨ Highlights Copy Section

• 🌍 30-Language Multilingual — Input text in any of the 30 supported languages and synthesize directly, no language tag needed
• 🎨 Voice Design — Create a brand-new voice from a natural-language description alone (gender, age, tone, emotion, pace …), no reference audio required
• 🎛️ Controllable Cloning — Clone any voice from a short reference clip, with optional style guidance to steer emotion, pace, and expression while preserving the original timbre
• 🎙️ Ultimate Cloning — Reproduce every vocal nuance: provide both reference audio and its transcript, and the model continues seamlessly from the reference, faithfully preserving every vocal detail — timbre, rhythm, emotion, and style (same as VoxCPM1.5)
• 🔊 48kHz High-Quality Audio — Accepts 16kHz reference audio and directly outputs 48kHz studio-quality audio via AudioVAE V2's asymmetric encode/decode design, with built-in super-resolution — no external upsampler needed
• 🧠 Context-Aware Synthesis — Automatically infers appropriate prosody and expressiveness from text content
• ⚡ Real-Time Streaming — RTF as low as ~0.3 on NVIDIA RTX 4090, and ~0.13 accelerated by Nano-vLLM or vLLM-Omni — official vLLM omni-modal serving for VoxCPM2 with PagedAttention and an OpenAI-compatible API
• 📜 Fully Open-Source & Commercial-Ready — Weights and code released under the Apache-2.0 license, free for commercial use

🌍 Supported Languages (30)
Arabic, Burmese, Chinese, Danish, Dutch, English, Finnish, French, German, Greek, Hebrew, Hindi, Indonesian, Italian, Japanese, Khmer, Korean, Lao, Malay, Norwegian, Polish, Portuguese, Russian, Spanish, Swahili, Swedish, Tagalog, Thai, Turkish, Vietnamese

Chinese Dialect: 四川话, 粤语, 吴语, 东北话, 河南话, 陕西话, 山东话, 天津话, 闽南话

News Copy Section

• [2026.04] 🔥 We release VoxCPM2 — 2B, 30 languages, Voice Design & Controllable Voice Cloning, 48kHz audio output! Weights | Docs | Playground | Technical Report
• [2025.12] 🎉 Open-source VoxCPM1.5 weights with SFT & LoRA fine-tuning. (🏆 #1 GitHub Trending)
• [2025.09] 🔥 Release VoxCPM Technical Report.
• [2025.09] 🎉 Open-source VoxCPM-0.5B weights (🏆 #1 HuggingFace Trending)

Contents Copy Chapter

• Quick Start
  • Installation
  • Python API
  • CLI Usage
  • Web Demo
  • Production Deployment
  • On-Device Inference (llama.cpp-omni)
• Models & Versions
• Performance
• Fine-tuning
• Documentation
• Ecosystem & Community
• Risks and Limitations
• Citation

🚀 Quick Start Copy Chapter

Installation Copy Section

pip install voxcpm

Requirements: Python ≥ 3.10 (<3.13), PyTorch ≥ 2.5.0, CUDA ≥ 12.0. See Quick Start Docs for details.

Python API Copy Section

🗣️ Text-to-Speech

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained(
  "openbmb/VoxCPM2",
  load_denoiser=False,
)

wav = model.generate(
    text="VoxCPM2 is the current recommended release for realistic multilingual speech synthesis.",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)
print("saved: demo.wav")

If you prefer downloading from ModelScope first, you can use:

pip install modelscope

from modelscope import snapshot_download
snapshot_download("OpenBMB/VoxCPM2", local_dir='./pretrained_models/VoxCPM2') # specify the local directory to save the model

from voxcpm import VoxCPM
import soundfile as sf
model = VoxCPM.from_pretrained("./pretrained_models/VoxCPM2", load_denoiser=False)

wav = model.generate(
    text="VoxCPM2 is the current recommended release for realistic multilingual speech synthesis.",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)

🎨 Voice Design

Create a voice from a natural-language description — no reference audio needed. Format: put the description in parentheses at the start of text(e.g. "(your voice description)The text to synthesize."):

wav = model.generate(
    text="(A young woman, gentle and sweet voice)Hello, welcome to VoxCPM2!",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)

🎛️ Controllable Voice Cloning

Upload a reference audio. The model clones the timbre, and you can still use control instructions to adjust speed, emotion, or style.

wav = model.generate(
    text="This is a cloned voice generated by VoxCPM2.",
    reference_wav_path="path/to/voice.wav",
)
sf.write("clone.wav", wav, model.tts_model.sample_rate)

wav = model.generate(
    text="(slightly faster, cheerful tone)This is a cloned voice with style control.",
    reference_wav_path="path/to/voice.wav",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("controllable_clone.wav", wav, model.tts_model.sample_rate)

🎙️ Ultimate Cloning

Provide both the reference audio and its exact transcript for audio-continuation-based cloning with every vocal nuance reproduced. For maximum cloning similarity, pass the same reference clip to both reference_wav_path and prompt_wav_path as shown below:

wav = model.generate(
    text="This is an ultimate cloning demonstration using VoxCPM2.",
    prompt_wav_path="path/to/voice.wav",
    prompt_text="The transcript of the reference audio.",
    reference_wav_path="path/to/voice.wav", # optional, for better simliarity 
)
sf.write("hifi_clone.wav", wav, model.tts_model.sample_rate)

🔄 Streaming API

import numpy as np

chunks = []
for chunk in model.generate_streaming(
    text="Streaming text to speech is easy with VoxCPM!",
):
    chunks.append(chunk)
wav = np.concatenate(chunks)
sf.write("streaming.wav", wav, model.tts_model.sample_rate)

CLI Usage Copy Section

# Voice design (no reference audio needed)
voxcpm design \
  --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
  --output out.wav

# Controllable voice cloning with style control
voxcpm design \
  --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
  --control "Young female voice, warm and gentle, slightly smiling" \
  --seed 42 \
  --output out.wav

# Voice cloning (reference audio)
voxcpm clone \
  --text "This is a voice cloning demo." \
  --reference-audio path/to/voice.wav \
  --output out.wav

# Ultimate cloning (prompt audio + transcript)
voxcpm clone \
  --text "This is a voice cloning demo." \
  --prompt-audio path/to/voice.wav \
  --prompt-text "reference transcript" \
  --reference-audio path/to/voice.wav \ # optional, for better simliarity
  --output out.wav

# Batch processing
voxcpm batch --input examples/input.txt --output-dir outs

# Optional post-generation timestamps with stable-ts
pip install "voxcpm[timestamps]"
voxcpm design \
  --text "VoxCPM2 brings studio-quality multilingual speech synthesis." \
  --output out.wav \
  --timestamps \
  --timestamp-level word \
  --timestamp-language en

# Character timestamps are best-effort and are derived from word alignment
voxcpm design \
  --text "欢迎使用 VoxCPM2。" \
  --output out.wav \
  --timestamps \
  --timestamp-level char \
  --timestamp-language zh

# Help
voxcpm --help

Web Demo Copy Section

python app.py --port 8808  # then open in browser: http://localhost:8808

Use --device to choose the runtime device:

python app.py --device auto

Supported values are auto, cpu, mps, cuda, and cuda:N. On Apple Silicon Macs, auto uses MPS when available.

🚢 Production Deployment (Nano-vLLM) Copy Section

For high-throughput serving, use Nano-vLLM-VoxCPM — a dedicated inference engine built on Nano-vLLM with concurrent request support and an async API.

pip install nano-vllm-voxcpm

from nanovllm_voxcpm import VoxCPM
import numpy as np, soundfile as sf

server = VoxCPM.from_pretrained(model="/path/to/VoxCPM", devices=[0])
chunks = list(server.generate(target_text="Hello from VoxCPM!"))
sf.write("out.wav", np.concatenate(chunks), 48000)
server.stop()

RTF as low as ~0.13 on NVIDIA RTX 4090 (vs ~0.3 with the standard PyTorch implementation), with support for batched concurrent requests and a FastAPI HTTP server. See the Nano-vLLM-VoxCPM repo for deployment details.

🏭 Production Serving (vLLM-Omni) Copy Section

For production multi-tenant deployments, use vLLM-Omni — the official vLLM project's omni-modal extension with native VoxCPM2 support. PagedAttention KV cache, continuous batching, and a drop-in OpenAI-compatible /v1/audio/speech endpoint.

# Install from source (latest main — vllm-omni is rapidly evolving)
uv pip install vllm==0.19.0 --torch-backend=auto
git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni
uv pip install -e .

See the vLLM-Omni installation guide for other platforms (ROCm, XPU, MUSA, NPU) and Docker images.

# Launch an OpenAI-compatible TTS server (--omni enables omni-modal serving)
vllm serve openbmb/VoxCPM2 --omni --port 8000

# Call it from any OpenAI client
curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"openbmb/VoxCPM2","input":"Hello from VoxCPM2 on vLLM-Omni!","voice":"default"}' \
  --output out.wav

Built on the upstream vLLM scheduler, with batched concurrent requests, streaming chunk delivery, and multi-GPU deployment out of the box. See the VoxCPM2 example for full deployment recipes.

📱 On-Device Inference (llama.cpp-omni) Copy Section

For on-device / edge deployment without Python, use llama.cpp-omni — a high-performance C++ inference engine built on llama.cpp, with native VoxCPM2 GGUF support on CPU / Metal / CUDA / Vulkan.

1. Download GGUF weights from HuggingFace | ModelScope — you need one BaseLM (F16 or Q8_0) + the Acoustic file. Q8_0 halves the download with negligible quality loss.

2. Build

git clone https://github.com/tc-mb/llama.cpp-omni.git && cd llama.cpp-omni
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target voxcpm2-cli -j

CMake auto-detects Metal (macOS) or CUDA (Linux with NVIDIA GPU).

3. Run

# Basic TTS
./build/bin/voxcpm2-cli \
    -t "Hello, this is VoxCPM2 running through llama.cpp-omni." \
    -o output.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

# Voice cloning (reference audio)
./build/bin/voxcpm2-cli \
    -t "Cloned voice." -r speaker.wav -o clone.wav \
    VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

# Ultimate cloning (reference audio + transcript)
./build/bin/voxcpm2-cli \
    -t "Target text." --prompt-wav speaker.wav --prompt-text "transcript of speaker.wav" \
    -o clone.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

RTF ~1.76 (Q8_0) on Apple M4 Pro / Metal. Key flags: --cfg (guidance scale), --timesteps (CFM steps), --seed, --temperature, --stream. See the llama.cpp-omni repo and GGUF weights page for full details.

Full parameter reference, multi-scenario examples, and voice cloning tips → Quick Start Guide | Usage Guide | Cookbook

📦 Models & Versions Copy Chapter

VoxCPM2 is built on a tokenizer-free, diffusion autoregressive paradigm. The model operates entirely in the latent space of AudioVAE V2, following a four-stage pipeline: LocEnc → TSLM → RALM → LocDiT, enabling rich expressiveness and 48kHz native audio output.

For full architectural details, VoxCPM2-specific upgrades, and a model comparison table, see the Architecture Design.

📊 Performance Copy Chapter

VoxCPM2 achieves state-of-the-art or comparable results on public zero-shot and controllable TTS benchmarks.

Seed-TTS-eval Copy Section

Seed-TTS-eval WER(⬇)&SIM(⬆) Results (click to expand)

CV3-eval Copy Section

CV3-eval Multilingual WER/CER(⬇) Results (click to expand)

MiniMax-Multilingual-Test Copy Section

Minimax-MLS-test WER(⬇) Results (click to expand)

Minimax-MLS-test SIM(⬆) Results (click to expand)

Internal 30-Language ASR Benchmark Copy Section

We additionally run an internal multilingual intelligibility benchmark with 30 languages × 500 samples. ASR transcription is evaluated via Gemini 3.1 Flash Lite API.

Internal 30-Language ASR Benchmark (click to expand)

InstructTTSEval Copy Section

Instruction-Guided Voice Design Results (click to expand)

⚙️ Fine-tuning Copy Chapter

VoxCPM supports both full fine-tuning (SFT) and LoRA fine-tuning. With as little as 5–10 minutes of audio, you can adapt to a specific speaker, language, or domain.

# LoRA fine-tuning (parameter-efficient, recommended)
python scripts/train_voxcpm_finetune.py \
    --config_path conf/voxcpm_v2/voxcpm_finetune_lora.yaml

# Full fine-tuning
python scripts/train_voxcpm_finetune.py \
    --config_path conf/voxcpm_v2/voxcpm_finetune_all.yaml

# WebUI for training & inference
python lora_ft_webui.py   # then open http://localhost:7860

Full guide → Fine-tuning Guide (data preparation, configuration, training, LoRA hot-swapping, FAQ)

📚 Documentation Copy Chapter

Full documentation: voxcpm.readthedocs.io

🌟 Ecosystem & Community Copy Chapter

See the full Ecosystem in the docs. Community projects are not officially maintained by OpenBMB. Built something cool? Open an issue or PR to add it!

⚠️ Risks and Limitations Copy Chapter

• Potential for Misuse: VoxCPM's voice cloning can generate highly realistic synthetic speech. It is strictly forbidden to use VoxCPM for impersonation, fraud, or disinformation. We strongly recommend clearly marking any AI-generated content.
• Controllable Generation Stability: Voice Design and Controllable Voice Cloning results can vary between runs — you may try to generate 1~3 times to obtain the desired voice or style. We are actively working on improving controllability consistency.
• Language Coverage: VoxCPM2 officially supports 30 languages. For languages not on the list, you are welcome to test directly or try fine-tuning on your own data. We plan to expand language coverage in future releases.
• Usage: This model is released under the Apache-2.0 license. For production deployments, we recommend conducting thorough testing and safety evaluation tailored to your use case.

📖 Citation Copy Chapter

If you find VoxCPM helpful, please consider citing our work and starring ⭐ the repository!

@article{zhou2026voxcpm2,
  title   = {VoxCPM2 Technical Report},
  author  = {Zhou, Yixuan  and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Gui, Jiancheng and Wu, Jiaheng and Wang, Ziyang and Shen, Xudong and Ye, Runchuan  and Zhang, Zhisheng and Zhou, Jiuyang and Bai, Bingsong and Sun, Weiyue and Deng, Mengyuan and Shi, Qundong and Wu, Zhiyong and Liu, Zhiyuan},
  journal = {arXiv preprint arXiv:2606.06928},
  year    = {2026},
}

@article{zhou2025voxcpm,
  title = {Voxcpm: Tokenizer-free TTS for context-aware speech generation and true-to-life voice cloning},
  author = {Zhou, Yixuan and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Wang, Ziyang and Ye, Runchuan and Sun, Weiyue and Gui, Jiancheng and Li, Kehan and Wu, Zhiyong and Liu, Zhiyuan},
  journal = {arXiv preprint arXiv:2509.24650},
  year = {2025}
}

📄 License Copy Chapter

VoxCPM model weights and code are open-sourced under the Apache-2.0 license.

🙏 Acknowledgments Copy Chapter

• DiTAR for the diffusion autoregressive backbone
• MiniCPM-4 for the language model foundation
• CosyVoice for the Flow Matching-based LocDiT implementation
• DAC for the Audio VAE backbone
• Our community users for trying VoxCPM, reporting issues, sharing ideas, and contributing—your support helps the project keep getting better

Institutions Copy Chapter

ModelBest     THUHCSI

⭐ Star History Copy Chapter

Star History Chart

File: README_zh.md Copy Chapter

VoxCPM2：基于连续表征的多语言语音合成、创意音色设计与高保真声音克隆

VoxCPM 是一个无离散音频分词器（Tokenizer-Free）的语音合成系统，通过端到端的扩散自回归架构直接生成连续语音表征，绕过对音频的离散编码步骤，实现高度自然且富有表现力的语音合成。

VoxCPM2 是最新的版本 — 基于 MiniCPM-4 基座构建，总计 20亿 参数，在超过 200万小时 的多语种音频数据上训练，支持 30种全球语言+9种中文方言、音色设计、可控声音克隆，原生输出 48kHz 高质量音频。

✨ 核心特性 Copy Section

• 🌍 30种语言语音合成 — 直接输入原始文本即可合成（支持语言详见下文），无需额外语言标签
• 🎨 音色设计 — 用自然语言描述（性别、年龄、音色、情绪、语速……）凭空创建全新音色，无需参考音频
• 🎛️ 可控声音克隆 — 从参考音频片段克隆任意声音，可叠加风格指令控制情绪、语速和表现力，同时保持原始音色
• 🎙️ 极致克隆 — 提供参考音频及其文本内容，模型接着参考音频进行无缝续写，从而精准还原声音细节特征（与 VoxCPM1.5 一致）
• 🔊 48kHz 高质量音频 — 输入 16kHz 参考音频，通过 AudioVAE V2 的非对称编解码设计直接输出 48kHz 高质量音频，内置超分能力
• 🧠 语境感知合成 — 根据文本内容自动推断合适的韵律和表现力
• ⚡ 实时流式合成 — 在 NVIDIA RTX 4090 上 RTF 低至 ~0.3，通过 Nano-vLLM 或 vLLM-Omni（官方 vLLM 全模态服务，原生支持 VoxCPM2，提供 PagedAttention 与 OpenAI 兼容 API）加速后可达 ~0.13
• 📜 完全开源，商用就绪 — 权重和代码基于 Apache-2.0 协议发布，免费商用

🌍 支持的语言（30种）

阿拉伯语、缅甸语、中文、丹麦语、荷兰语、英语、芬兰语、法语、德语、希腊语、希伯来语、印地语、印尼语、意大利语、日语、高棉语、韩语、老挝语、马来语、挪威语、波兰语、葡萄牙语、俄语、西班牙语、斯瓦希里语、瑞典语、菲律宾语、泰语、土耳其语、越南语

中国方言：四川话、粤语、吴语、东北话、河南话、陕西话、山东话、天津话、闽南话

最新动态 Copy Section

• [2026.04] 🔥 发布 VoxCPM2 — 20亿参数，30种语言，音色设计与可控声音克隆，48kHz 音频输出！模型权重 | 使用文档 | 在线体验 | 官网体验 (适用国内访问) | 技术报告
• [2025.12] 🎉 开源 VoxCPM1.5 模型权重，支持 SFT 和 LoRA 微调。(🏆 GitHub Trending #1)
• [2025.09] 🔥 发布 VoxCPM 技术报告。
• [2025.09] 🎉 开源 VoxCPM-0.5B 模型权重 (🏆 HuggingFace Trending #1)

目录 Copy Chapter

• 快速开始
  • 安装
  • Python API
  • 命令行使用
  • Web Demo
  • 生产部署
  • 端侧推理（llama.cpp-omni）
• 模型与版本
• 性能评测
• 微调
• 文档
• 生态与社区
• 风险与局限性
• 引用

🚀 快速开始 Copy Chapter

安装 Copy Section

pip install voxcpm

环境要求： Python ≥ 3.10 (<3.13)，PyTorch ≥ 2.5.0，CUDA ≥ 12.0。详见 快速开始文档。

Python API Copy Section

🗣️ 文本转语音

from voxcpm import VoxCPM
import soundfile as sf

model = VoxCPM.from_pretrained(
  "openbmb/VoxCPM2",
  load_denoiser=False,
)

wav = model.generate(
    text="VoxCPM2 是目前推荐使用的多语言语音合成版本。",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)
print("已保存: demo.wav")

如果你希望先从 ModelScope 下载模型到本地（适用于国内网络访问），可以使用：

pip install modelscope

from modelscope import snapshot_download
snapshot_download("OpenBMB/VoxCPM2", local_dir='./pretrained_models/VoxCPM2') # 指定模型保存的本地路径

from voxcpm import VoxCPM
import soundfile as sf
model = VoxCPM.from_pretrained('./pretrained_models/VoxCPM2', load_denoiser=False)

wav = model.generate(
    text="VoxCPM2 是目前推荐使用的多语言语音合成版本。",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("demo.wav", wav, model.tts_model.sample_rate)

🎨 音色设计

用自然语言描述创建全新音色，无需参考音频。格式： 在 text 开头用括号写入音色描述（如 "(音色描述)要合成的文本。"）：

wav = model.generate(
    text="(年轻女性，声音温柔甜美)你好，欢迎使用VoxCPM2！",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("voice_design.wav", wav, model.tts_model.sample_rate)

🎛️ 可控声音克隆

上传一段参考音频，模型克隆其音色，同时可以使用控制指令调节语速、情绪或风格。

wav = model.generate(
    text="这是VoxCPM2生成的克隆语音。",
    reference_wav_path="path/to/voice.wav",
)
sf.write("clone.wav", wav, model.tts_model.sample_rate)

wav = model.generate(
    text="(稍快一点，欢快的语气)这是带风格控制的克隆语音。",
    reference_wav_path="path/to/voice.wav",
    cfg_value=2.0,
    inference_timesteps=10,
    seed=42,
)
sf.write("controllable_clone.wav", wav, model.tts_model.sample_rate)

🎙️ 极致克隆

提供参考音频及其精确文本转录，实现基于音频续写的高保真克隆。为获得最高克隆相似度，可将同一音频同时传给 reference_wav_path 和 prompt_wav_path：

wav = model.generate(
    text="这是使用VoxCPM2的极致克隆演示。",
    prompt_wav_path="path/to/voice.wav",
    prompt_text="参考音频的文本转录。",
    reference_wav_path="path/to/voice.wav",  # 可选，提升相似度
)
sf.write("hifi_clone.wav", wav, model.tts_model.sample_rate)

🔄 流式 API

import numpy as np

chunks = []
for chunk in model.generate_streaming(
    text="使用VoxCPM进行流式语音合成非常简单！",
):
    chunks.append(chunk)
wav = np.concatenate(chunks)
sf.write("streaming.wav", wav, model.tts_model.sample_rate)

命令行使用 Copy Section

# 音色设计（无需参考音频）
voxcpm design \
  --text "VoxCPM2带来全新语音合成体验。" \
  --output out.wav

# 可控声音克隆（带风格控制）
voxcpm design \
  --text "VoxCPM2带来全新语音合成体验。" \
  --control "年轻女声，温暖温柔，略带微笑" \
  --seed 42 \
  --output out.wav

# 声音克隆（参考音频）
voxcpm clone \
  --text "这是一个声音克隆的演示。" \
  --reference-audio path/to/voice.wav \
  --output out.wav

# 极致克隆（提示音频 + 转录文本）
voxcpm clone \
  --text "这是一个声音克隆的演示。" \
  --prompt-audio path/to/voice.wav \
  --prompt-text "参考音频转录文本" \
  --reference-audio path/to/voice.wav \
  --output out.wav

# 批量处理
voxcpm batch --input examples/input.txt --output-dir outs

# 可选的生成后时间戳对齐（基于 stable-ts）
pip install "voxcpm[timestamps]"
voxcpm design \
  --text "VoxCPM2带来全新语音合成体验。" \
  --output out.wav \
  --timestamps \
  --timestamp-level word \
  --timestamp-language zh

# 字级时间戳是 best-effort，会基于词级对齐结果拆分
voxcpm design \
  --text "欢迎使用 VoxCPM2。" \
  --output out.wav \
  --timestamps \
  --timestamp-level char \
  --timestamp-language zh

# 帮助
voxcpm --help

Web Demo Copy Section

python app.py --port 8808  # 然后在浏览器打开 http://localhost:8808

使用 --device 选择运行设备：

python app.py --device auto

支持的取值包括 auto、cpu、mps、cuda 和 cuda:N。在 Apple Silicon Mac 上，auto 会在可用时使用 MPS。

🚢 生产部署（Nano-vLLM） Copy Section

如需高吞吐量部署，使用 Nano-vLLM-VoxCPM — 基于 Nano-vLLM 构建的专用推理引擎，支持并发请求和异步 API。

pip install nano-vllm-voxcpm

from nanovllm_voxcpm import VoxCPM
import numpy as np, soundfile as sf

server = VoxCPM.from_pretrained(model="/path/to/VoxCPM", devices=[0])
chunks = list(server.generate(target_text="你好，我来自VoxCPM！"))
sf.write("out.wav", np.concatenate(chunks), 48000)
server.stop()

在 NVIDIA RTX 4090 上 RTF 低至 ~0.13（标准 PyTorch 实现约 ~0.3），支持批量并发请求和 FastAPI HTTP 服务。详见 Nano-vLLM-VoxCPM 仓库。

🏭 生产环境部署（vLLM-Omni） Copy Section

如需生产级多租户部署，使用 vLLM-Omni — 官方 vLLM 项目的全模态扩展，原生支持 VoxCPM2。具备 PagedAttention KV 缓存、连续批处理，以及与 OpenAI 完全兼容的 /v1/audio/speech 接口。

# 从源码安装（最新 main 分支 —— vllm-omni 正在快速迭代）
uv pip install vllm==0.19.0 --torch-backend=auto
git clone https://github.com/vllm-project/vllm-omni.git && cd vllm-omni
uv pip install -e .

其他平台（ROCm、XPU、MUSA、NPU）与 Docker 镜像请参考 vLLM-Omni 安装文档。

# 启动 OpenAI 兼容的 TTS 服务（--omni 启用全模态服务）
vllm serve openbmb/VoxCPM2 --omni --port 8000

# 任意 OpenAI 客户端均可调用
curl http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model":"openbmb/VoxCPM2","input":"你好，欢迎使用 VoxCPM2 on vLLM-Omni！","voice":"default"}' \
  --output out.wav

基于上游 vLLM 调度器构建，开箱即用支持批量并发、流式分块输出和多 GPU 部署。完整示例见 VoxCPM2 部署样例。

📱 端侧推理（llama.cpp-omni） Copy Section

如需在端侧/消费级硬件上无 Python 运行，使用 llama.cpp-omni — 基于 llama.cpp 的高性能 C++ 推理引擎，原生支持 VoxCPM2 GGUF，可在 CPU / Metal / CUDA / Vulkan 上运行。

1. 下载 GGUF 权重：从 HF下载 | ModelScope，需要一个 BaseLM（F16 或 Q8_0）+ Acoustic 文件。Q8_0 体积减半，质量损失可忽略。

2. 编译

git clone https://github.com/tc-mb/llama.cpp-omni.git && cd llama.cpp-omni
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build --target voxcpm2-cli -j

CMake 会自动检测并启用 Metal（macOS）或 CUDA（Linux + NVIDIA GPU）。

3. 运行

# 基础 TTS
./build/bin/voxcpm2-cli \
    -t "你好，我是通过 llama.cpp-omni 运行的 VoxCPM2。" \
    -o output.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

# 声音克隆（参考音频）
./build/bin/voxcpm2-cli \
    -t "克隆的声音。" -r speaker.wav -o clone.wav \
    VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

# 精准克隆（参考音频 + 转写文本）
./build/bin/voxcpm2-cli \
    -t "目标文本。" --prompt-wav speaker.wav --prompt-text "参考音频的转写文本" \
    -o clone.wav VoxCPM2-BaseLM-Q8_0.gguf VoxCPM2-Acoustic-F16.gguf

在 Apple M4 Pro / Metal 上 RTF ~1.76（Q8_0）。 主要参数：--cfg（引导尺度）、--timesteps（CFM 步数）、--seed、--temperature、--stream。详见 llama.cpp-omni 仓库 和 GGUF 权重页面。

完整参数说明、多场景示例与声音克隆技巧 → 快速开始指南 | 使用指南 | Cookbook

📦 模型与版本 Copy Chapter

VoxCPM2 采用连续音频表征、扩散自回归范式，模型在 AudioVAE 的连续隐空间中通过四阶段处理：LocEnc → TSLM → RALM → LocDiT，实现丰富的表现力语音合成和 48kHz 原生音频输出。

完整架构细节、VoxCPM2 升级内容和模型对比表见 架构设计文档。

📊 性能评测 Copy Chapter

VoxCPM2 在公开的零样本和可控 TTS 基准测试中取得了 SOTA 或可比的结果。

Seed-TTS-eval Copy Section

Seed-TTS-eval WER(⬇)&SIM(⬆) 结果（点击展开）

CV3-eval Copy Section

CV3-eval 多语言 WER/CER(⬇) 结果（点击展开）

MiniMax-Multilingual-Test Copy Section

Minimax-MLS-test WER(⬇) 结果（点击展开）

Minimax-MLS-test SIM(⬆) 结果（点击展开）

Internal 30-Language ASR Benchmark Copy Section

我们额外进行了内部多语言可懂度评测：30 语种 × 500 样本，ASR 转写评估使用 Gemini 3.1 Flash Lite API。

内部30语种评测集ASR结果（点击展开）

InstructTTSEval Copy Section

指令驱动音色设计结果 (点击展开)

⚙️ 微调 Copy Chapter

VoxCPM 支持全参数微调（SFT） 和 LoRA 微调。仅需 5-10分钟 的音频数据，即可适配特定说话人、语言或领域。

# LoRA 微调（参数高效，推荐）
python scripts/train_voxcpm_finetune.py \
    --config_path conf/voxcpm_v2/voxcpm_finetune_lora.yaml

# 全参数微调
python scripts/train_voxcpm_finetune.py \
    --config_path conf/voxcpm_v2/voxcpm_finetune_all.yaml

# WebUI 训练与推理
python lora_ft_webui.py   # 然后打开 http://localhost:7860

完整指南 → 微调文档（数据准备、配置、训练、LoRA 热切换、常见问题）

📚 文档 Copy Chapter

完整文档：voxcpm.readthedocs.io

🌟 生态与社区 Copy Chapter

完整生态见文档。社区项目非 OpenBMB 官方维护。做了什么有趣的东西？提 Issue 或 PR 把它加进来！

⚠️ 风险与局限性 Copy Chapter

• 滥用风险： VoxCPM 的声音克隆能力可生成高度逼真的合成语音。严禁将 VoxCPM 用于冒充他人、欺诈或虚假信息传播。我们强烈建议对所有 AI 生成的内容进行明确标注。
• 可控生成稳定性： 音色设计和可控声音克隆的结果可能因生成次数而异 — 建议尝试生成 1~3 次以获得理想的音色或风格。我们正在积极提升可控性的一致性。
• 语言覆盖： VoxCPM2 官方支持 30 种语言。对于未列入的语言，欢迎直接测试或使用自有数据进行微调。我们计划在未来版本中扩展语言覆盖。
• 使用说明： 本模型基于 Apache-2.0 协议发布。用于生产部署时，我们建议针对具体场景进行充分的测试和安全评估。

📖 引用 Copy Chapter

如果 VoxCPM 对您有帮助，请考虑引用我们的工作并为仓库加星 ⭐！

@article{zhou2026voxcpm2,
  title   = {VoxCPM2 Technical Report},
  author  = {Zhou, Yixuan  and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Gui, Jiancheng and Wu, Jiaheng and Wang, Ziyang and Shen, Xudong and Ye, Runchuan  and Zhang, Zhisheng and Zhou, Jiuyang and Bai, Bingsong and Sun, Weiyue and Deng, Mengyuan and Shi, Qundong and Wu, Zhiyong and Liu, Zhiyuan},
  journal = {arXiv preprint arXiv:2606.06928},
  year    = {2026},
}

@article{zhou2025voxcpm,
  title = {Voxcpm: Tokenizer-free TTS for context-aware speech generation and true-to-life voice cloning},
  author = {Zhou, Yixuan and Zeng, Guoyang and Liu, Xin and Li, Xiang and Yu, Renjie and Wang, Ziyang and Ye, Runchuan and Sun, Weiyue and Gui, Jiancheng and Li, Kehan and Wu, Zhiyong and Liu, Zhiyuan},
  journal = {arXiv preprint arXiv:2509.24650},
  year = {2025}
}

📄 许可证 Copy Chapter

VoxCPM 模型权重和代码基于 Apache-2.0 协议开源。

🙏 致谢 Copy Chapter

• DiTAR 扩散自回归骨干架构
• MiniCPM-4 语言模型基座
• CosyVoice 基于 Flow Matching 的 LocDiT 实现
• DAC Audio VAE 骨干
• 感谢所有社区用户试用 VoxCPM、反馈问题、分享想法和贡献——你们的支持让项目持续进步

机构 Copy Chapter

面壁智能
   
清华大学人机交互实验室

⭐ Star 历史 Copy Chapter