## File: README.md # Pocket TTS A lightweight text-to-speech (TTS) application designed to run efficiently on CPUs. Forget about the hassle of using GPUs and web APIs serving TTS models. With Kyutai's Pocket TTS, generating audio is just a pip install and a function call away. Supports Python 3.10, 3.11, 3.12, 3.13 and 3.14. Requires PyTorch 2.5+. Does not require the gpu version of PyTorch. [🔊 Demo](https://kyutai.org/pocket-tts) | [🐱‍💻GitHub Repository](https://github.com/kyutai-labs/pocket-tts) | [🤗 Hugging Face Model Card](https://huggingface.co/kyutai/pocket-tts) | [⚙️ Tech report](https://kyutai.org/blog/2026-01-13-pocket-tts) | [📄 Paper](https://arxiv.org/abs/2509.06926) | [📚 Documentation](https://kyutai-labs.github.io/pocket-tts/) ## Main takeaways * Runs on CPU * Small model size, 100M parameters * Audio streaming * Low latency, ~200ms to get the first audio chunk * Faster than real-time, ~6x real-time on a CPU of MacBook Air M4 * Uses only 2 CPU cores * Python API and CLI * Voice cloning * Multi-language support: english, french, german, portuguese, italian, spanish * Can handle infinitely long text inputs * [Can run on client-side in the browser](#in-browser-implementations) Additional languages may be added in the future. ## Trying it from the website, without installing anything Navigate to the [Kyutai website](https://kyutai.org/pocket-tts) to try it out directly in your browser. You can input text, select different voices, and generate speech without any installation. ## Trying it with the CLI ### The `generate` command You can use pocket-tts directly from the command line. We recommend using `uv` as it installs any dependencies on the fly in an isolated environment (uv installation instructions [here](https://docs.astral.sh/uv/getting-started/installation/#standalone-installer)). You can also use `pip install pocket-tts` to install it manually. This will generate a wav file `./tts_output.wav` saying the default text with the default voice, and display some speed statistics. ```bash uvx pocket-tts generate # or if you installed it manually with pip: pocket-tts generate ``` Modify the voice with `--voice` and the text with `--text`. We provide a small catalog of voices. Choose a pretrained language model with `--language` when running `generate`, `export-voice`, or `serve` (default: `english`). Non-english languages have also biggers 24 layers variants that are higher quality but slower. You can select them by using for example `--language italian_24l`. The `--config` option accepts only a local YAML path for custom weights. You can take a look at [this page](https://huggingface.co/kyutai/tts-voices) which details the licenses for each voice. * [alba](https://huggingface.co/kyutai/tts-voices/blob/main/alba-mackenna/casual.wav) (en) * [giovanni](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/common_voice_it_36520747-enhanced-v2.mp3) (it) * [lola](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/common_voice_es_19762977-enhanced-v2.mp3) (es) * [juergen](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/de-DE-juergen.mp3) (de) * [rafael](https://huggingface.co/kyutai/pocket-tts/blob/add_lang_not_documented/g-Vi8PgmSY0-enhanced-v2.wav) (pt) * [estelle](https://huggingface.co/kyutai/tts-voices/blob/main/unmute-prod-website/developpeuse-3.wav) (fr) * [anna](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p228_023_enhanced.wav) (en) * [azelma](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p303_023_enhanced.wav) (en) * [bill_boerst](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/bill_boerst.wav) (en) * [caro_davy](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/caro_davy.wav) (en) * [charles](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p254_023_enhanced.wav) (en) * [cosette](https://huggingface.co/kyutai/tts-voices/blob/main/expresso/ex04-ex02_confused_001_channel1_499s.wav) (en) * [eponine](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p262_023_enhanced.wav) (en) * [eve](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p361_023_enhanced.wav) (en) * [fantine](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p244_023_enhanced.wav) (en) * [george](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p315_023_enhanced.wav) (en) * [jane](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p339_023_enhanced.wav) (en) * [jean](https://huggingface.co/kyutai/tts-voices/blob/main/ears/p010/freeform_speech_01_enhanced.wav) (en) * [javert](https://huggingface.co/kyutai/tts-voices/blob/main/voice-donations/Butter.wav) (en) * [marius](https://huggingface.co/kyutai/tts-voices/blob/main/voice-donations/Selfie.wav) (en) * [mary](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p333_023_enhanced.wav) (en) * [michael](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p360_023_enhanced.wav) (en) * [paul](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p259_023_enhanced.wav) (en) * [peter_yearsley](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/peter_yearsley.wav) (en) * [stuart_bell](https://huggingface.co/kyutai/tts-voices/blob/main/voice-zero/stuart_bell.wav) (en) * [vera](https://huggingface.co/kyutai/tts-voices/blob/main/vctk/p229_023_enhanced.wav) (en) The `--voice` argument can also take a plain wav file as input for voice cloning. You can use your own or check out our [voice repository](https://huggingface.co/kyutai/tts-voices). We recommend [cleaning the sample](https://podcast.adobe.com/en/enhance) before using it with Pocket TTS, because the audio quality of the sample is also reproduced. Feel free to check out the [generate documentation](https://kyutai-labs.github.io/pocket-tts/CLI%20Commands/generate/) for more details and examples. For trying multiple voices and prompts quickly, prefer using the `serve` command. ### The `serve` command You can also run a local server to generate audio via HTTP requests. ```bash uvx pocket-tts serve # or if you installed it manually with pip: pocket-tts serve ``` Navigate to `http://localhost:8000` to try the web interface, it's faster than the command line as the model is kept in memory between requests. You can check out the [serve documentation](https://kyutai-labs.github.io/pocket-tts/CLI%20Commands/serve/) for more details and examples. ### The `export-voice` command Processing an audio file (e.g., a .wav or .mp3) for voice cloning is relatively slow, but loading a safetensors file -- a voice embedding converted from an audio file -- is very fast. You can use the `export-voice` command to do this conversion. See the [export-voice documentation](https://kyutai-labs.github.io/pocket-tts/CLI%20Commands/export_voice/) for more details and examples. ## Using it as a Python library You can try out the Python library on Colab [here](https://colab.research.google.com/github/kyutai-labs/pocket-tts/blob/main/docs/pocket-tts-example.ipynb). Install the package with ```bash pip install pocket-tts # or uv add pocket-tts ``` You can use this package as a simple Python library to generate audio from text. ```python from pocket_tts import TTSModel import scipy.io.wavfile tts_model = TTSModel.load_model() voice_state = tts_model.get_state_for_audio_prompt( "alba" # One of the pre-made voices, see above # You can also use any voice file you have locally or from Hugging Face: # "./some_audio.wav" # or "hf://kyutai/tts-voices/expresso/ex01-ex02_default_001_channel2_198s.wav" ) audio = tts_model.generate_audio(voice_state, "Hello world, this is a test.") # Audio is a 1D torch tensor containing PCM data. scipy.io.wavfile.write("output.wav", tts_model.sample_rate, audio.numpy()) ``` You can have multiple voice states around if you have multiple voices you want to use. `load_model()` and `get_state_for_audio_prompt()` are relatively slow operations, so we recommend to keep the model and voice states in memory if you can. For faster voice loading, you can export voice states to safetensors files: ```python from pocket_tts import TTSModel, export_model_state model = TTSModel.load_model() # Export a voice state for fast loading later model_state = model.get_state_for_audio_prompt("some_voice.wav") export_model_state(model_state, "./some_voice.safetensors") # Later, load it quickly, this is quite fast as it's just reading the kvcache # from disk and doesn't do any others computations. model_state_copy = model.get_state_for_audio_prompt("./some_voice.safetensors") audio = model.generate_audio(model_state_copy, "Hello world!") ``` You can check out the [Python API documentation](https://kyutai-labs.github.io/pocket-tts/API%20Reference/python-api/) for more details and examples. ## Running on GPU Pocket TTS is designed to run on CPU, and on hardware with strong single-thread CPU performance (e.g. Apple Silicon) we did not observe a GPU speedup, notably because we use a batch size of 1 and a very small model. However, this turns out to be hardware-dependent: measured on a cloud x86 VM (4 vCPUs) with a Tesla T4, moving the model to GPU gave a consistent ~2.6x speedup over CPU (RTF ~2.3-2.5x on CPU vs. ~6.28x on GPU, for both short and long input text). If your CPU is thread-limited or otherwise weaker than a modern laptop chip, it's worth trying the GPU. This is not officially supported (there is no `device` argument on `TTSModel.load_model()`), but since `TTSModel` is a regular `nn.Module` you can move it yourself: ```python tts_model = TTSModel.load_model() tts_model.to("cuda") ... audio = tts_model.generate_audio(voice_state, "Hello world, this is a test.") # generate_audio() returns a tensor on the same device as the model, so on GPU you need # to move it back to CPU before calling .numpy(): scipy.io.wavfile.write("output.wav", tts_model.sample_rate, audio.detach().cpu().numpy()) ``` A few things to be aware of if you want to use the GPU: - The `generate` CLI command has a `--device` option (defaults to `cpu`, documented in the [CLI reference](docs/CLI%20Commands/generate.md) — note that page's own description ("you may not get a speedup by using a gpu since it's a small model") is what this section is correcting, based on the T4 measurements above); the `serve` command and the Docker image do not expose any device option and will always run on CPU. - `pip install pocket-tts` / `uv add pocket-tts` install whatever `torch` build is current on PyPI, which may require a newer CUDA version than your driver supports. In that case `torch.cuda.is_available()` silently returns `False` (you'll only see a `UserWarning` about an outdated driver, not an error). If this happens, install a `torch` build matching your driver's CUDA version explicitly, e.g. `pip install torch --index-url https://download.pytorch.org/whl/cu121`. - `quantize=True` (int8 dynamic quantization) only works on CPU; calling it on a model moved to CUDA raises `NotImplementedError: Could not run 'quantized::linear_dynamic' ... 'CUDA' backend`. Separately, the optional `torchao` backend (`pip install pocket-tts[quantize]`) declares `torch>=2.11` — fine with a fresh install (torch 2.11+ is on PyPI as of this writing), but if you've pinned an older `torch` (e.g. to match an older GPU driver's CUDA build, per the point above), adding this extra can pull in a `torchao` that's incompatible with your pinned `torch` and break `quantize=True` even on CPU. Match `torchao`'s `torch` requirement to whatever `torch` you actually have installed. ## Unsupported features At the moment, we do not support (but would love pull requests adding): - [Adding silence in the text input to generate pauses.](https://github.com/kyutai-labs/pocket-tts/issues/6) We tried running this TTS model on the GPU but did not observe a speedup compared to CPU execution on hardware with very strong single-thread CPU performance, notably because we use a batch size of 1 and a very small model. See the ["Running on GPU"](#running-on-gpu) section above for measurements on other hardware and caveats if you want to try it yourself. ## Development and local setup We accept contributions! Feel free to open issues or pull requests on GitHub. You can find development instructions in the [CONTRIBUTING.md](https://github.com/kyutai-labs/pocket-tts/tree/main/CONTRIBUTING.md) file. You'll also find there how to have an editable install of the package for local development. ## In-browser implementations Pocket TTS is small enough to run directly in your browser in WebAssembly/JavaScript. We don't have official support for this yet, but you can try out one of these community implementations: - [wasm-pocket-tts](https://github.com/LaurentMazare/xn/tree/main/wasm-pocket-tts) by @LaurentMazare: Rust port of pocket TTS with XN. Demo [here](https://laurentmazare.github.io/pocket-tts/) - [pocket-tts-onnx-export](https://github.com/KevinAHM/pocket-tts-onnx-export) by @KevinAHM: Model exported to .onnx and run using [ONNX Runtime Web](https://onnxruntime.ai/docs/tutorials/web/). Demo [here](https://huggingface.co/spaces/KevinAHM/pocket-tts-web) - [pocket-tts](https://github.com/babybirdprd/pocket-tts) by @babybirdprd: Candle version (Rust) with WebAssembly and PyO3 bindings, meaning it can run on the web too. - [jax-js](https://github.com/ekzhang/jax-js/tree/main/website/src/routes/tts) by @ekzhang: Using jax-js, a ML library for the web. Demo [here](https://jax-js.com/tts) ## Alterative implementations - [pocket-tts-mlx](https://github.com/jishnuvenugopal/pocket-tts-mlx) by @jishnuvenugopal - MLX backend optimized for Apple Silicon - [pocket-tts-xn](https://github.com/LaurentMazare/xn/tree/main/pocket-tts) by @LaurentMazare - A Rust port of Pocket TTS implemented with XN. - [pocket-tts-candle](https://github.com/babybirdprd/pocket-tts) by @babybirdprd - Candle version (Rust) with WebAssembly and PyO3 bindings. - [PocketTTS.cpp](https://github.com/VolgaGerm/PocketTTS.cpp) by @VolgaGerm - Single-file C++ runtime using ONNX Runtime, with CLI, HTTP server, and FFI C API. - [sherpa-onnx](https://github.com/k2-fsa/sherpa-onnx) by @csukuangfj - Run PocketTTS on **Windows, macOS, Linux**, and embedded boards (Raspberry Pi, Jetson, RK3588, etc.) with bindings for 12 programming languages: **C++, C, Python, JavaScript, Java, C#, Kotlin, Swift, Go, Dart, Rust, Pascal**, plus [WebAssembly](https://huggingface.co/spaces/k2-fsa/web-assembly-en-tts-pocket). - [pocket-tts-csharp](https://github.com/TheAjaykrishnanR/pocket-tts-csharp) by @TheAjaykrishnanR - A C# port of Pocket TTS implemented using [TorchSharp](https://github.com/dotnet/TorchSharp) and [TorchSharp.PyBridge](https://github.com/shaltielshmid/TorchSharp.PyBridge) for ease of use as a library in .NET projects. ## Projects using Pocket TTS - [pocket-reader](https://github.com/lukasmwerner/pocket-reader) by @lukasmwerner- Browser screen reader - [pocket-tts-wyoming](https://github.com/ikidd/pocket-tts-wyoming) by @ikidd - Docker container for pocket-tts using Wyoming protocol, ready for Home Assistant Voice use. - [Sonorus](https://www.nexusmods.com/hogwartslegacy/mods/2409) by @KevinAHM - Talk to any named character in Hogwarts Legacy with their original voice. - [Native macOS App](https://github.com/slaughters85j/pocket-tts-macos) by @slaughters85j - Native macOS app, Python-free. Runs Pocket-TTS via Core ML, fully on-device. Includes signed and notarized .app releases. - [Electron macOS App](https://github.com/slaughters85j/pocket-tts) by @slaughters85j - Electron Mac Desktop App + macOS Quick Action - [pocket-tts-openai_streaming_server](https://github.com/teddybear082/pocket-tts-openai_streaming_server) by @teddybear082 - OpenAI-compatible streaming server, dockerized and with an `.exe` release - [pocket-tts-unity](https://github.com/lookbe/pocket-tts-unity) by @lookbe - A Unity 6 integration for Pocket-TTS. - [ComfyUI-Pocket-TTS](https://github.com/ai-joe-git/ComfyUI-Pocket-TTS) by @ai-joe-git Lightweight CPU-based Text-to-Speech for ComfyUI - [pocket-tts-server](https://github.com/ai-joe-git/pocket-tts-server) by @ai-joe-git A lightweight, real-time voice cloning and chat server with OpenAI-compatible API. Clone any voice with just 20 seconds of audio and chat with AI using that voice instantly. - [discord-tts](https://github.com/alkmei/discord-tts) by @alkmei - Multivoice Discord text-to-speech bot that uses Pocket TTS. - [cursed-codex](https://github.com/dooart/cursed-codex) by @dooart - AI coding agent with unhinged live football commentary - [pocket-tts-deno](https://github.com/ohmstone/pocket-tts-deno) Port of [pocket-tts-server](https://github.com/ai-joe-git/pocket-tts-server) as a wasm + onnx deno server with voice TTS API. - [FrontPocket](https://github.com/markd89/FrontPocket) by @markd89 - Front-end for Pocket-TTS to speak text from clipboard, file, CLI (hotkeys) & GUI toolbar. Change playback speed, voice, and move forward/backward between sentences instantaneously. - [openclaw-pockettts](https://github.com/dodgyrabbit/openclaw-pockettts) by @dodgyrabbit - A Docker container with the Python implementation but exposed as an OpenAI TTS API for easy integration with OpenClaw. - [openclaw-pocketts.cpp](https://github.com/dodgyrabbit/openclaw-pockettts.cpp) by @dodgyrabbit - A Docker container with the PocketTTS.cpp version, packaged for easy integration with OpenClaw. - [tts-audiobook-tool](https://github.com/zeropointnine/tts-audiobook-tool) by @zeropointnine - Multi-model audiobook generator with automatic error detection, 48khz upscaling, synced browser reader, stand-alone server-mode. - [seshat-tts](https://github.com/scriptriva/seshat-tts) by @scriptriva - Accessibility tool that provides real-time audio synthesis for games and apps. It also features a voice manager capable of cloning voices based on user presets. - [LocalVocal.ai](https://localvocal.ai) by @joshwhiton - Fully local conversational voice-harness for Macs with Apple Silicon. Includes voice-activity & turn detection, dictation, voice cloning, CLI to talk to Claude, Codex... and more. ## Prohibited use Use of our model must comply with all applicable laws and regulations and must not result in, involve, or facilitate any illegal, harmful, deceptive, fraudulent, or unauthorized activity. Prohibited uses include, without limitation, voice impersonation or cloning without explicit and lawful consent; misinformation, disinformation, or deception (including fake news, fraudulent calls, or presenting generated content as genuine recordings of real people or events); and the generation of unlawful, harmful, libelous, abusive, harassing, discriminatory, hateful, or privacy-invasive content. We disclaim all liability for any non-compliant use. ## Authors Manu Orsini*, Simon Rouard*, Gabriel De Marmiesse*, Václav Volhejn, Neil Zeghidour, Alexandre Défossez *equal contribution --- ## File: docs/CLI Commands/export_voice.md # Export Voice Kyutai Pocket TTS allows you to generate speech with voice cloning from an audio sample. However, processing an audio file each time is relatively slow and inefficient. The `export-voice` command allows you to convert an audio file to a voice embedding (it's actually the kvcache) in safetensors format. The safetensors file can then be loaded very quickly whenever you generate speech. ## Basic Usage ```bash uvx pocket-tts export-voice audio-path export-path # or if installed manually: pocket-tts export-voice audio-path export-path ``` Only the first 30 seconds of the audio file will be processed. ## Command Options ### Required Parameters - `audio-path`: Path of the audio file to convert. `audio-path` can point to an `http:` or `hf:` (hugging face) file. Supports popular audio file formats like wav and mp3. - `export-path`: Path of the output safetensors file to write. ### Options - `--quiet`: Do not print any output except errors. - `--language`: Language for the TTS model, one of `'english_2026-01'`, `'english_2026-04'`, `'english'`, `'french_24l'`, `'german_24l'`, `'portuguese_24l'`, `'italian_24l'`, `'spanish_24l'` (default: `english`, which is the same model as `'english_2026-04'`). Incompatible with `--config`. The "24l" variants are bigger models, not distilled yet and here only as preview. - `--config`: Model config local yaml path. Incompatible with `--language`. ## Examples ```bash # export a single file pocket-tts export-voice voice_memo127762.mp3 jack.safetensors # export an online file to current directory pocket-tts export-voice https://huggingface.co/kyutai/tts-voices/resolve/main/alba-mackenna/announcer.wav ./announcer.safetensors # use the exported safetensors pocket-tts generate --text "Hello, welcome to today's game between the Bears and Cubs." --voice announcer.safetensors ``` --- ## File: docs/CLI Commands/generate.md # Generate The `generate` command allows you to generate speech from text directly from the command line using Kyutai Pocket TTS. ## Basic Usage ```bash uvx pocket-tts generate # or if installed manually: pocket-tts generate ``` This will generate a WAV file `./tts_output.wav` with the default text and voice. ## Command Options ### Core Options - `--text TEXT`: Text to generate (default: "Hello world! I am Kyutai Pocket TTS. I'm fast enough to run on small CPUs. I hope you'll like me.") - `--voice VOICE`: Path to audio conditioning file (voice to clone). Defaults to a built-in voice chosen from the language: `giovanni` (it) for italian, `lola` (es) for spanish, `juergen` (de) for german, `rafael` (pt) for portuguese, `estelle` (fr) for french, `alba` (en) otherwise. Urls and local paths are supported. - `--output-path OUTPUT_PATH`: Output path for generated audio (default: "./tts_output.wav") - `--language LANGUAGE`: Language for the TTS model, one of `'english_2026-01'`, `'english_2026-04'`, `'english'`, `'french_24l'`, `'german'`, `'german_24l'`, `'portuguese'`, `'portuguese_24l'`, `'italian'`, `'italian_24l'`, `'spanish'`, `'spanish_24l'` (default: `english`, which is the same model as `'english_2026-04'`). Incompatible with `--config`. The "24l" variants are bigger models, not distilled yet and here only as preview. ### Generation Parameters - `--config CONFIG_PATH`: Path to custom config.yaml (for loading local model files). Incompatible with `--language`. - `--lsd-decode-steps LSD_DECODE_STEPS`: Number of generation steps (default: 1) - `--temperature TEMPERATURE`: Temperature for generation (default: the model's recommended value from its config — 0.3 for the English model, 0.7 otherwise) - `--noise-clamp NOISE_CLAMP`: Noise clamp value (default: None) - `--eos-threshold EOS_THRESHOLD`: EOS threshold (default: -4.0) - `--frames-after-eos FRAMES_AFTER_EOS`: Number of frames to generate after EOS (default: None, auto-calculated based on the text length). Each frame is 80ms. ### Performance Options - `--device DEVICE`: Device to use (default: "cpu"). Whether GPU helps is hardware-dependent — see the main [README's "Running on GPU" section](../../README.md#running-on-gpu) for measured numbers (no speedup observed on some CPUs with strong single-thread performance like Apple Silicon, but a measured ~2.6x speedup on a cloud x86 VM with a Tesla T4). - `--quantize`: Use int8 quantization for the model (default: False). This can reduce memory usage and increase speed, with minimal impact on audio quality. - `--quiet`, `-q`: Disable logging output ## Examples ### Basic Generation ```bash # Generate with default settings pocket-tts generate # Custom text pocket-tts generate --text "Hello, this is a custom message." # Custom output path pocket-tts generate --output-path "./my_audio.wav" ``` ### Voice Selection ```bash # Use different voice from HuggingFace pocket-tts generate --voice "hf://kyutai/tts-voices/jessica-jian/casual.wav" # Use local voice file pocket-tts generate --voice "./my_voice.wav" # Use a safetensors file (such as one created using `pocket-tts export-voice`) pocket-tts generate --voice "./my_voice.safetensors" ``` ### Quality Tuning ```bash # Higher quality (more steps) pocket-tts generate --lsd-decode-steps 5 --temperature 0.5 # More expressive (higher temperature) pocket-tts generate --temperature 1.0 # Adjust EOS threshold, smaller means finishing earlier. pocket-tts generate --eos-threshold -3.0 ``` ### Custom Model Config If you'd like to override the paths from which the models are loaded, you can provide a custom YAML configuration. Copy one of the files in `pocket_tts/config` (for example `pocket_tts/config/english.yaml`) and change `weights_path`, `weights_path_without_voice_cloning`, and `tokenizer_path` to the paths of the models you want to load. Then, use the --config option to point to your newly created config. ```bash # Use a different config pocket-tts generate --config "C://pocket-tts/my_config.yaml" ``` ## Output Format The generate command always outputs WAV files in the following format: - **Sample Rate**: 24kHz - **Channels**: Mono - **Bit Depth**: 16-bit PCM - **Format**: Standard WAV file For more advanced usage, see the [Python API documentation](python-api.md) or consider using the [serve command](serve.md) for web-based generation and quick iteration. --- ## File: docs/CLI Commands/serve.md # Serve The `serve` command starts a FastAPI web server that provides both a web interface and HTTP API for text-to-speech generation. ## Basic Usage ```bash uvx pocket-tts serve # or if installed manually: pocket-tts serve ``` This starts a server on `http://localhost:8000` with the default voice model. ## Command Options - `--host HOST`: Host to bind to (default: "localhost") - `--port PORT`: Port to bind to (default: 8000) - `--reload`: Enable auto-reload for development - `--language`: Language for the TTS model, one of `'english_2026-01'`, `'english_2026-04'`, `'english'`, `'french_24l'`, `'german_24l'`, `'portuguese_24l'`, `'italian_24l'`, `'spanish_24l'` (default: `english`, which is the same model as `'english_2026-04'`). Incompatible with `--config`. The "24l" variants are bigger models, not distilled yet and here only as preview. - `--config`: Path to a custom config .yaml. Incompatible with `--language`. - `--quantize`: Use int8 quantization for the model (default: False). This can reduce memory usage and increase speed, with minimal impact on audio quality. ## Examples ### Basic Server ```bash # Start with default settings pocket-tts serve # Custom host and port pocket-tts serve --host "localhost" --port 8080 ``` ### Custom Language To select the default language model, pass `--language`: ```bash pocket-tts serve --language french_24l ``` ### Custom Model Config If you'd like to override the paths from which the models are loaded, you can provide a custom YAML configuration. Copy one of the files in `pocket_tts/config` (for example `pocket_tts/config/english.yaml`) and change `weights_path`, `weights_path_without_voice_cloning:`, and `tokenizer_path:` to the paths of the models you want to load. Then, use the --config option to point to your newly created config. ```bash # Use a different config pocket-tts serve --config "C://pocket-tts/my_config.yaml" ``` ## Web Interface Once the server is running, navigate to `http://localhost:8000` to access the web interface. For more advanced usage, see the [Python API documentation](python-api.md) for direct integration with the TTS model. --- ## File: docs/API Reference/python-api.md # Python API Documentation Kyutai Pocket TTS provides a Python API for integrating text-to-speech capabilities into your applications. ## Installation ```bash pip install pocket-tts ``` ## Quick Start ```python from pocket_tts import TTSModel import scipy.io.wavfile # Load the model tts_model = TTSModel.load_model() # Get voice state from an audio file voice_state = tts_model.get_state_for_audio_prompt( "hf://kyutai/tts-voices/alba-mackenna/casual.wav" ) # Generate audio audio = tts_model.generate_audio(voice_state, "Hello world, this is a test.") # Save to file scipy.io.wavfile.write("output.wav", tts_model.sample_rate, audio.numpy()) ``` ## Core Classes ### TTSModel The main class for text-to-speech generation. #### Class Methods ##### `load_model(language=None, config=None, temp=None, lsd_decode_steps=1, noise_clamp=None, eos_threshold=-4.0, quantize=False)` Load and return a TTSModel instance with pre-trained weights. **Parameters:** - `language` (str | None): Name of built-in language config to load. Supported values: `"english_2026-01"`, `"english_2026-04"`, `"english"`, `"french_24l"`, `"german_24l"`, `"portuguese_24l"`, `"italian_24l"`, `"spanish_24l"`. If both `language` and `config` are omitted, defaults to `"english"`, which is the same model as `"english_2026-04"`. The "24l" variants are larger models that are not distilled yet and are here only as a preview. - `config` (str | None): Path to model config YAML file. Incompatible with `language`. - `temp` (float | None): Sampling temperature for generation. `None` uses the model's recommended default from its config file (`default_temperature`; 0.3 for the English model, 0.7 otherwise). - `lsd_decode_steps` (int): Number of generation steps (default: 1) - `noise_clamp` (float | None): Maximum value for noise sampling (default: None) - `eos_threshold` (float): Threshold for end-of-sequence detection (default: -4.0) - `quantize` (bool): Enable int8 quantization when loading the model (default: `False`) **Returns:** - `TTSModel`: Loaded model instance on CPU **Example:** ```python from pocket_tts import TTSModel # Load with default settings model = TTSModel.load_model() # Load with custom parameters model = TTSModel.load_model( language="english_2026-01", temp=0.5, lsd_decode_steps=5, eos_threshold=-3.0 ) ``` #### Properties ##### `device` (str) Returns the device type where the model is running ("cpu" or "cuda"). By default, the model runs on CPU. ```python from pocket_tts import TTSModel model = TTSModel.load_model() print(f"Model running on: {model.device}") ``` ##### `sample_rate` (int) Returns the generated audio sample rate (typically 24000 Hz). ```python from pocket_tts import TTSModel model = TTSModel.load_model() print(f"Sample rate: {model.sample_rate} Hz") ``` #### Methods ##### `get_state_for_audio_prompt(audio_conditioning, truncate=False)` Extract model state for a given audio file or URL (voice cloning), or load from a .safetensors file. **Parameters:** - `audio_conditioning` (Path | str | torch.Tensor): Audio or .safetensors file path, URL, or tensor - `truncate` (bool): Whether to truncate the audio (default: False) **Returns:** - `dict`: Model state dictionary containing hidden states and positional information **Example:** ```python from pocket_tts import TTSModel model = TTSModel.load_model() # From HuggingFace URL voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav") # From local file voice_state = model.get_state_for_audio_prompt("./my_voice.wav") # Reload state from a .safetensors file (much faster than extracting from an audio file) voice_state = model.get_state_for_audio_prompt("./my_voices.safetensors") # From HTTP URL voice_state = model.get_state_for_audio_prompt( "https://huggingface.co/kyutai/tts-voices/resolve" "/main/expresso/ex01-ex02_default_001_channel1_168s.wav" ) ``` ##### `generate_audio(model_state, text_to_generate, frames_after_eos=None, copy_state=True)` Generate complete audio tensor from text input. **Parameters:** - `model_state` (dict): Model state from `get_state_for_audio_prompt()` - `text_to_generate` (str): Text to convert to speech - `frames_after_eos` (int | None): Frames to generate after EOS detection (default: None) - `copy_state` (bool): Whether to copy the state (default: True) **Returns:** - `torch.Tensor`: Audio 1D tensor with shape [samples] **Example:** ```python from pocket_tts import TTSModel model = TTSModel.load_model() voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav") # Generate audio audio = model.generate_audio(voice_state, "Hello world!", frames_after_eos=2, copy_state=True) print(f"Generated audio shape: {audio.shape}") print(f"Audio duration: {audio.shape[-1] / model.sample_rate:.2f} seconds") ``` ##### `generate_audio_stream(model_state, text_to_generate, frames_after_eos=None, copy_state=True)` Generate audio streaming chunks from text input. **Parameters:** Same as `generate_audio()` **Yields:** - `torch.Tensor`: Audio chunks with shape [samples] **Example:** ```python from pocket_tts import TTSModel model = TTSModel.load_model() voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav") # Stream generation for chunk in model.generate_audio_stream(voice_state, "Long text content..."): # Process each chunk as it's generated print(f"Generated chunk: {chunk.shape[0]} samples") # Could save chunks to file or play in real-time ``` ## Functions ### export_model_state Export a model state for a given voice conditioning to a safetensors file for fast loading later. You can then load it again with the method `get_state_for_audio_prompt()`. **Parameters:** - `model_state` (dict): Model state dictionary from `get_state_for_audio_prompt()` - `dest` (str | Path): Path to save the safetensors file **Example:** ```python from pocket_tts import TTSModel, export_model_state model = TTSModel.load_model() # Get voice state from an audio file model_state_for_voice = model.get_state_for_audio_prompt( "hf://kyutai/tts-voices/alba-mackenna/casual.wav" ) # Export to safetensors for fast loading later export_model_state(model_state_for_voice, "my_voice.safetensors") # Quite fast, it's just loading the tensors without running any pytorch code model_state_for_voice_copy = model.get_state_for_audio_prompt("my_voice.safetensors") ``` ## Advanced Usage ### Voice Management ```python from pocket_tts import TTSModel model = TTSModel.load_model() # Preload multiple voices voices = { "casual": model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav"), "funny": model.get_state_for_audio_prompt( "https://huggingface.co/kyutai/tts-voices/resolve/main/expresso/ex01-ex02_default_001_channel1_168s.wav" ), } # Generate with different voices casual_audio = model.generate_audio(voices["casual"], "Hey there!") funny_audio = model.generate_audio(voices["funny"], "Good morning.") ``` ### Batch Processing ```python from pocket_tts import TTSModel import scipy.io.wavfile import torch model = TTSModel.load_model() voice_state = model.get_state_for_audio_prompt("hf://kyutai/tts-voices/alba-mackenna/casual.wav") # Process multiple texts efficiently by re-using the same voice state texts = [ "First sentence to generate.", "Second sentence to generate.", "Third sentence to generate.", ] audios = [] for text in texts: audio = model.generate_audio(voice_state, text) audios.append(audio) # Concatenate all audio full_audio = torch.cat(audios, dim=0) scipy.io.wavfile.write("batch_output.wav", model.sample_rate, full_audio.numpy()) ``` ### Streaming to File You can refer to our CLI implementation which can stream audio to a wav file. For more information about the command-line interface, see the [Generate Documentation](generate.md) or [Serve Documentation](serve.md). --- ## File: docs/quantization.md # Quantization Pocket TTS supports dynamic int8 quantization to reduce runtime memory usage and improve inference speed on x86 CPUs. ## Quick Start ### CLI ```bash pocket-tts generate --quantize --text "Hello world" pocket-tts serve --quantize ``` ### Python API ```python from pocket_tts import TTSModel model = TTSModel.load_model(quantize=True) voice_state = model.get_state_for_audio_prompt("alba") audio = model.generate_audio(voice_state, "Hello world!") ``` ## Installation Quantization works out of the box on any supported PyTorch version (2.5+) using `torch.ao`. For optimized performance, install `torchao` (requires torch 2.10+): ```bash pip install 'pocket-tts[quantize]' ``` The quantization module automatically selects the best available backend: - **torchao** (torch 2.10+): optimized C++ kernels, faster on both ARM and x86 - **torch.ao** (torch 2.5-2.9): deprecated but functional fallback ## Performance Benchmarks on the full eval paragraph across 8 voices, 5 isolated runs per config. ### x86 (FBGEMM, ubuntu-latest GitHub Actions runner) | Config | Runtime Memory | RTS | Speedup vs Baseline | | --------------------------- | -------------- | --------- | ------------------- | | baseline | 450 MB | 3.17x | -- | | **attention_ffn (default)** | **234 MB** | **4.04x** | **1.27x** | | all | 206 MB | 4.01x | 1.26x | ### ARM (QNNPACK, Apple M4 MacBook Air, torchao backend) | Config | Runtime Memory | RTS | Speedup vs Baseline | | --------------------------- | -------------- | --------- | ------------------- | | baseline | 450 MB | 6.33x | -- | | **attention_ffn (default)** | **234 MB** | **7.76x** | **1.23x** | With the `torch.ao` fallback (torch 2.5-2.9), ARM performance is ~16% slower than baseline rather than faster. Upgrading to torch 2.10+ with torchao is recommended for ARM users. ## Quality Quantization has no measurable impact on speech quality: - **WER (Word Error Rate)**: WER delta for the default attention_ffn config is −0.022 ±0.032 - the ± range crosses zero, meaning the delta is indistinguishable from measurement noise. - **Subjective listening**: no audible difference across all 8 voices ## What gets quantized When `quantize=True`, the following layer groups in the FlowLM transformer are quantized to int8: | Group | Params | Description | | ----------- | ------ | ------------------------------------------------ | | `attention` | ~25M | Q/K/V/output projections in transformer layers | | `ffn` | ~50M | Feed-forward linear layers in transformer layers | The flow matching network (`flow_net`, ~7M params) and the Mimi VAE decoder (convolutional) remain in float32.