## File: README.md

# MLX-Audio

[](https://trendshift.io/repositories/13625)

[](https://pypi.org/project/mlx-audio/)
[](https://pypi.org/project/mlx-audio/)
[](https://opensource.org/licenses/MIT)
[](https://github.com/Blaizzy/mlx-audio)

The best audio processing library built on Apple's MLX framework, providing fast and efficient text-to-speech (TTS), speech-to-text (STT), speech-to-speech (STS), music generation, and more on Apple Silicon.

## Table of Contents

- [Features](#features)
- [Installation](#installation)
- [Quick Start](#quick-start)
- [Supported Models](#supported-models)
- [Model Examples](#model-examples)
- [Web Interface \& API Server](#web-interface--api-server)
- [Quantization](#quantization)
- [Swift](#swift)
- [Requirements](#requirements)
- [License](#license)
- [Citation](#citation)
- [Acknowledgements](#acknowledgements)

## Features

- Fast inference optimized for Apple Silicon (M series chips)
- Multiple model architectures for TTS, STT, STS, and music generation
- Multilingual support across models
- Voice customization and cloning capabilities
- Adjustable speech speed control
- Interactive web interface with 3D audio visualization
- OpenAI-compatible REST API
- Quantization support (3-bit, 4-bit, 6-bit, 8-bit, and more) for optimized performance
- Swift package for iOS/macOS integration

## Installation

### Using pip
```bash
pip install mlx-audio
```

### Using uv to install only the command line tools
Latest release from pypi:
```bash
uv tool install --force mlx-audio --prerelease=allow
```

Latest code from github:
```bash
uv tool install --force git+https://github.com/Blaizzy/mlx-audio.git --prerelease=allow
```

### For development or web interface:

```bash
git clone https://github.com/Blaizzy/mlx-audio.git
cd mlx-audio
pip install -e ".[dev, server]"
```

## Quick Start

### Command Line

```bash
# Basic TTS generation
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello, world!' --voice Chelsie

# With a different voice and language hint
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit --text 'Welcome to MLX-Audio!' --voice Ethan --lang_code English

# Play audio immediately
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --play

# Save to a specific directory
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --output_path ./my_audio

# Stream audio during generation
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --stream

# Stream audio during generation and save it to disk
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit --text 'Hello!' --voice Chelsie --stream --save

# Join multiple generated segments into one file
mlx_audio.tts.generate --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit --text $'Hello!\nHow are you?' --voice Chelsie --join_audio
```

By default, when generation yields multiple segments, mlx-audio saves numbered files such as `audio_000.wav` and `audio_001.wav`. Use `--join_audio` to save one combined file instead. When using `--stream`, add `--save` to write the streamed audio to disk.

### Python API

```python
from mlx_audio.tts.utils import load_model

# Load model
model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit")

# Generate speech
for result in model.generate(
    "Hello from MLX-Audio!",
    voice="Chelsie",
    lang_code="English",
):
    print(f"Generated {result.audio.shape[0]} samples")
    # result.audio contains the waveform as mx.array
```

## Supported Models

### Text-to-Speech (TTS)

| Model | Description | Languages | Repo |
|-------|-------------|-----------|------|
| **Kokoro** | Fast, high-quality multilingual TTS | EN, JA, ZH, FR, ES, IT, PT, HI | [bf16](https://huggingface.co/mlx-community/Kokoro-82M-bf16), [8bit](https://huggingface.co/mlx-community/Kokoro-82M-8bit), [6bit](https://huggingface.co/mlx-community/Kokoro-82M-6bit), [4bit](https://huggingface.co/mlx-community/Kokoro-82M-4bit) |
| **KittenTTS** | Compact KittenTTS 0.8 models for edge-friendly TTS | EN | [nano](https://huggingface.co/mlx-community/kitten-tts-nano-0.8), [micro](https://huggingface.co/mlx-community/kitten-tts-micro-0.8), [mini](https://huggingface.co/mlx-community/kitten-tts-mini-0.8), [collection](https://huggingface.co/collections/mlx-community/kittentts) |
| **Qwen3-TTS** | Alibaba's multilingual TTS with voice design | ZH, EN, JA, KO, + more | [mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16](https://huggingface.co/mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16) |
| **Higgs Audio v3** | 4B conversational TTS with voice cloning and inline control tokens | 100 languages | [bosonai/higgs-audio-v3-tts-4b](https://huggingface.co/bosonai/higgs-audio-v3-tts-4b) |
| **OmniVoice** | Zero-shot multilingual TTS with voice cloning, batch generation, and nonverbal tags | 646+ languages | [mlx-community/OmniVoice-bf16](https://huggingface.co/mlx-community/OmniVoice-bf16) |
| **CSM / MisoTTS** | Sesame-style conversational speech models with voice cloning | EN | [mlx-community/csm-1b](https://huggingface.co/mlx-community/csm-1b), [MisoTTS bf16](https://huggingface.co/mlx-community/MisoLabs-MisoTTS-bf16), [MisoTTS 8bit](https://huggingface.co/mlx-community/MisoLabs-MisoTTS-8bit) |
| **Dia** | Dialogue-focused TTS | EN | [mlx-community/Dia-1.6B-fp16](https://huggingface.co/mlx-community/Dia-1.6B-fp16) |
| **OuteTTS** | Efficient TTS model | EN | [mlx-community/OuteTTS-1.0-0.6B-fp16](https://huggingface.co/mlx-community/OuteTTS-1.0-0.6B-fp16) |
| **Spark** | SparkTTS model | EN, ZH | [mlx-community/Spark-TTS-0.5B-bf16](https://huggingface.co/mlx-community/Spark-TTS-0.5B-bf16) |
| **Chatterbox** | Expressive multilingual TTS (v2/v3) | 23 languages | [v3](https://huggingface.co/mlx-community/chatterbox-multilingual-v3), [v2](https://huggingface.co/mlx-community/chatterbox-fp16) |
| **Soprano** | High-quality TTS | EN | [mlx-community/Soprano-1.1-80M-bf16](https://huggingface.co/mlx-community/Soprano-1.1-80M-bf16) |
| **Ming Omni TTS (BailingMM)** | Multimodal generation with voice cloning, style control, and speech/music/event generation | EN, ZH | [mlx-community/Ming-omni-tts-16.8B-A3B-bf16](https://huggingface.co/mlx-community/Ming-omni-tts-16.8B-A3B-bf16) |
| **Ming Omni TTS (Dense)** | Lightweight dense Ming Omni variant for voice cloning and style control | EN, ZH | [mlx-community/Ming-omni-tts-0.5B-bf16](https://huggingface.co/mlx-community/Ming-omni-tts-0.5B-bf16) |
| **KugelAudio** | SOTA 7B AR+Diffusion TTS for European languages | EN, DE, FR, ES, IT, PT, NL, PL, RU, UK, + 14 more | [kugelaudio/kugelaudio-0-open](https://huggingface.co/kugelaudio/kugelaudio-0-open) |
| **Voxtral TTS** | Mistral's 4B multilingual TTS (20 voices, 9 languages) | EN, FR, ES, DE, IT, PT, NL, AR, HI | [mlx-community/Voxtral-4B-TTS-2603-mlx-bf16](https://huggingface.co/mlx-community/Voxtral-4B-TTS-2603-mlx-bf16) |
| **LongCat-AudioDiT** | SOTA diffusion TTS in waveform latent space with voice cloning | ZH, EN | [mlx-community/LongCat-AudioDiT-1B-bf16](https://huggingface.co/mlx-community/LongCat-AudioDiT-1B-bf16) |
| **MeloTTS** | Lightweight VITS2-based TTS with streaming | EN (more coming) | [mlx-community/MeloTTS-English-MLX](https://huggingface.co/mlx-community/MeloTTS-English-MLX) |
| **MOSS-TTS** | 8B delay-pattern and local-transformer multilingual TTS with voice cloning | 31 languages | [OpenMOSS-Team/MOSS-TTS-v1.5](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-v1.5), [OpenMOSS-Team/MOSS-TTS](https://huggingface.co/OpenMOSS-Team/MOSS-TTS), [OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer-v1.5), [OpenMOSS-Team/MOSS-TTS-Local-Transformer](https://huggingface.co/OpenMOSS-Team/MOSS-TTS-Local-Transformer) |
| **MOSS-TTS-Nano** | Tiny multilingual voice-cloning TTS | 20 languages | [mlx-community/MOSS-TTS-Nano-100M](https://huggingface.co/mlx-community/MOSS-TTS-Nano-100M) |
| **Higgs Audio v2** | 3B Llama-backed TTS with real-time voice cloning | EN, ZH, KO, DE, ES | [bf16 (upstream)](https://huggingface.co/bosonai/higgs-audio-v2-generation-3B-base), [q8](https://huggingface.co/mlx-community/higgs-audio-v2-3B-mlx-q8), [q6](https://huggingface.co/mlx-community/higgs-audio-v2-3B-mlx-q6) |

### Speech-to-Text (STT)

| Model | Description | Languages | Repo |
|-------|-------------|-----------|------|
| **Whisper** | OpenAI's robust STT model | 99+ languages | [mlx-community/whisper-large-v3-turbo-asr-fp16](https://huggingface.co/mlx-community/whisper-large-v3-turbo-asr-fp16) |
| **Distil-Whisper** | Distilled fast Whisper variants | EN | [distil-whisper/distil-large-v3](https://huggingface.co/distil-whisper/distil-large-v3) |
| **Qwen3-ASR** | Alibaba's multilingual ASR | ZH, EN, JA, KO, + more | [mlx-community/Qwen3-ASR-1.7B-8bit](https://huggingface.co/mlx-community/Qwen3-ASR-1.7B-8bit) |
| **Mega-ASR** | Routed Qwen3-ASR with automatic clean/base vs degraded/LoRA switching | EN (fixtures), multilingual Qwen3-ASR backbone | [README](mlx_audio/stt/models/mega_asr/README.md) |
| **Qwen3-ForcedAligner** | Word-level audio alignment | ZH, EN, JA, KO, + more | [mlx-community/Qwen3-ForcedAligner-0.6B-8bit](https://huggingface.co/mlx-community/Qwen3-ForcedAligner-0.6B-8bit) |
| **MOSS-Transcribe-Diarize** | Timestamped transcription with speaker labels | Multiple major languages | https://huggingface.co/OpenMOSS-Team/MOSS-Transcribe-Diarize |
| **Parakeet** | NVIDIA's accurate STT | EN (v2), 25 EU languages (v3) | [mlx-community/parakeet-tdt-0.6b-v3](https://huggingface.co/mlx-community/parakeet-tdt-0.6b-v3) |
| **Nemotron 3.5 ASR (streaming)** | NVIDIA's cache-aware streaming FastConformer-RNNT with language-ID prompting | 40 language-locales | [mlx-community/nemotron-3.5-asr-streaming-0.6b](https://huggingface.co/mlx-community/nemotron-3.5-asr-streaming-0.6b) · [README](mlx_audio/stt/models/nemotron_asr/README.md) |
| **Voxtral** | Mistral's speech model | Multiple | [mlx-community/Voxtral-Mini-3B-2507-bf16](https://huggingface.co/mlx-community/Voxtral-Mini-3B-2507-bf16) |
| **Voxtral Realtime** | Mistral's 4B streaming STT | Multiple | [4bit](https://huggingface.co/mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit), [fp16](https://huggingface.co/mlx-community/Voxtral-Mini-4B-Realtime-2602-fp16) |
| **VibeVoice-ASR** | Microsoft's 9B ASR with diarization & timestamps | Multiple | [mlx-community/VibeVoice-ASR-bf16](https://huggingface.co/mlx-community/VibeVoice-ASR-bf16) |
| **Canary** | NVIDIA's multilingual ASR with translation | 25 EU + RU, UK | [README](mlx_audio/stt/models/canary/README.md) |
| **Moonshine** | Useful Sensors' lightweight ASR | EN | [README](mlx_audio/stt/models/moonshine/README.md) |
| **MMS** | Meta's massively multilingual ASR with adapters | 1000+ | [README](mlx_audio/stt/models/mms/README.md) |
| **Granite Speech** | IBM's ASR + speech translation | EN, FR, DE, ES, PT, JA | [README](mlx_audio/stt/models/granite_speech/README.md) |
| **Qwen2-Audio** | Alibaba's multimodal audio understanding (ASR, captioning, emotion, translation) | Multiple | [mlx-community/Qwen2-Audio-7B-Instruct-4bit](https://huggingface.co/mlx-community/Qwen2-Audio-7B-Instruct-4bit) |
| **MOSS-Music** | OpenMOSS music understanding and lyrics ASR | EN, ZH | [README](mlx_audio/stt/models/moss_music/README.md) |


### Voice Activity Detection / Speaker Diarization (VAD)

| Model | Description | Languages | Repo |
|-------|-------------|-----------|------|
| **Silero VAD** | Lightweight speech/non-speech detection with streaming state | Language-agnostic | [mlx-community/silero-vad](https://huggingface.co/mlx-community/silero-vad) |
| **Sortformer v1** | NVIDIA's end-to-end speaker diarization (up to 4 speakers) | Language-agnostic | [mlx-community/diar_sortformer_4spk-v1-fp32](https://huggingface.co/mlx-community/diar_sortformer_4spk-v1-fp32) |
| **Sortformer v2.1** | NVIDIA's streaming speaker diarization with AOSC compression | Language-agnostic | [mlx-community/diar_streaming_sortformer_4spk-v2.1-fp32](https://huggingface.co/mlx-community/diar_streaming_sortformer_4spk-v2.1-fp32) |

See the model READMEs for API details, streaming examples, and conversion steps.

### Speech-to-Speech (STS)

| Model | Description | Use Case | Repo |
|-------|-------------|----------|------|
| **SAM-Audio** | Text-guided source separation | Extract specific sounds | [mlx-community/sam-audio-large](https://huggingface.co/mlx-community/sam-audio-large) |
| **Liquid2.5-Audio*** | Speech-to-Speech, Text-to-Speech and Speech-to-Text | Speech interactions | [mlx-community/LFM2.5-Audio-1.5B-8bit](https://huggingface.co/mlx-community/LFM2.5-Audio-1.5B-8bit) |
| **MossFormer2 SE** | Speech enhancement | Noise removal | [starkdmi/MossFormer2_SE_48K_MLX](https://huggingface.co/starkdmi/MossFormer2_SE_48K_MLX) |
| **DeepFilterNet (1/2/3)** | Speech enhancement | Noise suppression | [mlx-community/DeepFilterNet-mlx](https://huggingface.co/mlx-community/DeepFilterNet-mlx) |

### Music Generation

| Model | Description | Languages | Repo |
|-------|-------------|-----------|------|
| **MiniMax Music 3** | Hierarchical AR + flow-matching song generation with lyrics and 44.1 kHz stereo output | Multilingual lyrics | [BF16](https://huggingface.co/mlx-community/MiniMax-Music3-bf16), [8-bit](https://huggingface.co/mlx-community/MiniMax-Music3-8bit), [6-bit](https://huggingface.co/mlx-community/MiniMax-Music3-6bit), [4-bit](https://huggingface.co/mlx-community/MiniMax-Music3-4bit), [MXFP8](https://huggingface.co/mlx-community/MiniMax-Music3-mxfp8), [MXFP4](https://huggingface.co/mlx-community/MiniMax-Music3-mxfp4), [NVFP4](https://huggingface.co/mlx-community/MiniMax-Music3-nvfp4), [guide](mlx_audio/music/models/minimax_music3/README.md) |

## Model Examples

### Qwen3-TTS

Alibaba's state-of-the-art multilingual TTS with voice cloning, emotion control, and voice design capabilities.

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16")
results = list(model.generate(
    text="Hello, welcome to MLX-Audio!",
    voice="Chelsie",
    language="English",
))

audio = results[0].audio  # mx.array
```

See the [Qwen3-TTS README](mlx_audio/tts/models/qwen3_tts/README.md) for voice cloning, CustomVoice, VoiceDesign, and all available models.

### OmniVoice

OmniVoice is a zero-shot multilingual TTS model for 646+ languages with voice cloning, batch generation, pronunciation controls, and nonverbal tags such as `[laughter]` and `[sigh]`. It uses a bidirectional Qwen3 backbone with iterative masked generation and a HiggsAudioV2 acoustic tokenizer.

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/OmniVoice-bf16")

# Basic multilingual TTS
for result in model.generate(
    text="Hello from OmniVoice running on Apple Silicon.",
    language="english",
    duration_s=5.0,
    num_steps=32,
):
    audio = result.audio

# Zero-shot voice cloning
for result in model.generate(
    text="This sentence uses the reference speaker.",
    language="english",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference audio.",
    duration_s=5.0,
):
    audio = result.audio
```

For stable voice cloning, provide `ref_text` that matches the reference clip. OmniVoice also supports `generate_batch()` for batched TTS and inline pronunciation controls.

### Ming Omni TTS (BailingMM)

```bash
mlx_audio.tts.generate \
    --model mlx-community/Ming-omni-tts-16.8B-A3B-bf16 \
    --prompt "Please generate speech based on the following description.\n" \
    --text "This is a quick Ming Omni test." \
    --lang_code en \
    --output_path audio_io \
    --file_prefix ming_basic \
    --verbose
```

See the [Ming Omni TTS README](mlx_audio/tts/models/bailingmm/README.md) for CLI and Python cookbook examples, and the [Ming Omni Dense README](mlx_audio/tts/models/dense/README.md) for the `mlx-community/Ming-omni-tts-0.5B-bf16` workflow.

### Kokoro TTS

Kokoro is a fast, multilingual TTS model with 54 voice presets.

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Kokoro-82M-bf16")
# Or use a quantized variant for lower memory usage:
# model = load_model("mlx-community/Kokoro-82M-8bit")
# model = load_model("mlx-community/Kokoro-82M-4bit")

# Generate with different voices
for result in model.generate(
    text="Welcome to MLX-Audio!",
    voice="af_heart",  # American female
    speed=1.0,
    lang_code="a"  # American English
):
    audio = result.audio
```

**Available Voices:**
- American English: `af_heart`, `af_bella`, `af_nova`, `af_sky`, `am_adam`, `am_echo`, etc.
- British English: `bf_alice`, `bf_emma`, `bm_daniel`, `bm_george`, etc.
- Japanese: `jf_alpha`, `jm_kumo`, etc.
- Chinese: `zf_xiaobei`, `zm_yunxi`, etc.

Kokoro requires `pip install misaki` for text processing. Japanese and Mandarin may additionally require `pip install misaki[ja]` or `pip install misaki[zh]`.

**Language Codes:**
| Code | Language | Note |
|------|----------|------|
| `a` | American English | Default; requires `pip install misaki` |
| `b` | British English | Requires `pip install misaki` |
| `j` | Japanese | Requires `pip install misaki[ja]` |
| `z` | Mandarin Chinese | Requires `pip install misaki[zh]` |
| `e` | Spanish | Requires `pip install misaki` |
| `f` | French | Requires `pip install misaki` |

### CSM (Voice Cloning)

Clone any voice using a reference audio sample:

```bash
mlx_audio.tts.generate \
    --model mlx-community/csm-1b \
    --text "Hello from Sesame." \
    --ref_audio ./reference_voice.wav \
    --play
```

### Whisper STT

```python
from mlx_audio.stt.generate import generate_transcription

result = generate_transcription(
    model="mlx-community/whisper-large-v3-turbo-asr-fp16",
    audio="audio.wav",
)
print(result.text)
```

### Qwen3-ASR & ForcedAligner

Alibaba's multilingual speech models for transcription and word-level alignment.

```python
from mlx_audio.stt import load

# Speech recognition
model = load("mlx-community/Qwen3-ASR-0.6B-8bit")
result = model.generate("audio.wav", language="English")
print(result.text)

# Word-level forced alignment
aligner = load("mlx-community/Qwen3-ForcedAligner-0.6B-8bit")
result = aligner.generate("audio.wav", text="I have a dream", language="English")
for item in result:
    print(f"[{item.start_time:.2f}s - {item.end_time:.2f}s] {item.text}")
```

See the [Qwen3-ASR README](mlx_audio/stt/models/qwen3_asr/README.md) for CLI usage, all models, and more examples.

### VibeVoice-ASR

Microsoft's 9B parameter speech-to-text model with speaker diarization and timestamps. Supports long-form audio (up to 60 minutes) and outputs structured JSON.

```python
from mlx_audio.stt.utils import load

model = load("mlx-community/VibeVoice-ASR-bf16")

# Basic transcription
result = model.generate(audio="meeting.wav", max_tokens=8192, temperature=0.0)
print(result.text)
# [{"Start":0,"End":5.2,"Speaker":0,"Content":"Hello everyone, let's begin."},
#  {"Start":5.5,"End":9.8,"Speaker":1,"Content":"Thanks for joining today."}]

# Access parsed segments
for seg in result.segments:
    print(f"[{seg['start_time']:.1f}-{seg['end_time']:.1f}] Speaker {seg['speaker_id']}: {seg['text']}")
```

**Streaming transcription:**

```python
# Stream tokens as they are generated
for text in model.stream_transcribe(audio="speech.wav", max_tokens=4096):
    print(text, end="", flush=True)
```

**With context (hotwords/metadata):**

```python
result = model.generate(
    audio="technical_talk.wav",
    context="MLX, Apple Silicon, PyTorch, Transformer",
    max_tokens=8192,
    temperature=0.0,
)
```

**CLI usage:**

```bash
# Basic transcription
python -m mlx_audio.stt.generate \
    --model mlx-community/VibeVoice-ASR-bf16 \
    --audio meeting.wav \
    --output-path output \
    --format json \
    --max-tokens 8192 \
    --verbose

# With context/hotwords
python -m mlx_audio.stt.generate \
    --model mlx-community/VibeVoice-ASR-bf16 \
    --audio technical_talk.wav \
    --output-path output \
    --format json \
    --max-tokens 8192 \
    --context "MLX, Apple Silicon, PyTorch, Transformer" \
    --verbose
```

### Parakeet (Multilingual STT)

NVIDIA's high-accuracy speech-to-text model. Parakeet v3 supports 25 European languages.

```python
from mlx_audio.stt.utils import load

# Load the multilingual v3 model
model = load("mlx-community/parakeet-tdt-0.6b-v3")

# Transcribe audio
result = model.generate("audio.wav")
print(f"Text: {result.text}")

# Access word-level timestamps
for sentence in result.sentences:
    print(f"[{sentence.start:.2f}s - {sentence.end:.2f}s] {sentence.text}")
```

**Streaming transcription:**

```python
for chunk in model.generate("long_audio.wav", stream=True):
    print(chunk.text, end="", flush=True)
```

**Supported languages (v3):**
Bulgarian, Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hungarian, Italian, Latvian, Lithuanian, Maltese, Polish, Portuguese, Romanian, Slovak, Slovenian, Spanish, Swedish, Russian, Ukrainian

**CLI usage:**

```bash
python -m mlx_audio.stt.generate \
    --model mlx-community/parakeet-tdt-0.6b-v3 \
    --audio speech.wav \
    --output-path output \
    --format json \
    --verbose
```

### KugelAudio

SOTA open-source 7B TTS model for 24 European languages, based on Microsoft VibeVoice.
Uses a hybrid AR + Diffusion architecture (Qwen2.5 LM + SDE-DPM-Solver++ diffusion head + VAE decoder).

```python
from mlx_audio.tts.utils import load_model

model = load_model("kugelaudio/kugelaudio-0-open")

for result in model.generate(
    text="Hello, welcome to MLX-Audio!",
    cfg_scale=3.0,       # Classifier-free guidance (1.0=fast, 3.0=quality)
    ddpm_steps=10,       # Diffusion steps (5=fast, 10=balanced, 20=max quality)
):
    audio = result.audio  # mx.array, 24kHz
```

The model loads directly from HuggingFace (weights are remapped automatically via `sanitize()`).
To quantize or save in a pre-converted format:

```bash
python -m mlx_audio.convert \
    --hf-path kugelaudio/kugelaudio-0-open \
    --mlx-path ./kugelaudio-0-open-bf16 \
    --dtype bfloat16
```

**Supported languages (24):** English, German, French, Spanish, Italian, Portuguese, Dutch, Polish, Russian, Ukrainian, Czech, Romanian, Hungarian, Swedish, Danish, Finnish, Norwegian, Greek, Bulgarian, Slovak, Croatian, Serbian, Turkish

> **Note:** Requires ~17GB memory (7B params in bfloat16).
> Pre-encoded voice presets (voice cloning) are not yet available in the upstream model — the model generates speech with a default voice.

### LongCat-AudioDiT

SOTA diffusion-based TTS operating in the waveform latent space. Uses Conditional Flow Matching with a DiT backbone and WAV-VAE codec at 24kHz. Supports zero-shot voice cloning.

```python
from mlx_audio.tts.utils import load

model = load("mlx-community/LongCat-AudioDiT-1B-bf16")

# Zero-shot TTS
result = next(model.generate("Hello, this is a test of AudioDiT."))
audio = result.audio  # mx.array, 24kHz

# Voice cloning (use "apg" guidance for best similarity)
result = next(model.generate(
    text="Today is warm turning to rain.",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference audio.",
    guidance_method="apg",
    cfg_strength=4.0,
    steps=16,
))
```

See the [LongCat-AudioDiT README](mlx_audio/tts/models/longcat_audiodit/README.md) for all parameters and CLI usage.

### Voxtral TTS

Mistral's 4B multilingual text-to-speech with 20 voice presets across 9 languages.

```python
from mlx_audio.tts.utils import load

model = load("mlx-community/Voxtral-4B-TTS-2603-mlx-bf16")

for result in model.generate(text="Hello, how are you today?", voice="casual_male"):
    print(result.audio_duration)
```

Voices: `casual_male`, `casual_female`, `cheerful_female`, `neutral_male`, `neutral_female`, `fr_male`, `fr_female`, `es_male`, `es_female`, `de_male`, `de_female`, `it_male`, `it_female`, `pt_male`, `pt_female`, `nl_male`, `nl_female`, `ar_male`, `hi_male`, `hi_female`

### Voxtral Realtime

Mistral's 4B parameter streaming speech-to-text model, optimized for low-latency transcription.

Available variants: [4bit](https://huggingface.co/mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit) (smaller/faster) | [fp16](https://huggingface.co/mlx-community/Voxtral-Mini-4B-Realtime-2602-fp16) (full precision)

```python
from mlx_audio.stt.utils import load

# Use 4bit for faster inference, fp16 for full precision
model = load("mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit")

# Transcribe audio
result = model.generate("audio.wav")
print(result.text)

# Streaming transcription
for chunk in model.generate("audio.wav", stream=True):
    print(chunk, end="", flush=True)

# Adjust transcription delay (lower = faster but less accurate)
result = model.generate("audio.wav", transcription_delay_ms=240)
```

### MedASR (Medical Transcription)

Specialized model for medical terms and dictation.

```python
from mlx_audio.stt.utils import load, transcribe

model = load("mlx-community/medasr")
result = transcribe("medical_dictation.wav", model=model)
print(result["text"])
```

**Live Transcription Example:**
```bash
# Continuous live transcription with VAD
python examples/medasr_live.py
```

### SAM-Audio (Source Separation)

Separate specific sounds from audio using text prompts:

```python
from mlx_audio.sts import SAMAudio, SAMAudioProcessor, save_audio

model = SAMAudio.from_pretrained("mlx-community/sam-audio-large")
processor = SAMAudioProcessor.from_pretrained("mlx-community/sam-audio-large")

batch = processor(
    descriptions=["A person speaking"],
    audios=["mixed_audio.wav"],
)

result = model.separate_long(
    batch.audios,
    descriptions=batch.descriptions,
    anchors=batch.anchor_ids,
    chunk_seconds=10.0,
    overlap_seconds=3.0,
    ode_opt={"method": "midpoint", "step_size": 2/32},
)

save_audio(result.target[0], "voice.wav")
save_audio(result.residual[0], "background.wav")
```

### MossFormer2 (Speech Enhancement)

Remove noise from speech recordings:

```python
from mlx_audio.sts import MossFormer2SEModel, save_audio

model = MossFormer2SEModel.from_pretrained("starkdmi/MossFormer2_SE_48K_MLX")
enhanced = model.enhance("noisy_speech.wav")
save_audio(enhanced, "clean.wav", 48000)
```

## Web Interface & API Server

MLX-Audio includes a modern web interface and OpenAI-compatible API.

### Starting the Server

```bash
# Start API server
mlx_audio.server --host 0.0.0.0 --port 8000

# Start web UI (in another terminal)
cd mlx_audio/ui
npm install && npm run dev
```

### API Endpoints

**Text-to-Speech** (OpenAI-compatible):
```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{"model": "mlx-community/Kokoro-82M-bf16", "input": "Hello!", "voice": "af_heart"}' \
  --output speech.wav
```

**Speech-to-Text**:
```bash
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F "file=@audio.wav" \
  -F "model=mlx-community/whisper-large-v3-turbo-asr-fp16"
```

## Quantization

Reduce model size and improve performance with quantization using the convert script:

```bash
# Convert and quantize to 4-bit
python -m mlx_audio.convert \
    --hf-path prince-canuma/Kokoro-82M \
    --mlx-path ./Kokoro-82M-4bit \
    --quantize \
    --q-bits 4 \
    --upload-repo username/Kokoro-82M-4bit (optional: if you want to upload the model to Hugging Face)

# Convert with MXFP4 quantization
python -m mlx_audio.convert \
    --hf-path prince-canuma/Kokoro-82M \
    --mlx-path ./Kokoro-82M-mxfp4 \
    --quantize \
    --q-mode mxfp4

# Convert with specific dtype (bfloat16)
python -m mlx_audio.convert \
    --hf-path prince-canuma/Kokoro-82M \
    --mlx-path ./Kokoro-82M-bf16 \
    --dtype bfloat16 \
    --upload-repo username/Kokoro-82M-bf16 (optional: if you want to upload the model to Hugging Face)
```

**Options:**
| Flag | Description |
|------|-------------|
| `--hf-path` | Source Hugging Face model or local path |
| `--mlx-path` | Output directory for converted model |
| `-q, --quantize` | Enable quantization |
| `--q-bits` | Bits per weight (optional, defaults depend on `--q-mode`) |
| `--q-group-size` | Group size for quantization (optional, defaults depend on `--q-mode`) |
| `--q-mode` | Quantization mode: `affine`, `mxfp4`, `mxfp8`, `nvfp4` |
| `--dtype` | Weight dtype: `float16`, `bfloat16`, `float32` |
| `--upload-repo` | Upload converted model to HF Hub |

## Swift

Looking for Swift/iOS support? Check out [mlx-audio-swift](https://github.com/Blaizzy/mlx-audio-swift) for on-device TTS using MLX on macOS and iOS.

## Requirements

- Python 3.10+
- Apple Silicon Mac (M1/M2/M3/M4)
- MLX framework
- **ffmpeg** (required for MP3/FLAC/OGG/Opus/Vorbis audio encoding)

### Installing ffmpeg

ffmpeg is required for saving audio in MP3, FLAC, OGG, Opus, or Vorbis format. Install it using:

```bash
# macOS (using Homebrew)
brew install ffmpeg

# Ubuntu/Debian
sudo apt install ffmpeg
```

WAV format works without ffmpeg.

## License

[MIT License](LICENSE)

## Citation

```bibtex
@misc{mlx-audio,
  author = {Canuma, Prince},
  title = {MLX Audio},
  year = {2025},
  howpublished = {\url{https://github.com/Blaizzy/mlx-audio}},
  note = {Audio processing library for Apple Silicon with TTS, STT, and STS capabilities.}
}
```

## Acknowledgements

- [Apple MLX Team](https://github.com/ml-explore/mlx) for the MLX framework

---

## File: docs/guides/index.md

# Guides

Practical how-to guides for common mlx-audio workflows.



-   :material-play-speed:{ .lg .middle } **Streaming Audio**

    ---

    Stream TTS output and STT transcription for low-latency applications.

    [:octicons-arrow-right-24: Streaming Guide](streaming.md)

-   :material-account-voice:{ .lg .middle } **Voice Cloning**

    ---

    Clone voices using reference audio with CSM, Qwen3-TTS, and other models.

    [:octicons-arrow-right-24: Voice Cloning Guide](voice-cloning.md)

-   :material-compress:{ .lg .middle } **Quantization**

    ---

    Reduce model size and speed up inference with 3-bit to 8-bit quantization.

    [:octicons-arrow-right-24: Quantization Guide](quantization.md)

-   :material-web:{ .lg .middle } **Web UI & API Server**

    ---

    Run the OpenAI-compatible REST API and interactive web interface.

    [:octicons-arrow-right-24: Web UI & API Guide](web-ui-api-server.md)

---

## File: docs/guides/quantization.md

# Quantization

Quantization reduces model size and can improve inference speed by storing weights at lower precision. MLX Audio supports several quantization formats through the `mlx_audio.convert` module.

## Why Quantize?

| Benefit | Details |
|---------|---------|
| **Smaller models** | A 4-bit model is roughly 4x smaller than float16 |
| **Faster inference** | Lower-precision weights move through memory faster on Apple Silicon |
| **Lower memory usage** | Fit larger models into unified memory |

The trade-off is a potential reduction in audio quality, especially at very low bit widths.

## Available Bit Widths

| Bits | Typical Use Case | Quality | Size Reduction |
|------|------------------|---------|----------------|
| **3-bit** | Maximum compression, acceptable for quick prototyping | Lower | ~5x vs fp16 |
| **4-bit** | Good balance of quality and size | Good | ~4x vs fp16 |
| **6-bit** | Near-lossless for most models | Very good | ~2.5x vs fp16 |
| **8-bit** | Minimal quality loss | Excellent | ~2x vs fp16 |

## Quantization Modes

MLX Audio supports several quantization modes:

| Mode | Description |
|------|-------------|
| `affine` | Standard affine quantization (default) |
| `mxfp4` | Microscaling FP4 format |
| `mxfp8` | Microscaling FP8 format |
| `nvfp4` | NVIDIA FP4 format |

## Converting and Quantizing Models

Use the `mlx_audio.convert` CLI to convert a HuggingFace model to MLX format with quantization:

### Basic 4-bit Quantization

```bash
python -m mlx_audio.convert \
    --hf-path prince-canuma/Kokoro-82M \
    --mlx-path ./Kokoro-82M-4bit \
    --quantize \
    --q-bits 4
```

### MXFP4 Quantization

```bash
python -m mlx_audio.convert \
    --hf-path prince-canuma/Kokoro-82M \
    --mlx-path ./Kokoro-82M-mxfp4 \
    --quantize \
    --q-mode mxfp4
```

### Convert to bfloat16 (No Quantization)

```bash
python -m mlx_audio.convert \
    --hf-path prince-canuma/Kokoro-82M \
    --mlx-path ./Kokoro-82M-bf16 \
    --dtype bfloat16
```

### Upload to Hugging Face Hub

Add `--upload-repo` to push the converted model directly:

```bash
python -m mlx_audio.convert \
    --hf-path prince-canuma/Kokoro-82M \
    --mlx-path ./Kokoro-82M-4bit \
    --quantize \
    --q-bits 4 \
    --upload-repo username/Kokoro-82M-4bit
```

## Conversion Options Reference

| Flag | Description |
|------|-------------|
| `--hf-path` | Source HuggingFace model or local path |
| `--mlx-path` | Output directory for the converted model |
| `-q`, `--quantize` | Enable quantization |
| `--q-bits` | Bits per weight (e.g., 3, 4, 6, 8) |
| `--q-group-size` | Group size for quantization (defaults depend on mode) |
| `--q-mode` | Quantization mode: `affine`, `mxfp4`, `mxfp8`, `nvfp4` |
| `--dtype` | Weight dtype when not quantizing: `float16`, `bfloat16`, `float32` |
| `--upload-repo` | Upload converted model to HuggingFace Hub |

## Using Pre-Quantized Models

The [mlx-community](https://huggingface.co/mlx-community) organization on Hugging Face hosts many pre-quantized models ready to use. For example:

```python
from mlx_audio.tts.utils import load_model

# Load a pre-quantized model -- no conversion needed
model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-CustomVoice-6bit")

results = list(model.generate(
    text="This is running from a 6-bit quantized model!",
    voice="serena",
))
```

```python
from mlx_audio.stt.utils import load

# 4-bit Voxtral for faster STT
model = load("mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit")
result = model.generate("audio.wav")
print(result.text)
```

## Quality vs. Performance Trade-offs

!!! info "General guidance"
    Start with **4-bit** or **6-bit** quantization. Move to 8-bit only if you hear artifacts, or to 3-bit if you need the smallest possible model.

- **TTS models** tend to be sensitive to quantization at 3-bit. 4-bit is usually the sweet spot.
- **STT models** (e.g., Whisper, Voxtral Realtime) often tolerate 4-bit quantization with minimal accuracy loss.
- **Larger models** (1B+ parameters) generally tolerate lower bit widths better than smaller ones.
- Always **listen to output** when evaluating TTS quantization -- word error rate alone does not capture prosody degradation.

## Mixed Quantization Recipes

MLX Audio also supports mixed-precision quantization recipes that apply different bit widths to different layers:

| Recipe | Description |
|--------|-------------|
| `mixed_2_6` | 2-bit for some layers, 6-bit for others |
| `mixed_3_4` | 3-bit / 4-bit mix |
| `mixed_3_6` | 3-bit / 6-bit mix |
| `mixed_4_6` | 4-bit / 6-bit mix |

These can provide better quality-to-size ratios by keeping critical layers at higher precision.

---

## File: docs/guides/streaming.md

# Streaming Audio

MLX Audio supports streaming audio generation for low-latency playback. Instead of waiting for the entire utterance to be synthesized, you can start playing audio chunks as they are produced.

## CLI Streaming

Add the `--stream` flag to any TTS generation command. When streaming is enabled, audio is played back in real time as chunks become available:

```bash
mlx_audio.tts.generate \
    --model mlx-community/Kokoro-82M-bf16 \
    --text "Hello, this is a streaming example!" \
    --lang_code a \
    --stream
```

!!! tip "Streaming implies playback"
    The `--stream` flag automatically enables `--play`, so there is no need to pass both.

### Controlling Chunk Size

The `--streaming_interval` argument controls how frequently audio chunks are emitted (in seconds). Smaller values reduce latency but add per-chunk overhead:

```bash
mlx_audio.tts.generate \
    --model mlx-community/Kokoro-82M-bf16 \
    --text "Adjusting the streaming interval changes latency." \
    --lang_code a \
    --stream \
    --streaming_interval 1.5
```

The default interval is **2.0 seconds**.

## Python Streaming

### TTS Streaming

Every model's `generate()` method accepts `stream=True`. When enabled, it yields `GenerationResult` objects as chunks rather than waiting for full synthesis:

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Kokoro-82M-bf16")

for result in model.generate(
    text="This audio will stream chunk by chunk.",
    voice="af_heart",
    lang_code="a",
    stream=True,
    streaming_interval=2.0,
):
    # result.audio is an mx.array with one chunk of audio
    print(f"Chunk: {result.audio.shape[0]} samples")
    # Feed result.audio to an audio player or buffer
```

Each `GenerationResult` chunk contains:

| Attribute | Description |
|-----------|-------------|
| `audio` | `mx.array` waveform for this chunk |
| `sample_rate` | Sample rate in Hz |
| `is_streaming_chunk` | `True` for intermediate chunks |
| `is_final_chunk` | `True` for the last chunk |

### Qwen3-TTS Streaming

Qwen3-TTS models support streaming across all generation methods -- `generate()`, `generate_custom_voice()`, and `generate_voice_design()`:

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-CustomVoice-6bit")

audio_chunks = []
for result in model.generate(
    text="Hello, how are you today?",
    voice="serena",
    stream=True,
    streaming_interval=0.32,  # ~4 tokens at 12.5 Hz
):
    audio_chunks.append(result.audio)
    # Play or process each chunk for low-latency output
```

!!! info "Streaming interval for Qwen3-TTS"
    At 12.5 Hz token rate, a `streaming_interval` of **0.32** seconds corresponds to roughly 4 tokens per chunk. Lower values reduce latency but increase overhead.

### STT Streaming

Several speech-to-text models support streaming transcription. Pass `stream=True` to `generate()`:

=== "Voxtral Realtime"

    ```python
    from mlx_audio.stt.utils import load

    model = load("mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit")

    for chunk in model.generate("audio.wav", stream=True):
        print(chunk, end="", flush=True)
    ```

=== "Parakeet"

    ```python
    from mlx_audio.stt.utils import load

    model = load("mlx-community/parakeet-tdt-0.6b-v3")

    for chunk in model.generate("long_audio.wav", stream=True):
        print(chunk.text, end="", flush=True)
    ```

=== "VibeVoice-ASR"

    ```python
    from mlx_audio.stt.utils import load

    model = load("mlx-community/VibeVoice-ASR-bf16")

    for text in model.stream_transcribe(audio="speech.wav", max_tokens=4096):
        print(text, end="", flush=True)
    ```

### STT Streaming via CLI

```bash
python -m mlx_audio.stt.generate \
    --model mlx-community/whisper-large-v3-turbo-asr-fp16 \
    --audio speech.wav \
    --output-path output \
    --format json \
    --stream
```

## API Server Streaming

The API server supports streaming for both TTS and STT. Set `"stream": true` in your request:

### Streaming TTS

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Kokoro-82M-bf16",
    "input": "Streaming over HTTP!",
    "voice": "af_heart",
    "stream": true,
    "streaming_interval": 2.0,
    "response_format": "wav"
  }' \
  --output streamed_speech.wav
```

### Streaming STT

```bash
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F "file=@audio.wav" \
  -F "model=mlx-community/whisper-large-v3-turbo-asr-fp16" \
  -F "stream=true"
```

Streaming STT returns newline-delimited JSON (`application/x-ndjson`), with each line containing a `text` field and optional timing information.

## Real-Time WebSocket Transcription

The API server also exposes a WebSocket endpoint for real-time audio transcription:

```
ws://localhost:8000/v1/audio/transcriptions/realtime
```

This is useful for live microphone input or continuous audio streams.

## Tips

- **Latency vs. quality** -- Shorter `streaming_interval` values give lower latency but may produce more chunk boundaries. Start with the default and decrease as needed.
- **Memory** -- Streaming does not significantly change peak memory usage since model weights remain loaded; it only changes _when_ decoded audio is returned.
- **Joining chunks** -- If you need a single contiguous file, concatenate the `result.audio` arrays after the loop and write once with `mlx_audio.audio_io.write()`.

---

## File: docs/guides/voice-cloning.md

# Voice Cloning

Several MLX Audio models can clone a speaker's voice from a short reference audio sample. This guide covers the supported models and how to use them.

## Overview

| Model | Method | Reference Audio Required | Notes |
|-------|--------|--------------------------|-------|
| **CSM / MisoTTS** | `--ref_audio` CLI / `ref_audio` kwarg | Yes (WAV) | Sesame-style conversational speech models |
| **Qwen3-TTS Base** | `ref_audio` + `ref_text` kwargs | Yes (WAV) + transcript | Alibaba multilingual TTS |
| **OmniVoice** | `ref_audio` + `ref_text` kwargs | Yes (WAV) + transcript recommended | 646+ language zero-shot cloning, best with prompt preprocessing |
| **Spark** | `ref_audio` kwarg | Yes | SparkTTS voice cloning |
| **Chatterbox** | `ref_audio` kwarg | Yes | Expressive multilingual TTS |
| **OuteTTS** | `ref_audio` kwarg | Yes | Efficient TTS with cloning |
| **Ming Omni TTS** | `ref_audio` kwarg | Yes | Multimodal with voice cloning |

## CSM (Conversational Speech Model)

CSM is the simplest path to voice cloning. Provide a WAV file of the target voice and CSM will match it:

### CLI

```bash
mlx_audio.tts.generate \
    --model mlx-community/csm-1b \
    --text "Hello from Sesame." \
    --ref_audio ./reference_voice.wav \
    --play
```

### Python

```python
from mlx_audio.tts.utils import load_model
from mlx_audio.utils import load_audio

model = load_model("mlx-community/csm-1b")

for result in model.generate(
    text="Hello from Sesame.",
    ref_audio="./reference_voice.wav",
):
    audio = result.audio
```

!!! tip "Automatic transcription"
    If you do not provide `--ref_text`, MLX Audio will automatically transcribe the reference audio using a Whisper model. You can specify which STT model to use with `--stt_model`.

## Qwen3-TTS

Qwen3-TTS Base models support voice cloning by providing both a reference audio file and its transcript:

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16")

results = list(model.generate(
    text="Hello, welcome to MLX-Audio!",
    ref_audio="sample_audio.wav",
    ref_text="This is what my voice sounds like.",
))

audio = results[0].audio  # mx.array
```

### CustomVoice (Emotion Control)

The CustomVoice variant lets you combine a predefined voice with emotion and style instructions:

```python
model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-CustomVoice-bf16")

results = list(model.generate_custom_voice(
    text="I'm so excited to meet you!",
    speaker="Vivian",
    language="English",
    instruct="Very happy and excited.",
))
```

### VoiceDesign (Create Any Voice)

The VoiceDesign variant creates a voice from a text description -- no reference audio needed:

```python
model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16")

results = list(model.generate_voice_design(
    text="Big brother, you're back!",
    language="English",
    instruct="A cheerful young female voice with high pitch and energetic tone.",
))
```

### Available Qwen3-TTS Models

| Model | Method | Description |
|-------|--------|-------------|
| `Qwen3-TTS-12Hz-0.6B-Base-bf16` | `generate()` | Fast, predefined voices + cloning |
| `Qwen3-TTS-12Hz-1.7B-Base-bf16` | `generate()` | Higher quality |
| `Qwen3-TTS-12Hz-0.6B-CustomVoice-bf16` | `generate_custom_voice()` | Voices + emotion |
| `Qwen3-TTS-12Hz-1.7B-CustomVoice-bf16` | `generate_custom_voice()` | Better emotion control |
| `Qwen3-TTS-12Hz-1.7B-VoiceDesign-bf16` | `generate_voice_design()` | Create any voice from a description |

### Available Speakers (Base / CustomVoice)

- **Chinese:** `Vivian`, `Serena`, `Uncle_Fu`, `Dylan` (Beijing Dialect), `Eric` (Sichuan Dialect)
- **English:** `Ryan`, `Aiden`

## Ming Omni TTS

Ming Omni TTS supports voice cloning and style control:

```bash
mlx_audio.tts.generate \
    --model mlx-community/Ming-omni-tts-16.8B-A3B-bf16 \
    --text "This is a Ming Omni voice cloning test." \
    --ref_audio ./reference_voice.wav \
    --lang_code en \
    --verbose
```

See the [Ming Omni TTS model page](../models/tts/index.md) for detailed cookbook examples.

## Best Practices

### Preparing Reference Audio

!!! warning "Quality matters"
    The quality of your cloned voice depends heavily on the reference audio.

- **Duration** -- 5 to 15 seconds of clean speech works best. Very short clips lack enough speaker information; very long clips may confuse the model.
- **Format** -- Use WAV at 16 kHz or higher. The library will resample automatically, but starting with a good sample rate avoids artifacts.
- **Noise** -- Record in a quiet environment. Background noise will be cloned along with the voice. Consider using MossFormer2 or DeepFilterNet to enhance noisy recordings first.
- **Content** -- Natural, conversational speech produces the best results. Avoid whispering or shouting unless that is the target style.

### Providing Reference Text

Some models (Qwen3-TTS) require a transcript of the reference audio (`ref_text`). If you omit it, MLX Audio will transcribe the clip automatically using Whisper:

```bash
mlx_audio.tts.generate \
    --model mlx-community/Qwen3-TTS-12Hz-0.6B-Base-bf16 \
    --text "Cloned speech output." \
    --ref_audio reference.wav \
    --stt_model mlx-community/whisper-large-v3-turbo-asr-fp16
```

Providing the transcript yourself avoids loading the STT model and speeds up generation.

## OmniVoice

OmniVoice supports multilingual zero-shot voice cloning with a HiggsAudioV2 acoustic tokenizer and iterative masked generation.

!!! warning "ref_text must match preprocessed audio"
    OmniVoice preprocessing removes silence and trims the reference clip. If you transcribe the **original** file, the ASR transcript may be longer than the preprocessed audio, causing the extra text to leak into generation. Always transcribe the **preprocessed** audio, not the raw recording. See `examples/omnivoice_clone_demo.py` for the correct workflow.

```python
from mlx_audio.tts.utils import load_model as load_tts
from mlx_audio.tts.models.omnivoice.utils import create_voice_clone_prompt
from mlx_audio.stt.utils import load_model as load_stt
from mlx_audio.audio_io import write as audio_write
import mlx.core as mx, numpy as np, tempfile, os

tts = load_tts("mlx-community/OmniVoice-bf16")

# Preprocess → encode → decode → transcribe (matches original pipeline)
ref_tokens = create_voice_clone_prompt("reference.wav", tokenizer=tts.audio_tokenizer)
mx.eval(ref_tokens)

preprocessed = np.array(tts.audio_tokenizer.decode(ref_tokens).astype(mx.float32))
tmp = tempfile.NamedTemporaryFile(suffix=".wav", delete=False)
audio_write(tmp.name, preprocessed, 24000); tmp.close()

ref_text = load_stt("mlx-community/Qwen3-ASR-0.6B-8bit").generate(tmp.name).text
os.unlink(tmp.name)

results = list(tts.generate(
    text="Hello from OmniVoice.",
    language="english",
    ref_tokens=ref_tokens,
    ref_text=ref_text,
))

audio_write("output.wav", np.array(results[0].audio), results[0].sample_rate)
```

### OmniVoice-specific notes

- **Reference text is required for stable cloning.** Without it, output quality degrades significantly — garbled speech, wrong language, or missing words.
- **Transcribe after preprocessing, not before.** The original k2-fsa/OmniVoice runs Whisper on audio after silence removal. This demo replicates that approach.
- **Prompt preprocessing matters.** MLX Audio mirrors the original Python pipeline with RMS normalization, silence removal, trimming at silence gaps, and torchaudio-compatible resampling before reference encoding.
- **Best reference length:** roughly 5–15 seconds of actual speech after silence trimming.
- **Supported inline controls:** nonverbal tags such as `[laughter]`, `[sigh]`, and pronunciation overrides for English CMU dictionary forms and Chinese pinyin forms.

### Example: English CMU pronunciation control

```python
results = list(model.generate(
    text="He plays the [B EY1 S] guitar while catching a [B AE1 S] fish.",
    language="english",
))
```

### Example: Nonverbal tags

```python
results = list(model.generate(
    text="I just heard the funniest joke [laughter] that was incredible.",
    language="english",
))
```

### Combining Cloning with Streaming

Voice cloning and streaming work together. Add `--stream` to any cloning command:

```bash
mlx_audio.tts.generate \
    --model mlx-community/csm-1b \
    --text "Streaming with a cloned voice." \
    --ref_audio reference.wav \
    --stream
```

---

## File: docs/guides/web-ui-api-server.md

# Web UI & API Server

MLX Audio ships with a FastAPI-based API server and a Next.js web interface (Studio UI) for interactive audio generation and transcription.

## Starting the Server

### API Server Only

```bash
mlx_audio.server --host 0.0.0.0 --port 8000
```

### API Server with Studio UI

Pass `--start-ui` to launch the Next.js web interface alongside the API server:

```bash
mlx_audio.server --host 0.0.0.0 --port 8000 --start-ui
```

The API will be available at `http://localhost:8000` and the Studio UI at `http://localhost:3000`.

### Server Options

| Flag | Default | Description |
|------|---------|-------------|
| `--host` | `localhost` | Host to bind the server to |
| `--port` | `8000` | Port for the API server |
| `--reload` | `false` | Auto-reload on code changes (development) |
| `--start-ui` | `false` | Start the Studio UI alongside the API |
| `--allowed-origins` | `*` | CORS allowed origins (space-separated) |
| `--log-dir` | `logs` | Directory for server logs |
| `--realtime-model` | `null` | Default model for `/v1/realtime` when the client omits `?model=` |
| `--realtime-transcription-delay-ms` | `null` | Transcription latency/quality knob for models that support it (e.g. `voxtral_realtime`) |
| `--vad-model` | `mlx-community/silero-vad` | Streaming VAD model used for server-side turn detection (`server_vad`) on `/v1/realtime` |
| `--tts-max-batch-size` | `8` | Maximum compatible TTS speech requests per continuous batch session |

The two realtime flags also read from `MLX_AUDIO_REALTIME_MODEL` and `MLX_AUDIO_REALTIME_TRANSCRIPTION_DELAY_MS` if present; the CLI flags take precedence. `--vad-model` likewise reads from `MLX_AUDIO_VAD_MODEL`.
The TTS batching flag also reads from `MLX_AUDIO_TTS_MAX_BATCH_SIZE`; the CLI flag takes precedence.

### CORS Configuration

By default, all origins are allowed. To restrict origins:

```bash
mlx_audio.server --allowed-origins http://localhost:3000 https://myapp.example.com
```

Or set the `MLX_AUDIO_ALLOWED_ORIGINS` environment variable with a comma-separated list.

### TTS Continuous Batching

Compatible non-streaming TTS `/v1/audio/speech` requests are routed through the server's continuous batching path when the model exposes a `create_tts_batch_session(...)` hook.

Requests that are not supported by the model's continuous batching hook continue through the existing serial or fixed-window batch paths.

```bash
mlx_audio.server --host 127.0.0.1 --port 8000 \
  --tts-max-batch-size 8
```

## OpenAI-Compatible API

The server implements OpenAI-compatible endpoints, so existing code that targets the OpenAI audio API can point to MLX Audio with minimal changes.

### Text-to-Speech

**`POST /v1/audio/speech`**

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Kokoro-82M-bf16",
    "input": "Hello, world!",
    "voice": "af_heart",
    "response_format": "mp3"
  }' \
  --output speech.mp3
```

**Request body:**

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `model` | string | required | Model ID (HuggingFace repo or local path) |
| `input` | string | required | Text to synthesize |
| `voice` | string | `null` | Voice preset name |
| `speed` | float | `1.0` | Playback speed multiplier |
| `lang_code` | string | `"a"` | Language code |
| `response_format` | string | `"mp3"` | Output format: `mp3`, `wav`, `flac`, `ogg`, `opus` |
| `stream` | bool | `false` | Stream audio chunks |
| `streaming_interval` | float | `2.0` | Seconds between stream chunks |
| `temperature` | float | `0.7` | Sampling temperature |
| `max_tokens` | int | `1200` | Maximum generation tokens |
| `ref_audio` | string | `null` | Path to reference audio for voice cloning |
| `ref_text` | string | `null` | Transcript of reference audio |
| `instruct` | string | `null` | Style/emotion instruction |

#### Streaming TTS

```bash
curl -X POST http://localhost:8000/v1/audio/speech \
  -H "Content-Type: application/json" \
  -d '{
    "model": "mlx-community/Kokoro-82M-bf16",
    "input": "Streaming audio over HTTP.",
    "voice": "af_heart",
    "stream": true,
    "response_format": "wav"
  }' \
  --output streamed.wav
```

### Speech-to-Text

**`POST /v1/audio/transcriptions`**

```bash
curl -X POST http://localhost:8000/v1/audio/transcriptions \
  -F "file=@audio.wav" \
  -F "model=mlx-community/whisper-large-v3-turbo-asr-fp16"
```

**Form fields:**

| Field | Type | Default | Description |
|-------|------|---------|-------------|
| `file` | file | required | Audio file to transcribe |
| `model` | string | required | STT model ID |
| `language` | string | `null` | Language code |
| `max_tokens` | int | `1024` | Maximum output tokens |
| `stream` | bool | `false` | Stream results as NDJSON |
| `context` | string | `null` | Hotwords or metadata to guide transcription |
| `verbose` | bool | `false` | Include extra details |

### Audio Source Separation

**`POST /v1/audio/separations`**

```bash
curl -X POST http://localhost:8000/v1/audio/separations \
  -F "file=@mixed.wav" \
  -F "model=mlx-community/sam-audio-large-fp16" \
  -F "description=speech"
```

Returns JSON with base64-encoded `target` and `residual` WAV buffers.

### Model Management

```bash
# List loaded models
curl http://localhost:8000/v1/models

# Load a model
curl -X POST "http://localhost:8000/v1/models?model_name=mlx-community/Kokoro-82M-bf16"

# Unload a model
curl -X DELETE "http://localhost:8000/v1/models?model_name=mlx-community/Kokoro-82M-bf16"
```

### Real-Time WebSocket Transcription

The server exposes two WebSocket endpoints for live transcription. Both accept 16-bit signed little-endian PCM audio; they differ in wire protocol and intended consumers.

#### `/v1/audio/transcriptions/realtime` (VAD-based streaming)

```
ws://localhost:8000/v1/audio/transcriptions/realtime
```

Send raw PCM frames as binary WebSocket messages; the server performs VAD, chunks on silence, and emits transcription JSON messages back. Uses the preloaded Whisper-style STT model — connect after calling `POST /v1/models` to select it.

#### `/v1/realtime` (OpenAI Realtime-compatible)

```
ws://localhost:8000/v1/realtime?model=<model-id>
```

Implements a subset of the [OpenAI Realtime API](https://platform.openai.com/docs/guides/realtime) wire protocol, so existing Realtime clients can target MLX Audio by swapping the base URL. Any STT model that exposes `create_streaming_session` works — including `voxtral_realtime`, which is streaming-first and recommended.

**Model selection order** (first match wins):

1. `?model=` query parameter on connect.
2. `session.update.model` (or `session.audio.input.transcription.model`) event after connect.
3. `--realtime-model` CLI flag / `MLX_AUDIO_REALTIME_MODEL` env var.

If none is set, the server replies with an `error` event and closes the socket.

**Client → server events** (subset of OpenAI Realtime):

| Type | Purpose |
|------|---------|
| `session.update` | Change the model, input sample rate, transcription config, or turn detection mode |
| `input_audio_buffer.append` | Append a chunk of base64-encoded PCM16 to the current item |
| `input_audio_buffer.commit` | Signal end-of-utterance; the server drains deltas and emits `completed`. Manual-commit mode only — unused under `server_vad` |

**Declaring the input sample rate.** The server assumes incoming PCM is 24 kHz by default (the OpenAI Realtime client convention). If you are sending audio at a different rate — e.g. a 16 kHz microphone capture or a 48 kHz file — tell the server via `session.update` so it resamples correctly. Without this, 16 kHz audio interpreted as 24 kHz would sound sped-up to the model and transcribe as garbage.

```json
{
  "type": "session.update",
  "session": {
    "audio": {"input": {"format": {"type": "audio/pcm", "rate": 16000}}}
  }
}
```

This only declares *your* input rate. The server resamples from that to whatever rate the model expects internally — you never need to match the model's native rate.

**Turn detection (`server_vad`).** By default (`turn_detection: null`) the client owns turn boundaries and signals end-of-utterance with `input_audio_buffer.commit`. Set `turn_detection` to `{"type": "server_vad"}` via `session.update` and the server instead runs a streaming VAD (Silero by default — see `--vad-model`) over the incoming audio: it detects when the speaker starts and stops, emits `input_audio_buffer.speech_started` / `input_audio_buffer.speech_stopped`, and auto-commits each turn. In this mode the client never sends `input_audio_buffer.commit`.

```json
{
  "type": "session.update",
  "session": {
    "audio": {
      "input": {
        "turn_detection": {
          "type": "server_vad",
          "threshold": 0.5,
          "prefix_padding_ms": 300,
          "silence_duration_ms": 500
        }
      }
    }
  }
}
```

| Field | Default | Purpose |
|-------|---------|---------|
| `threshold` | `0.5` | Speech-probability threshold above which a VAD frame counts as speech |
| `prefix_padding_ms` | `300` | Shifts the reported `audio_start_ms` earlier; does not gate transcription |
| `silence_duration_ms` | `500` | Sub-threshold audio after speech before the turn is considered finished |

To use a different VAD model, launch the server with `--vad-model ` (or set `MLX_AUDIO_VAD_MODEL`). `semantic_vad` is not implemented yet — requesting it returns an `error` event.

**Server → client events:**

| Type | Purpose |
|------|---------|
| `session.created` / `session.updated` | Session snapshot (id, model, input format, turn detection) |
| `conversation.item.added` | New user item opened for the next audio chunk |
| `input_audio_buffer.speech_started` | (`server_vad`) Speaker started; carries `audio_start_ms` |
| `input_audio_buffer.speech_stopped` | (`server_vad`) Speaker stopped; the server auto-commits the turn |
| `input_audio_buffer.committed` | Acknowledges a commit — client `commit` or `server_vad` auto-commit |
| `conversation.item.input_audio_transcription.delta` | Incremental transcript token(s) |
| `conversation.item.input_audio_transcription.completed` | Final transcript for the item |
| `error` | Error message; the socket may close afterwards |

**Minimal Python client:**

```python
import asyncio, base64, json, websockets, numpy as np, soundfile as sf

async def transcribe(path: str):
    audio, sr = sf.read(path, dtype="int16", always_2d=False)
    uri = "ws://localhost:8000/v1/realtime?model=iris-sfg/Voxtral-Mini-4B-Realtime-2602-4bit"
    async with websockets.connect(uri) as ws:
        # Wait for session.created
        await ws.recv()
        # Declare the input rate (the server will resample to the model's rate)
        await ws.send(json.dumps({
            "type": "session.update",
            "session": {"audio": {"input": {"format": {"type": "audio/pcm", "rate": sr}}}},
        }))
        # Stream audio in 100 ms chunks
        step = sr // 10
        for i in range(0, len(audio), step):
            chunk = audio[i : i + step].tobytes()
            await ws.send(json.dumps({
                "type": "input_audio_buffer.append",
                "audio": base64.b64encode(chunk).decode(),
            }))
        await ws.send(json.dumps({"type": "input_audio_buffer.commit"}))

        async for raw in ws:
            evt = json.loads(raw)
            if evt["type"].endswith(".delta"):
                print(evt["delta"], end="", flush=True)
            elif evt["type"].endswith(".completed"):
                print()
                break

asyncio.run(transcribe("audio.wav"))
```

**Tuning transcription delay** (for models like `voxtral_realtime` that expose the knob):

```bash
mlx_audio.server \
  --realtime-model iris-sfg/Voxtral-Mini-4B-Realtime-2602-4bit \
  --realtime-transcription-delay-ms 960
```

Lower values reduce latency at the cost of accuracy. Models that don't declare a `transcription_delay_ms` parameter silently ignore the flag.

!!! note "Single-process MLX serialization"
    MLX inference on a given device is serialized inside the server. `/v1/realtime` schedules `session.step()` calls cooperatively — two concurrent websocket clients share GPU time rather than running in parallel. This is fine as long as each stream transcribes faster than real-time; otherwise run multiple server instances behind a load balancer.

## Web Interface (Studio UI)

The Studio UI is a Next.js application located in `mlx_audio/ui/`. It provides:

- A text input for TTS generation with voice and model selectors
- Audio upload and recording for STT
- 3D audio visualization
- Model loading and management

### Running the UI Separately

If you prefer to run the UI outside the server process:

```bash
cd mlx_audio/ui
npm install
npm run dev
```

The UI runs on `http://localhost:3000` and expects the API server at `http://localhost:8000`.

## Using with the OpenAI Python Client

Because the API is OpenAI-compatible, you can use the official OpenAI Python client:

```python
from openai import OpenAI

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="not-needed",  # MLX Audio does not require an API key
)

# Text-to-Speech
response = client.audio.speech.create(
    model="mlx-community/Kokoro-82M-bf16",
    voice="af_heart",
    input="Hello from the OpenAI client!",
)
response.stream_to_file("output.mp3")

# Speech-to-Text
with open("audio.wav", "rb") as f:
    transcript = client.audio.transcriptions.create(
        model="mlx-community/whisper-large-v3-turbo-asr-fp16",
        file=f,
    )
print(transcript.text)
```

## Installation

The server requires the `server` optional dependency group:

```bash
pip install "mlx-audio[server]"
```

This installs FastAPI, Uvicorn, and python-multipart.

---

## File: docs/getting-started/index.md

# Getting Started

Welcome to MLX Audio! This section will help you install the library and run your first text-to-speech generation or speech transcription in minutes.

## Requirements

- **Python 3.10+**
- **Apple Silicon Mac** (M1 / M2 / M3 / M4)
- **ffmpeg** (required for MP3, FLAC, OGG, Opus, or Vorbis encoding; WAV works without it)

## Where to Go Next



-   :octicons-download-24:{ .lg .middle } **Installation**

    ---

    Install mlx-audio with pip or uv, including optional extras for TTS, STT, and development.

    [:octicons-arrow-right-24: Installation](installation.md)

-   :octicons-terminal-24:{ .lg .middle } **Quick Start: CLI**

    ---

    Generate speech and transcribe audio from the command line.

    [:octicons-arrow-right-24: CLI Quick Start](quickstart-cli.md)

-   :octicons-code-24:{ .lg .middle } **Quick Start: Python**

    ---

    Use the Python API to generate, transcribe, and process audio programmatically.

    [:octicons-arrow-right-24: Python Quick Start](quickstart-python.md)

---

## File: docs/getting-started/installation.md

# Installation

## Quick Install

=== "pip"

    ```bash
    pip install mlx-audio
    ```

=== "uv (tool)"

    Install the CLI tools globally with uv:

    ```bash
    # Latest release from PyPI
    uv tool install --force mlx-audio --prerelease=allow
    ```

    Or install from the latest source:

    ```bash
    # Latest code from GitHub
    uv tool install --force git+https://github.com/Blaizzy/mlx-audio.git --prerelease=allow
    ```

## Optional Extras

mlx-audio ships optional dependency groups so you only install what you need:

| Extra | What it includes |
|-------|-----------------|
| `tts` | Text-to-speech dependencies |
| `stt` | Speech-to-text dependencies |
| `sts` | Speech-to-speech dependencies |
| `docs` | MkDocs and API reference tooling |
| `all` | Everything (TTS + STT + STS) |
| `dev` | Development and testing tools |

Install extras with bracket syntax:

```bash
# TTS only
pip install "mlx-audio[tts]"

# STT only
pip install "mlx-audio[stt]"

# Everything
pip install "mlx-audio[all]"

# Documentation tooling
pip install "mlx-audio[docs]"
```

## Development Setup

Clone the repository and install in editable mode with dev dependencies:

```bash
git clone https://github.com/Blaizzy/mlx-audio.git
cd mlx-audio
pip install -e ".[dev]"
```

If you are working on the docs site, use:

```bash
pip install -e ".[dev,docs]"
```

## ffmpeg

ffmpeg is required for saving audio in **MP3**, **FLAC**, **OGG**, **Opus**, or **Vorbis** format. WAV works without it.

=== "macOS (Homebrew)"

    ```bash
    brew install ffmpeg
    ```

=== "Ubuntu / Debian"

    ```bash
    sudo apt install ffmpeg
    ```

!!! tip
    If you only need WAV output, you can skip installing ffmpeg entirely.

---

## File: docs/getting-started/quickstart-cli.md

# Quick Start: CLI

mlx-audio provides command-line tools for both text-to-speech generation and speech-to-text transcription.

## Text-to-Speech

!!! note
    These TTS quickstart examples use `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit`.

### Basic Generation

Generate speech from text with a single command:

```bash
mlx_audio.tts.generate \
    --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit \
    --text "Hello, world!" \
    --voice Chelsie \
    --lang_code English
```

### Play Audio Immediately

Add `--play` to hear the result without saving:

```bash
mlx_audio.tts.generate \
    --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit \
    --text "Hello, world!" \
    --voice Chelsie \
    --lang_code English \
    --play
```

### Voice and Language Selection

Choose a voice preset and provide a language hint:

```bash
mlx_audio.tts.generate \
    --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit \
    --text "Welcome to MLX-Audio!" \
    --voice Ethan \
    --lang_code English
```

### Save to a Directory

```bash
mlx_audio.tts.generate \
    --model mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit \
    --text "Hello!" \
    --voice Chelsie \
    --lang_code English \
    --output_path ./my_audio
```

### Voice Cloning (CSM)

Clone a voice from a reference audio file:

```bash
mlx_audio.tts.generate \
    --model mlx-community/csm-1b \
    --text "Hello from Sesame." \
    --ref_audio ./reference_voice.wav \
    --play
```

## Speech-to-Text

### Transcribe with Whisper

```bash
python -m mlx_audio.stt.generate \
    --model mlx-community/whisper-large-v3-turbo-asr-fp16 \
    --audio audio.wav \
    --output-path output \
    --format json \
    --verbose
```

### Transcribe with Parakeet

```bash
python -m mlx_audio.stt.generate \
    --model mlx-community/parakeet-tdt-0.6b-v3 \
    --audio speech.wav \
    --output-path output \
    --format json \
    --verbose
```

### Transcribe with VibeVoice-ASR

```bash
python -m mlx_audio.stt.generate \
    --model mlx-community/VibeVoice-ASR-bf16 \
    --audio meeting.wav \
    --output-path output \
    --format json \
    --max-tokens 8192 \
    --verbose
```

Add context/hotwords for better accuracy on domain-specific terms:

```bash
python -m mlx_audio.stt.generate \
    --model mlx-community/VibeVoice-ASR-bf16 \
    --audio technical_talk.wav \
    --output-path output \
    --format json \
    --max-tokens 8192 \
    --context "MLX, Apple Silicon, PyTorch, Transformer" \
    --verbose
```

## Common Flags

| Flag | Description |
|------|-------------|
| `--model` | Hugging Face model ID or local path |
| `--text` | Input text for TTS generation |
| `--audio` | Input audio file for STT transcription |
| `--voice` | Voice preset name (e.g., `Chelsie`, `Ethan`, `casual_male`) |
| `--speed` | Speech speed multiplier (default: `1.0`) |
| `--lang_code` | Language hint (e.g., `English`, `Chinese`, or `auto`) |
| `--play` | Play audio immediately after generation |
| `--output_path` | Directory to save output files |
| `--verbose` | Show detailed generation info |
| `--format` | Output format for STT (`json`, etc.) |

!!! note
    Different models support different flags. Check the [Models](../models/index.md) section for model-specific options.

---

## File: docs/getting-started/quickstart-python.md

# Quick Start: Python

Use the mlx-audio Python API to generate speech, transcribe audio, and process audio programmatically.

## Text-to-Speech

!!! note
    These TTS quickstart examples use `mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit`.

### Basic Generation

```python
from mlx_audio.tts.utils import load_model

# Load a TTS model
model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit")

# Generate speech
for result in model.generate(
    "Hello from MLX-Audio!",
    voice="Chelsie",
    lang_code="English",
):
    print(f"Generated {result.audio.shape[0]} samples")
    # result.audio contains the waveform as mx.array
```

### Voice and Language Options

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit")

for result in model.generate(
    text="Welcome to MLX-Audio!",
    voice="Ethan",
    lang_code="English",
):
    audio = result.audio
```

!!! tip "Qwen3-TTS Base Model"
    Pass `voice` to pick a preset speaker and `lang_code` to hint the language, for example `voice="Chelsie"` and `lang_code="English"`.

### Voxtral TTS

```python
from mlx_audio.tts.utils import load

model = load("mlx-community/Voxtral-4B-TTS-2603-mlx-bf16")

for result in model.generate(text="Hello, how are you today?", voice="casual_male"):
    print(result.audio_duration)
```

### Qwen3-TTS

```python
from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/Qwen3-TTS-12Hz-1.7B-Base-8bit")
results = list(model.generate(
    text="Hello, welcome to MLX-Audio!",
    voice="Chelsie",
    lang_code="English",
))

audio = results[0].audio  # mx.array
```

## Speech-to-Text

### Whisper

```python
from mlx_audio.stt.generate import generate_transcription

result = generate_transcription(
    model="mlx-community/whisper-large-v3-turbo-asr-fp16",
    audio="audio.wav",
)
print(result.text)
```

### Qwen3-ASR

```python
from mlx_audio.stt import load

# Speech recognition
model = load("mlx-community/Qwen3-ASR-0.6B-8bit")
result = model.generate("audio.wav", language="English")
print(result.text)
```

### Word-Level Alignment (Qwen3-ForcedAligner)

```python
from mlx_audio.stt import load

aligner = load("mlx-community/Qwen3-ForcedAligner-0.6B-8bit")
result = aligner.generate("audio.wav", text="I have a dream", language="English")

for item in result:
    print(f"[{item.start_time:.2f}s - {item.end_time:.2f}s] {item.text}")
```

### Parakeet (with Timestamps)

```python
from mlx_audio.stt.utils import load

model = load("mlx-community/parakeet-tdt-0.6b-v3")
result = model.generate("audio.wav")

print(f"Text: {result.text}")

# Word-level timestamps
for sentence in result.sentences:
    print(f"[{sentence.start:.2f}s - {sentence.end:.2f}s] {sentence.text}")
```

### Streaming Transcription

Several STT models support streaming for low-latency output:

=== "Voxtral Realtime"

    ```python
    from mlx_audio.stt.utils import load

    model = load("mlx-community/Voxtral-Mini-4B-Realtime-2602-4bit")

    for chunk in model.generate("audio.wav", stream=True):
        print(chunk, end="", flush=True)
    ```

=== "Parakeet"

    ```python
    from mlx_audio.stt.utils import load

    model = load("mlx-community/parakeet-tdt-0.6b-v3")

    for chunk in model.generate("long_audio.wav", stream=True):
        print(chunk.text, end="", flush=True)
    ```

=== "VibeVoice"

    ```python
    from mlx_audio.stt.utils import load

    model = load("mlx-community/VibeVoice-ASR-bf16")

    for text in model.stream_transcribe(audio="speech.wav", max_tokens=4096):
        print(text, end="", flush=True)
    ```

## Saving Audio

### Save with STS Utilities

```python
from mlx_audio.sts import save_audio

# Save an audio array to a WAV file
save_audio(audio_array, "output.wav", sample_rate=24000)
```

!!! note
    Saving to MP3, FLAC, OGG, Opus, or Vorbis requires [ffmpeg](installation.md#ffmpeg). WAV works without it.