sherpa-onnx Documentation

File: README.md Copy Chapter

Supported functions Copy Section

Supported platforms Copy Section

Supported programming languages Copy Section

It also supports WebAssembly.

Supported NPUs Copy Section

Join our discord

Introduction Copy Chapter

This repository supports running the following functions locally

• Speech-to-text (i.e., ASR); both streaming and non-streaming are supported
• Text-to-speech (i.e., TTS)
• Speaker diarization
• Speaker identification
• Speaker verification
• Spoken language identification
• Audio tagging
• VAD (e.g., silero-vad)
• Speech enhancement (e.g., gtcrn, DPDFNet)
• Keyword spotting
• Source separation (e.g., spleeter, UVR)

on the following platforms and operating systems:

• x86, x86_64, 32-bit ARM, 64-bit ARM (arm64, aarch64), RISC-V (riscv64), RK NPU, Ascend NPU
• Linux, macOS, Windows, openKylin
• Android, WearOS
• iOS
• HarmonyOS
• NodeJS
• WebAssembly
• NVIDIA Jetson Orin NX (Support running on both CPU and GPU)
• NVIDIA Jetson Nano B01 (Support running on both CPU and GPU)
• Raspberry Pi
• RV1126
• LicheePi4A
• VisionFive 2
• 旭日X3派
• 爱芯派
• RK3588
• SpacemiT-K1
• SpacemiT-K3
• etc

with the following APIs

• C++, C, Python, Go, C#
• Java, Kotlin, JavaScript
• Swift, Rust
• Dart, Object Pascal

Links for Huggingface Spaces Copy Section

You can visit the following Huggingface spaces to try sherpa-onnx without
installing anything. All you need is a browser.

We also have spaces built using WebAssembly. They are listed below:

Links for pre-built Android APKs Copy Section

You can find pre-built Android APKs for this repository in the following table

Links for pre-built Flutter APPs Copy Section

Real-time speech recognition

Text-to-speech

Note: You need to build from source for iOS.

Links for pre-built Lazarus APPs Copy Section

Generating subtitles

Links for pre-trained models Copy Section

Some pre-trained ASR models (Streaming)

Please see

• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/online-transducer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/online-paraformer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/online-ctc/index.html

for more models. The following table lists only SOME of them.

Some pre-trained ASR models (Non-Streaming)

Please see

• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-transducer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-paraformer/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/offline-ctc/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/telespeech/index.html
• https://k2-fsa.github.io/sherpa/onnx/pretrained_models/whisper/index.html

for more models. The following table lists only SOME of them.

Useful links Copy Section

• Documentation: https://k2-fsa.github.io/sherpa/onnx/
• Bilibili 演示视频: https://search.bilibili.com/all?keyword=%E6%96%B0%E4%B8%80%E4%BB%A3Kaldi

How to reach us Copy Section

Please see
https://k2-fsa.github.io/sherpa/social-groups.html
for 新一代 Kaldi 微信交流群 and QQ 交流群.

Projects using sherpa-onnx Copy Chapter

[Sherpa Voice / @siteed/sherpa-onnx.rn](https://github.com/deeeed/audiolab) Copy Section

React Native wrapper and demo app for validating sherpa-onnx on iOS,
Android, and Web, including ASR, TTS, VAD, KWS, speaker ID, diarization,
language ID, punctuation, audio tagging, and speech enhancement.

• NPM package
• Live demo

[Speed of Sound](https://github.com/zugaldia/speedofsound) Copy Section

A voice-typing application for the Linux desktop (GTK4/Adwaita).
It captures microphone audio, transcribes it offline using Sherpa ONNX ASR models,
optionally polishes the text with an LLM, and types the result into the active window
via XDG Remote Desktop Portal keyboard simulation.

[VoxSherpa TTS](https://github.com/CodeBySonu95/VoxSherpa-TTS) Copy Section

VoxSherpa TTS is a 100% offline Android Text-to-Speech app powered by Sherpa-ONNX.
It supports Kokoro-82M, Piper, and VITS engines with multilingual support including
Hindi, English, British English, Japanese, Chinese and 50+ more languages.

• Download APK (All Versions)
• Android 11+ · 100% offline · No telemetry

[BreezeApp](https://github.com/mtkresearch/BreezeApp) from [MediaTek Research](https://github.com/mtkresearch) Copy Section

BreezeAPP is a mobile AI application developed for both Android and iOS platforms.
Users can download it directly from the App Store and enjoy a variety of features
offline, including speech-to-text, text-to-speech, text-based chatbot interactions,
and image question-answering

• Download APK for BreezeAPP
• APK 中国镜像

[Open-LLM-VTuber](https://github.com/t41372/Open-LLM-VTuber) Copy Section

Talk to any LLM with hands-free voice interaction, voice interruption, and Live2D taking
face running locally across platforms

See also https://github.com/t41372/Open-LLM-VTuber/pull/50

[voiceapi](https://github.com/ruzhila/voiceapi) Copy Section

Streaming ASR and TTS based on FastAPI

It shows how to use the ASR and TTS Python APIs with FastAPI.

[腾讯会议摸鱼工具 TMSpeech](https://github.com/jxlpzqc/TMSpeech) Copy Section

Uses streaming ASR in C# with graphical user interface.

Video demo in Chinese: 【开源】Windows实时字幕软件（网课/开会必备）

[lol互动助手](https://github.com/l1veIn/lol-wom-electron) Copy Section

It uses the JavaScript API of sherpa-onnx along with Electron

Video demo in Chinese: 爆了！炫神教你开打字挂！真正影响胜率的英雄联盟工具！英雄联盟的最后一块拼图！和游戏中的每个人无障碍沟通！

[Sherpa-ONNX 语音识别服务器](https://github.com/hfyydd/sherpa-onnx-server) Copy Section

A server based on nodejs providing Restful API for speech recognition.

[QSmartAssistant](https://github.com/xinhecuican/QSmartAssistant) Copy Section

一个模块化，全过程可离线，低占用率的对话机器人/智能音箱

It uses QT. Both ASR
and TTS
are used.

[Flutter-EasySpeechRecognition](https://github.com/Jason-chen-coder/Flutter-EasySpeechRecognition) Copy Section

It extends ./flutter-examples/streaming_asr by
downloading models inside the app to reduce the size of the app.

Note: [Team B] Sherpa AI backend also uses
sherpa-onnx in a Flutter APP.

[sherpa-onnx-unity](https://github.com/xue-fei/sherpa-onnx-unity) Copy Section

sherpa-onnx in Unity. See also #1695,
#1892, and #1859

[xiaozhi-esp32-server](https://github.com/xinnan-tech/xiaozhi-esp32-server) Copy Section

本项目为xiaozhi-esp32提供后端服务，帮助您快速搭建ESP32设备控制服务器
Backend service for xiaozhi-esp32, helps you quickly build an ESP32 device control server.

See also

• ASR新增轻量级sherpa-onnx-asr
• feat: ASR增加sherpa-onnx模型

[KaithemAutomation](https://github.com/EternityForest/KaithemAutomation) Copy Section

Pure Python, GUI-focused home automation/consumer grade SCADA.

It uses TTS from sherpa-onnx. See also ✨ Speak command that uses the new globally configured TTS model.

[Open-XiaoAI KWS](https://github.com/idootop/open-xiaoai-kws) Copy Section

Enable custom wake word for XiaoAi Speakers. 让小爱音箱支持自定义唤醒词。

Video demo in Chinese: 小爱同学启动～˶╹ꇴ╹˶！

[C++ WebSocket ASR Server](https://github.com/mawwalker/stt-server) Copy Section

It provides a WebSocket server based on C++ for ASR using sherpa-onnx.

[Go WebSocket Server](https://github.com/bbeyondllove/asr_server) Copy Section

It provides a WebSocket server based on the Go programming language for sherpa-onnx.

[Making robot Paimon, Ep10 "The AI Part 1"](https://www.youtube.com/watch?v=KxPKkwxGWZs) Copy Section

It is a YouTube video,
showing how the author tried to use AI so he can have a conversation with Paimon.

It uses sherpa-onnx for speech-to-text and text-to-speech.

[TtsReader - Desktop application](https://github.com/ys-pro-duction/TtsReader) Copy Section

A desktop text-to-speech application built using Kotlin Multiplatform.

[MentraOS](https://github.com/Mentra-Community/MentraOS) Copy Section

Smart glasses OS, with dozens of built-in apps. Users get AI assistant, notifications,
translation, screen mirror, captions, and more. Devs get to write 1 app that runs on
any pair of smart glasses.

It uses sherpa-onnx for real-time speech recognition on iOS and Android devices.
See also https://github.com/Mentra-Community/MentraOS/pull/861

It uses Swift for iOS and Java for Android.

[flet_sherpa_onnx](https://github.com/SamYuan1990/flet_sherpa_onnx) Copy Section

Flet ASR/STT component based on sherpa-onnx.
Example a chat box agent

[achatbot-go](https://github.com/ai-bot-pro/achatbot-go) Copy Section

a multimodal chatbot based on go with sherpa-onnx's speech lib api.

[fcitx5-vinput](https://github.com/xifan2333/fcitx5-vinput) Copy Section

Local offline voice input plugin for Fcitx5 (Linux input method framework).
It uses C++ with offline ASR for speech recognition, supporting push-to-talk,
command mode, and optional LLM post-processing.

Video demo in Chinese: fcitx5-vinput

[Wake Word](https://github.com/analyticsinmotion/wake-word) Copy Section

A VS Code extension for hands-free voice-activated coding. It uses sherpa-onnx for real-time
keyword spotting (KWS) to detect custom wake phrases and trigger VS Code commands by voice.
Audio capture is handled by decibri, a
cross-platform Node.js microphone streaming library with prebuilt native binaries.

• VS Code Marketplace
• Open VSX
• decibri integration guides for sherpa-onnx

[SmartSub](https://github.com/buxuku/SmartSub) Copy Section

SmartSub is a local-first cross-platform desktop application for the complete subtitle production pipeline: audio/video transcription, subtitle translation, proofreading, and subtitle burning/muxing.

It natively integrates sherpa-onnx to power three offline ASR engines — FunASR, Qwen3-ASR, and FireRedASR — delivering high-accuracy Chinese and multilingual speech recognition entirely on-device, with no file uploads required.

[Fono](https://github.com/bogdanr/fono) Copy Section

Talk to your computer. Fono is an open-source (GPL-3.0), complete voice-AI
stack (speech-to-text, text-to-speech, a local LLM, wake word, speaker ID) in
one small binary for the Linux, macOS, and Windows desktop. Press a key and
speak: it types into any app, answers as a voice assistant, or drives a coding
agent.

It uses sherpa-onnx's Supertonic TTS models for text-to-speech, with the
Supertonic text frontend ported to Rust. See also
#3750, which fixes dropped
diacritics in the Supertonic frontend for Romanian, Czech, Polish, Hungarian,
Turkish, Vietnamese, and other non-English Latin-script languages.

[Domia](https://github.com/domia-ai/domia-core) Copy Section

Domia is an open-source local AI companion you talk to, living entirely on
hardware you own. Each Domia is its own character, with a distinct
personality, voice, emotional state, and memory of you and its place — and it
acts, not just talks: skills connect it to the real world over MCP, Home
Assistant included. Domia nodes form a local peer-to-peer mesh that shares
speech and reasoning compute, while room satellites bring each companion into
the spaces where you live without giving up its identity. No cloud, no
accounts, and no data leaving your network.

Domia runs sherpa-onnx in-process for keyword spotting, voice activity
detection, streaming and offline speech recognition, and text-to-speech.
Speech engines and models are selected through per-companion configuration,
making it easy to match supported sherpa-onnx models to different hardware.

[qwen-audio-agent](https://github.com/QwenAudio/qwen-audio-agent) Copy Section

A realtime full-duplex voice runtime for AI coding agents
(Claude Code, Codex, OpenCode, and other ACP agents).
It uses sherpa-onnx keyword spotting with the 3M-parameter
zh-en transducer model to implement a local voice wake word:
the app sleeps on idle but keeps the microphone open,
and the wake phrase brings the session back with no cloud calls.

File: android/SherpaOnnxAar/README.md Copy Chapter

Usage of this project

git clone https://github.com/k2-fsa/sherpa-onnx
cd sherpa-onnx

wget https://github.com/k2-fsa/sherpa-onnx/releases/download/v1.13.5/sherpa-onnx-v1.13.5-android.tar.bz2
tar xvf sherpa-onnx-v1.13.5-android.tar.bz2

cp -v jniLibs/arm64-v8a/* android/SherpaOnnxAar/sherpa_onnx/src/main/jniLibs/arm64-v8a/
cp -v jniLibs/armeabi-v7a/* android/SherpaOnnxAar/sherpa_onnx/src/main/jniLibs/armeabi-v7a/
cp -v jniLibs/x86/* android/SherpaOnnxAar/sherpa_onnx/src/main/jniLibs/x86/
cp -v jniLibs/x86_64/* android/SherpaOnnxAar/sherpa_onnx/src/main/jniLibs/x86_64/

cd android/SherpaOnnxAar

./gradlew :sherpa_onnx:assembleRelease
ls -lh ./sherpa_onnx/build/outputs/aar/sherpa_onnx-release.aar
cp ./sherpa_onnx/build/outputs/aar/sherpa_onnx-release.aar ../../sherpa-onnx-1.13.5.aar

File: android/SherpaOnnxJavaDemo/README.md Copy Chapter

Introduction

Please run the following commands to download model files before you run this Android demo:

# Assume we are inside
# /Users/fangjun/open-source/sherpa-onnx/android/SherpaOnnxJavaDemo

cd app/src/main/assets/
wget https://github.com/k2-fsa/sherpa-onnx/releases/download/asr-models/sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20.tar.bz2

tar xvf sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20.tar.bz2
rm sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20.tar.bz2

mv sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/encoder-epoch-99-avg-1.int8.onnx ./
mv sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/decoder-epoch-99-avg-1.onnx ./
mv sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/joiner-epoch-99-avg-1.int8.onnx ./
mv sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/tokens.txt ./

rm -rf sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/*

mv encoder-epoch-99-avg-1.int8.onnx sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/
mv decoder-epoch-99-avg-1.onnx sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/
mv joiner-epoch-99-avg-1.int8.onnx sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/
mv tokens.txt sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20/

You should have the following directory structure:

(py38) fangjuns-MacBook-Pro:assets fangjun$ pwd
/Users/fangjun/open-source/sherpa-onnx/android/SherpaOnnxJavaDemo/app/src/main/assets

(py38) fangjuns-MacBook-Pro:assets fangjun$ tree .
.
└── sherpa-onnx-streaming-zipformer-bilingual-zh-en-2023-02-20
    ├── decoder-epoch-99-avg-1.onnx
    ├── encoder-epoch-99-avg-1.int8.onnx
    ├── joiner-epoch-99-avg-1.int8.onnx
    └── tokens.txt

1 directory, 4 files

Remember to remove unused files to reduce the file size of the final APK.

File: android/README.md Copy Chapter

Introduction

Please refer to
https://k2-fsa.github.io/sherpa/onnx/android/index.html
for usage.

File: c-api-examples/asr-microphone-example/README.md Copy Chapter

Introduction

This folder contains examples for real-time speech recognition from a microphone
using sherpa-onnx C API.

Note: You can call C API from C++ files.

./c-api-alsa.cc Copy Chapter

This file uses alsa to read a microphone. It runs only on Linux. This file
does not support macOS or Windows.

File: c-api-examples/README.md Copy Chapter

Introduction

This folder contains C API examples for sherpa-onnx.

Please refer to the documentation
https://k2-fsa.github.io/sherpa/onnx/c-api/index.html
for details.

File descriptions Copy Chapter

• decode-file-c-api.c This file shows how to use the C API
  for speech recognition with a streaming model.

• offline-tts-c-api.c This file shows how to use the C API
  to convert text to speech with a non-streaming model.

• speech-enhancement-gtcrn-c-api.c
  This file shows how to use the C API for speech enhancement with GTCRN
  models.

• speech-enhancement-dpdfnet-c-api.c
  This file shows how to use the C API for speech enhancement with DPDFNet
  models. Use 16 kHz DPDFNet models such as dpdfnet_baseline.onnx,
  dpdfnet2.onnx, dpdfnet4.onnx, or dpdfnet8.onnx for downstream ASR and
  dpdfnet2_48khz_hr.onnx for 48 kHz enhancement output.

• online-speech-enhancement-gtcrn-c-api.c
  This file shows how to use the C API for online speech enhancement with
  GTCRN models.

• online-speech-enhancement-dpdfnet-c-api.c
  This file shows how to use the C API for online speech enhancement with
  DPDFNet models. Use dpdfnet_baseline.onnx, dpdfnet2.onnx,
  dpdfnet4.onnx, or dpdfnet8.onnx for 16 kHz output.

File: dart-api-examples/add-punctuations/README.md Copy Chapter

Introduction

This example shows how to use the Dart API from sherpa-onnx to add punctuations to text.

File: dart-api-examples/audio-tagging/README.md Copy Chapter

Introduction

This example shows how to use the Dart API from sherpa-onnx for audio tagging.

File: dart-api-examples/keyword-spotter/README.md Copy Chapter

Introduction

This directory contains keyword spotting examples using
Dart API from sherpa-onnx

File: dart-api-examples/non-streaming-asr/README.md Copy Chapter

Introduction

This folder contains examples for non-streaming ASR with Dart API.