# Technical Documentation: snakers4/silero-vad > ℹ️ **Provenance:** Multi-Tier Ingestion · [CodeWiki Reference](https://codewiki.google/github.com/snakers4/silero-vad) ### Overview (`README.md`) [](mailto:hello@silero.ai) [](https://t.me/silero_speech) [](https://github.com/snakers4/silero-vad/blob/master/LICENSE) [](https://pypi.org/project/silero-vad/) [](https://colab.research.google.com/github/snakers4/silero-vad/blob/master/silero-vad.ipynb) [](https://github.com/snakers4/silero-vad/actions/workflows/test.yml) [](https://pypi.org/project/silero-vad/) [](https://pypi.org/project/silero-vad)

Silero VAD


**Silero VAD** - pre-trained enterprise-grade [Voice Activity Detector](https://en.wikipedia.org/wiki/Voice_activity_detection) (also see our [STT models](https://github.com/snakers4/silero-models)).

Real Time Example https://user-images.githubusercontent.com/36505480/144874384-95f80f6d-a4f1-42cc-9be7-004c891dd481.mp4 Please note, that video loads only if you are logged in your GitHub account.

Fast start


Dependencies System requirements to run python examples on `x86-64` systems: - `python 3.8+`; - 1G+ RAM; - A modern CPU with AVX, AVX2, AVX-512 or AMX instruction sets. Required: - `torch>=1.12.0`. Everything else is optional and installed via extras: | Extra | Installs | Needed for | |---|---|---| | `silero-vad[audio]` | `torchaudio>=0.12.0,<2.10`, plus `torchcodec` on Python 3.9+ | `read_audio` / `save_audio` | | `silero-vad[codec]` | `torchcodec` | `read_audio` / `save_audio` with no torchaudio at all | | `silero-vad[onnx-cpu]` | `onnxruntime>=1.16.1`, `numpy` | ONNX and sequence models | | `silero-vad[onnx-gpu]` | `onnxruntime-gpu>=1.16.1`, `numpy` | ONNX on GPU | | `silero-vad[all]` | all of the above | everything | `[audio]` pulls in torchcodec on Python 3.9+ because `torchaudio>=2.9` hands decoding over to it and does not work without it; on Python 3.8 pip resolves an older torchaudio that decodes on its own. torchcodec itself needs FFmpeg (4-9) on the system. The model itself needs only `torch` — if you already load audio yourself, pass a 1-D float32 `torch.Tensor` straight to `get_speech_timestamps` and install nothing extra. The bundled I/O helpers (`read_audio`, `save_audio`) work with **either** backend: torchaudio if present, otherwise torchcodec. Note that torchcodec resamples through FFmpeg rather than `torchaudio.transforms.Resample`, so results can differ by ~1e-3 on files that need resampling. With torchaudio, a proper audio backend is required: - Option №1 - [**FFmpeg**](https://www.ffmpeg.org/) backend. `conda install -c conda-forge 'ffmpeg<7'`; - Option №2 - [**sox_io**](https://pypi.org/project/sox/) backend. `apt-get install sox`, TorchAudio is tested on libsox 14.4.2; - Option №3 - [**soundfile**](https://pypi.org/project/soundfile/) backend. `pip install soundfile`. If you are planning to run the VAD using solely the `onnx-runtime`, it will run on any other system architectures where onnx-runtume is [supported](https://onnxruntime.ai/getting-started). In this case please note that: - You will have to implement the I/O; - You will have to adapt the existing wrappers / examples / post-processing for your use-case.
**Using pip**: `pip install silero-vad[audio]` (or plain `pip install silero-vad` if you load audio yourself) ```python3 from silero_vad import load_silero_vad, read_audio, get_speech_timestamps model = load_silero_vad() wav = read_audio('path_to_audio_file') speech_timestamps = get_speech_timestamps( wav, model, return_seconds=True, # Return speech timestamps in seconds (default is samples) ) ``` **Using torch.hub**: ```python3 import torch torch.set_num_threads(1) model, utils = torch.hub.load(repo_or_dir='snakers4/silero-vad', model='silero_vad') (get_speech_timestamps, _, read_audio, _, _) = utils wav = read_audio('path_to_audio_file') speech_timestamps = get_speech_timestamps( wav, model, return_seconds=True, # Return speech timestamps in seconds (default is samples) ) ```

Key Features


- **Stellar accuracy** Silero VAD has [excellent results](https://github.com/snakers4/silero-vad/wiki/Quality-Metrics#vs-other-available-solutions) on speech detection tasks. - **Fast** One audio chunk (30+ ms) [takes](https://github.com/snakers4/silero-vad/wiki/Performance-Metrics#silero-vad-performance-metrics) less than **1ms** to be processed on a single CPU thread. Using batching or GPU can also improve performance considerably. Under certain conditions ONNX may even run up to 4-5x faster. - **Lightweight** JIT model is around two megabytes in size. - **General** Silero VAD was trained on huge corpora that include over **6000** languages and it performs well on audios from different domains with various background noise and quality levels. - **Flexible sampling rate** Silero VAD [supports](https://github.com/snakers4/silero-vad/wiki/Quality-Metrics#sample-rate-comparison) **8000 Hz** and **16000 Hz** [sampling rates](https://en.wikipedia.org/wiki/Sampling_(signal_processing)#Sampling_rate). - **Highly Portable** Silero VAD reaps benefits from the rich ecosystems built around **PyTorch** and **ONNX** running everywhere where these runtimes are available. - **No Strings Attached** Published under permissive license (MIT) Silero VAD has zero strings attached - no telemetry, no keys, no registration, no built-in expiration, no keys or vendor lock.

Typical Use Cases


- Voice activity detection for IOT / edge / mobile use cases - Data cleaning and preparation, voice detection in general - Telephony and call-center automation, voice bots - Voice interfaces

Links


- [Examples and Dependencies](https://github.com/snakers4/silero-vad/wiki/Examples-and-Dependencies#dependencies) - [Quality Metrics](https://github.com/snakers4/silero-vad/wiki/Quality-Metrics) - [Performance Metrics](https://github.com/snakers4/silero-vad/wiki/Performance-Metrics) - [Versions and Available Models](https://github.com/snakers4/silero-vad/wiki/Version-history-and-Available-Models) - [Further reading](https://github.com/snakers4/silero-models#further-reading) - [FAQ](https://github.com/snakers4/silero-vad/wiki/FAQ)

Get In Touch


Try our models, create an [issue](https://github.com/snakers4/silero-vad/issues/new), start a [discussion](https://github.com/snakers4/silero-vad/discussions/new), join our telegram [chat](https://t.me/silero_speech), [email](mailto:hello@silero.ai) us, read our [news](https://t.me/silero_news). Please see our [wiki](https://github.com/snakers4/silero-models/wiki) for relevant information and [email](mailto:hello@silero.ai) us directly. **Citations** ``` @misc{Silero VAD, author = {Silero Team}, title = {Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier}, year = {2024}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/snakers4/silero-vad}}, commit = {insert_some_commit_here}, email = {hello@silero.ai} } ```

Examples and VAD-based Community Apps


- Example of VAD ONNX Runtime model usage in [C++](https://github.com/snakers4/silero-vad/tree/master/examples/cpp) - Example of VAD [ExecuTorch](https://github.com/pytorch/executorch) model usage in [C++](https://github.com/pytorch/executorch/tree/main/examples/models/silero_vad) - Voice activity detection for the [browser](https://github.com/ricky0123/vad) using ONNX Runtime Web - [Rust](https://github.com/snakers4/silero-vad/tree/master/examples/rust-example), [Rust (wavekat-vad)](https://github.com/snakers4/silero-vad/tree/master/examples/rust-wav-processing-with-wavekat-vad), [Go](https://github.com/snakers4/silero-vad/tree/master/examples/go), [Java](https://github.com/snakers4/silero-vad/tree/master/examples/java-example), [C++](https://github.com/snakers4/silero-vad/tree/master/examples/cpp), [C#](https://github.com/snakers4/silero-vad/tree/master/examples/csharp) and [other](https://github.com/snakers4/silero-vad/tree/master/examples) community examples - OpenVINO conversion [guidelines](https://github.com/snakers4/silero-vad/tree/master/examples/openvino) - Optional [offline ONNX sequence inference](examples/onnx_sequence) for long recordings - A tinygrad [model](https://github.com/snakers4/silero-vad/blob/master/src/silero_vad/tinygrad_model.py) with a pico example in the docsting + separate [weights](https://github.com/snakers4/silero-vad/blob/master/src/silero_vad/data/silero_vad_16k.safetensors) in safetensors format (for simplicity we provided just the 16k model) --- ### Overview (`datasets/README.md`) # Датасет Silero-VAD > Датасет создан при поддержке Фонда содействия инновациям в рамках федерального проекта «Искусственный интеллект» национальной программы «Цифровая экономика Российской Федерации». По ссылкам ниже представлены `.feather` файлы, содержащие размеченные с помощью Silero VAD открытые наборы аудиоданных, а также короткое описание каждого набора данных с примерами загрузки. `.feather` файлы можно открыть с помощью библиотеки `pandas`: ```python3 import pandas as pd dataframe = pd.read_feather(PATH_TO_FEATHER_FILE) ``` Каждый `.feather` файл с разметкой содержит следующие колонки: - `speech_timings` - разметка данного аудио. Это список, содержащий словари вида `{'start': START_SECOND, 'end': END_SECOND}`, где `START_SECOND` и `END_SECOND` - время начала и конца речи в секундах. Количество данных словарей равно количеству речевых аудио отрывков, найденных в данном аудио; - `language` - ISO код языка данного аудио. Колонки, содержащие информацию о загрузке аудио файла различаются и описаны для каждого набора данных ниже. **Все данные размечены при временной дискретизации в ~30 миллисекунд (`num_samples` - 512)** | Название | Число часов | Число языков | Ссылка | Лицензия | md5sum | |----------------------|-------------|-------------|--------|----------|----------| | **Bible.is** | 53,138 | 1,596 | [URL](https://live.bible.is/) | [Уникальная](https://live.bible.is/terms) | ea404eeaf2cd283b8223f63002be11f9 | | **globalrecordings.net** | 9,743 | 6,171[^1] | [URL](https://globalrecordings.net/en) | CC BY-NC-SA 4.0 | 3c5c0f31b0abd9fe94ddbe8b1e2eb326 | | **VoxLingua107** | 6,628 | 107 | [URL](https://bark.phon.ioc.ee/voxlingua107/) | CC BY 4.0 | 5dfef33b4d091b6d399cfaf3d05f2140 | | **Common Voice** | 30,329 | 120 | [URL](https://commonvoice.mozilla.org/en/datasets) | CC0 | 5e30a85126adf74a5fd1496e6ac8695d | | **MLS** | 50,709 | 8 | [URL](https://www.openslr.org/94/) | CC BY 4.0 | a339d0e94bdf41bba3c003756254ac4e | | **Итого** | **150,547** | **6,171+** | | | | ## Bible.is [Ссылка на `.feather` файл с разметкой](https://models.silero.ai/vad_datasets/BibleIs.feather) - Колонка `audio_link` содержит ссылки на конкретные аудио файлы. ## globalrecordings.net [Ссылка на `.feather` файл с разметкой](https://models.silero.ai/vad_datasets/globalrecordings.feather) - Колонка `folder_link` содержит ссылки на скачивание `.zip` архива для конкретного языка. Внимание! Ссылки на архивы дублируются, т.к каждый архив может содержать множество аудио. - Колонка `audio_path` содержит пути до конкретного аудио после распаковки соответствующего архива из колонки `folder_link` ``Количество уникальных ISO кодов данного датасета не совпадает с фактическим количеством представленных языков, т.к некоторые близкие языки могут кодироваться одним и тем же ISO кодом.`` ## VoxLingua107 [Ссылка на `.feather` файл с разметкой](https://models.silero.ai/vad_datasets/VoxLingua107.feather) - Колонка `folder_link` содержит ссылки на скачивание `.zip` архива для конкретного языка. Внимание! Ссылки на архивы дублируются, т.к каждый архив может содержать множество аудио. - Колонка `audio_path` содержит пути до конкретного аудио после распаковки соответствующего архива из колонки `folder_link` ## Common Voice [Ссылка на `.feather` файл с разметкой](https://models.silero.ai/vad_datasets/common_voice.feather) Этот датасет невозможно скачать по статичным ссылкам. Для загрузки необходимо перейти по [ссылке](https://commonvoice.mozilla.org/en/datasets) и, получив доступ в соответствующей форме, скачать архивы для каждого доступного языка. Внимание! Представленная разметка актуальна для версии исходного датасета `Common Voice Corpus 16.1`. - Колонка `audio_path` содержит уникальные названия `.mp3` файлов, полученных после скачивания соответствующего датасета. ## MLS [Ссылка на `.feather` файл с разметкой](https://models.silero.ai/vad_datasets/MLS.feather) - Колонка `folder_link` содержит ссылки на скачивание `.zip` архива для конкретного языка. Внимание! Ссылки на архивы дублируются, т.к каждый архив может содержать множество аудио. - Колонка `audio_path` содержит пути до конкретного аудио после распаковки соответствующего архива из колонки `folder_link` ## Лицензия Данный датасет распространяется под [лицензией](https://creativecommons.org/licenses/by-nc-sa/4.0/deed.en) `CC BY-NC-SA 4.0`. ## Цитирование ``` @misc{Silero VAD Dataset, author = {Silero Team}, title = {Silero-VAD Dataset: a large public Internet-scale dataset for voice activity detection for 6000+ languages}, year = {2024}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/snakers4/silero-vad/datasets/README.md}}, email = {hello@silero.ai} } ``` [^1]: ``Количество уникальных ISO кодов данного датасета не совпадает с фактическим количеством представленных языков, т.к некоторые близкие языки могут кодироваться одним и тем же ISO кодом.`` --- ### Overview (`tuning/README.md`) # Тюнинг Silero-VAD модели > Код тюнинга создан при поддержке Фонда содействия инновациям в рамках федерального проекта «Искусственный интеллект» национальной программы «Цифровая экономика Российской Федерации». Тюнинг используется для улучшения качества детекции речи Silero-VAD модели на кастомных данных. ## Зависимости Следующие зависимости используются при тюнинге VAD модели: - `torchaudio>=0.12.0` - `omegaconf>=2.3.0` - `sklearn>=1.2.0` - `torch>=1.12.0` - `pandas>=2.2.2` - `tqdm` ## Подготовка данных Датафреймы для тюнинга должны быть подготовлены и сохранены в формате `.feather`. Следующие колонки в `.feather` файлах тренировки и валидации являются обязательными: - **audio_path** - абсолютный путь до аудиофайла в дисковой системе. Аудиофайлы должны представлять собой `PCM` данные, предпочтительно в форматах `.wav` или `.opus` (иные популярные форматы аудио тоже поддерживаются). Для ускорения темпа дообучения рекомендуется предварительно выполнить ресемплинг аудиофайлов (изменить частоту дискретизации) до 16000 Гц; - **speech_ts** - разметка для соответствующего аудиофайла. Список, состоящий из словарей формата `{'start': START_SEC, 'end': 'END_SEC'}`, где `START_SEC` и `END_SEC` - время начало и конца речевого отрезка в секундах соответственно. Для качественного дообучения рекомендуется использовать разметку с точностью до 30 миллисекунд. Чем больше данных используется на этапе дообучения, тем эффективнее показывает себя адаптированная модель на целевом домене. Длина аудио не ограничена, т.к. каждое аудио будет обрезано до `max_train_length_sec` секунд перед подачей в нейросеть. Длинные аудио лучше предварительно порезать на кусочки длины `max_train_length_sec`. Пример `.feather` датафрейма можно посмотреть в файле `example_dataframe.feather` ## Файл конфигурации `config.yml` Файл конфигурации `config.yml` содержит пути до обучающей и валидационной выборки, а также параметры дообучения: - `train_dataset_path` - абсолютный путь до тренировочного датафрейма в формате `.feather`. Должен содержать колонки `audio_path` и `speech_ts`, описанные в пункте "Подготовка данных". Пример устройства датафрейма можно посмотреть в `example_dataframe.feather`; - `val_dataset_path` - абсолютный путь до валидационного датафрейма в формате `.feather`. Должен содержать колонки `audio_path` и `speech_ts`, описанные в пункте "Подготовка данных". Пример устройства датафрейма можно посмотреть в `example_dataframe.feather`; - `jit_model_path` - абсолютный путь до Silero-VAD модели в формате `.jit`. Если оставить это поле пустым, то модель будет загружена из репозитория в зависимости от значения поля `use_torchhub` - `use_torchhub` - Если `True`, то модель для дообучения будет загружена с помощью torch.hub. Если `False`, то модель для дообучения будет загружена с помощью библиотеки silero-vad (необходимо заранее установить командой `pip install silero-vad`); - `tune_8k` - данный параметр отвечает, какую голову Silero-VAD дообучать. Если `True`, дообучаться будет голова с 8000 Гц частотой дискретизации, иначе с 16000 Гц; - `model_save_path` - путь сохранения добученной модели; - `noise_loss` - коэффициент лосса, применяемый для неречевых окон аудио; - `max_train_length_sec` - максимальная длина аудио в секундах на этапе дообучения. Более длительные аудио будут обрезаны до этого показателя; - `aug_prob` - вероятность применения аугментаций к аудиофайлу на этапе дообучения; - `learning_rate` - темп дообучения; - `batch_size` - размер батча при дообучении и валидации; - `num_workers` - количество потоков, используемых для загрузки данных; - `num_epochs` - количество эпох дообучения. За одну эпоху прогоняются все тренировочные данные; - `device` - `cpu` или `cuda`. ## Дообучение Дообучение запускается командой `python tune.py` Длится в течение `num_epochs`, лучший чекпоинт по показателю ROC-AUC на валидационной выборке будет сохранен в `model_save_path` в формате jit. ## Поиск пороговых значений Порог на вход и порог на выход можно подобрать, используя команду `python search_thresholds` Данный скрипт использует файл конфигурации, описанный выше. Указанная в конфигурации модель будет использована для поиска оптимальных порогов на валидационном датасете. ## Цитирование ``` @misc{Silero VAD, author = {Silero Team}, title = {Silero VAD: pre-trained enterprise-grade Voice Activity Detector (VAD), Number Detector and Language Classifier}, year = {2024}, publisher = {GitHub}, journal = {GitHub repository}, howpublished = {\url{https://github.com/snakers4/silero-vad}}, commit = {insert_some_commit_here}, email = {hello@silero.ai} } ``` --- ### Overview (`examples/c++/README.md`) # Silero-VAD V6 in C++ (based on LibTorch) This is the source code for Silero-VAD V6 in C++, utilizing LibTorch & Onnxruntime. You should compare its results with the Python version. Results at 16 and 8kHz have been tested. Batch and CUDA inference options are deprecated. ## Requirements - GCC 11.4.0 (GCC >= 5.1) - Onnxruntime 1.11.0 (other versions are also acceptable) - LibTorch 1.13.0 (other versions are also acceptable) ## Download LibTorch ```bash -Onnxruntime $wget https://github.com/microsoft/onnxruntime/releases/download/v1.11.1/onnxruntime-linux-x64-1.11.1.tgz $tar -xvf onnxruntime-linux-x64-1.11.1.tgz $ln -s onnxruntime-linux-x64-1.11.1 onnxruntime-linux #soft-link -Libtorch $wget https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-1.13.0%2Bcpu.zip $unzip libtorch-shared-with-deps-1.13.0+cpu.zip ``` ## Compilation ```bash -ONNX-build $g++ main.cc silero.cc -I ./onnxruntime-linux/include/ -L ./onnxruntime-linux/lib/ -lonnxruntime -Wl,-rpath,./onnxruntime-linux/lib/ -o silero -std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -DUSE_ONNX -TORCH-build $g++ main.cc silero.cc -I ./libtorch/include/ -I ./libtorch/include/torch/csrc/api/include -L ./libtorch/lib/ -ltorch -ltorch_cpu -lc10 -Wl,-rpath,./libtorch/lib/ -o silero -std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -DUSE_TORCH ``` ## Optional Compilation Flags -DUSE_TORCH -DUSE_ONNX ## Run the Program To run the program, use the following command: `./silero ` `./silero aepyx.wav 16000 0.5` `./silero aepyx_8k.wav 8000 0.5` The sample file aepyx.wav is part of the Voxconverse dataset. File details: aepyx.wav is a 16kHz, 16-bit audio file. File details: aepyx_8k.wav is a 8kHz, 16-bit audio file. --- ### Overview (`examples/cpp/README.md`) # Stream example in C++ Here's a simple example of the vad model in c++ onnxruntime. ## Requirements The example has been tested in the environment below; other environments may work as well. - WSL2 + Debian-bullseye (docker) - gcc 12.2.0 - onnxruntime-linux-x64-1.12.1 ## Usage 1. Install gcc 12.2.0, or just pull the docker image with `docker pull gcc:12.2.0-bullseye` 2. Install onnxruntime-linux-x64-1.12.1 - Download lib onnxruntime: `wget https://github.com/microsoft/onnxruntime/releases/download/v1.12.1/onnxruntime-linux-x64-1.12.1.tgz` - Unzip. Assume the path is `/root/onnxruntime-linux-x64-1.12.1` 3. Modify the WAV path and test configuration in the main function `wav::WavReader wav_reader("${path_to_your_wav_file}");` The v5 ONNX model expects 8 kHz or 16 kHz audio. The example uses the corresponding 256- or 512-sample window and passes the sample rate as the `sr` input. Keep the model, WAV sample rate, and window size consistent when changing the configuration. Adjust the threshold and silence settings as needed for your audio. 4. Build with gcc and run ```bash # Build g++ silero-vad-onnx.cpp -I /root/onnxruntime-linux-x64-1.12.1/include/ -L /root/onnxruntime-linux-x64-1.12.1/lib/ -lonnxruntime -Wl,-rpath,/root/onnxruntime-linux-x64-1.12.1/lib/ -o test # Run ./test ``` --- ### Overview (`examples/cpp_libtorch/README.md`) # Silero-VAD V5 in C++ (based on LibTorch) This is the source code for Silero-VAD V5 in C++, utilizing LibTorch. The primary implementation is CPU-based, and you should compare its results with the Python version. Only results at 16kHz have been tested. Additionally, batch and CUDA inference options are available if you want to explore further. Note that when using batch inference, the speech probabilities may slightly differ from the standard version, likely due to differences in caching. Unlike individual input processing, batch inference may not use the cache from previous chunks. Despite this, batch inference offers significantly faster processing. For optimal performance, consider adjusting the threshold when using batch inference. ## Requirements - GCC 11.4.0 (GCC >= 5.1) - LibTorch 1.13.0 (other versions are also acceptable) ## Download LibTorch ```bash -CPU Version wget https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-1.13.0%2Bcpu.zip unzip libtorch-shared-with-deps-1.13.0+cpu.zip' -CUDA Version wget https://download.pytorch.org/libtorch/cu116/libtorch-shared-with-deps-1.13.0%2Bcu116.zip unzip libtorch-shared-with-deps-1.13.0+cu116.zip ``` ## Compilation ```bash -CPU Version g++ main.cc silero_torch.cc -I ./libtorch/include/ -I ./libtorch/include/torch/csrc/api/include -L ./libtorch/lib/ -ltorch -ltorch_cpu -lc10 -Wl,-rpath,./libtorch/lib/ -o silero -std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -CUDA Version g++ main.cc silero_torch.cc -I ./libtorch/include/ -I ./libtorch/include/torch/csrc/api/include -L ./libtorch/lib/ -ltorch -ltorch_cuda -ltorch_cpu -lc10 -Wl,-rpath,./libtorch/lib/ -o silero -std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -DUSE_GPU ``` ## Optional Compilation Flags -DUSE_BATCH: Enable batch inference -DUSE_GPU: Use GPU for inference ## Run the Program To run the program, use the following command: `./silero aepyx.wav 16000 0.5` The sample file aepyx.wav is part of the Voxconverse dataset. File details: aepyx.wav is a 16kHz, 16-bit audio file. --- ### Overview (`examples/cpp_libtorch_deprecated/README.md`) # Silero-VAD V5 in C++ (based on LibTorch) This is the source code for Silero-VAD V5 in C++, utilizing LibTorch. The primary implementation is CPU-based, and you should compare its results with the Python version. Only results at 16kHz have been tested. Additionally, batch and CUDA inference options are available if you want to explore further. Note that when using batch inference, the speech probabilities may slightly differ from the standard version, likely due to differences in caching. Unlike individual input processing, batch inference may not use the cache from previous chunks. Despite this, batch inference offers significantly faster processing. For optimal performance, consider adjusting the threshold when using batch inference. ## Requirements - GCC 11.4.0 (GCC >= 5.1) - LibTorch 1.13.0 (other versions are also acceptable) ## Download LibTorch ```bash -CPU Version wget https://download.pytorch.org/libtorch/cpu/libtorch-shared-with-deps-1.13.0%2Bcpu.zip unzip libtorch-shared-with-deps-1.13.0+cpu.zip' -CUDA Version wget https://download.pytorch.org/libtorch/cu116/libtorch-shared-with-deps-1.13.0%2Bcu116.zip unzip libtorch-shared-with-deps-1.13.0+cu116.zip ``` ## Compilation ```bash -CPU Version g++ main.cc silero_torch.cc -I ./libtorch/include/ -I ./libtorch/include/torch/csrc/api/include -L ./libtorch/lib/ -ltorch -ltorch_cpu -lc10 -Wl,-rpath,./libtorch/lib/ -o silero -std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -CUDA Version g++ main.cc silero_torch.cc -I ./libtorch/include/ -I ./libtorch/include/torch/csrc/api/include -L ./libtorch/lib/ -ltorch -ltorch_cuda -ltorch_cpu -lc10 -Wl,-rpath,./libtorch/lib/ -o silero -std=c++14 -D_GLIBCXX_USE_CXX11_ABI=0 -DUSE_GPU ``` ## Optional Compilation Flags -DUSE_BATCH: Enable batch inference -DUSE_GPU: Use GPU for inference ## Run the Program To run the program, use the following command: `./silero aepyx.wav 16000 0.5` The sample file aepyx.wav is part of the Voxconverse dataset. File details: aepyx.wav is a 16kHz, 16-bit audio file. --- ### Overview (`examples/go/README.md`) ## Golang Example This is a sample program of how to run speech detection using `silero-vad` from Golang (CGO + ONNX Runtime). ### Requirements - Golang >= v1.21 - ONNX Runtime ### Usage ```sh go run ./cmd/main.go test.wav ``` > **_Note_** > > Make sure you have the ONNX Runtime library and C headers installed in your path. --- ### Overview (`examples/haskell/README.md`) # Haskell example To run the example, make sure you put an ``example.wav`` in this directory, and then run the following: ```bash stack run ``` The ``example.wav`` file must have the following requirements: - Must be 16khz sample rate. - Must be mono channel. - Must be 16-bit audio. This uses the [silero-vad](https://hackage.haskell.org/package/silero-vad) package, a haskell implementation based on the C# example. --- ### Overview (`examples/microphone_and_webRTC_integration/README.md`) In this example, an integration with the microphone and the webRTC VAD has been done. I used [this](https://github.com/mozilla/DeepSpeech-examples/tree/r0.8/mic_vad_streaming) as a draft. Here a short video to present the results: https://user-images.githubusercontent.com/28188499/116685087-182ff100-a9b2-11eb-927d-ed9f621226ee.mp4 # Requirements: The libraries used for the following example are: ``` Python == 3.6.9 webrtcvad >= 2.0.10 torchaudio >= 0.8.1 torch >= 1.8.1 halo >= 0.0.31 Soundfile >= 0.13.3 ``` Using pip3: ``` pip3 install webrtcvad pip3 install torchaudio pip3 install torch pip3 install halo pip3 install soundfile ``` Moreover, to make the code easier, the default sample_rate is 16KHz without resampling. This example has been tested on ``` ubuntu 18.04.3 LTS``` --- ### Overview (`examples/onnx_sequence/README.md`) # Offline ONNX Sequence Inference This optional example reduces ONNX Runtime call overhead when processing long offline recordings. It exports a graph that evaluates consecutive Silero VAD frames in one call while preserving the temporal LSTM state. The sequence axis represents consecutive frames from one recording. It is not a batch of independent recordings, and this example does not change the package loader, streaming API, or shipped models. ## Setup Run the commands from the repository root. The exporter requires PyTorch and ONNX; the runner requires NumPy and ONNX Runtime: ```bash python -m pip install torch onnx onnxruntime numpy ``` ## Export Generate the model locally from the packaged TorchScript weights: ```bash python examples/onnx_sequence/export.py \ --sample-rate 16000 \ --output silero_vad_16k_sequence.onnx ``` Use `--sample-rate 8000` to export the 8 kHz architecture. Generated ONNX files are not part of the package and should not be committed. The exporter reconstructs the public v5/v6 architecture, validates expected weight names and shapes, uses a dynamic sequence axis, and runs the ONNX model checker. The resulting model accepts: ```text input: [sequence_length, frame_samples + context_samples] h: [1, 1, 128] c: [1, 1, 128] ``` It returns one speech probability per frame and the final hidden and cell states. The states and the previous frame's audio context must be carried into the next block. ## Validate and benchmark `run.py` implements a bounded runner, compares it with the stock frame-at-a-time ONNX model, and exits with an error if probabilities or final recurrent state differ by more than `1e-6`. It also reports whether the results are array-exact: ```bash python examples/onnx_sequence/run.py \ silero_vad_16k_sequence.onnx \ --sample-rate 16000 \ --seconds 60.01 \ --trials 5 ``` You can validate an uncompressed mono PCM16 WAV with `--wav path/to/audio.wav`. The default 512-frame block covers 16.384 seconds at either supported sample rate and keeps model inputs bounded for recordings of any length. This example returns frame probabilities. Applications that need speech timestamps should feed those probabilities into their offline timestamp logic; the core `get_speech_timestamps` API remains unchanged. ## Reference results The submitted `run.py` was measured on one pinned AMD Ryzen 5 5600X core with ONNX Runtime 1.27.0 using `CPUExecutionProvider` and one runtime thread. Timings include framing and probability inference; they exclude audio decoding, model export, session creation, and timestamp post-processing. The long input was this [69m20.679s recording](https://www.youtube.com/watch?v=Cdle09QLPLI), decoded once to 16 kHz mono PCM16 before timing. | Input | Stock frame ONNX | Sequence ONNX | Speedup | Output | | --- | ---: | ---: | ---: | --- | | 69m20.679s, 16 kHz | 11.8733 s | 4.8382 s | 2.45x | exact | | Repository 60s fixture, 16 kHz | 0.1651 s | 0.0705 s | 2.34x | exact | | Repository fixture resampled to 8 kHz | 0.1363 s | 0.0425 s | 3.21x | exact | Each result is the median of five alternating warmed trials. Timing varies by CPU and ONNX Runtime version; no timing threshold is used as a correctness check. The temporal-sequence approach was proposed in [discussion #408](https://github.com/snakers4/silero-vad/discussions/408) and later used by [faster-whisper](https://github.com/SYSTRAN/faster-whisper/pull/936). Bounded blocks follow the long-input handling added in [faster-whisper #1198](https://github.com/SYSTRAN/faster-whisper/pull/1198), and the v6 architecture follows [faster-whisper #1390](https://github.com/SYSTRAN/faster-whisper/pull/1390). --- ### Overview (`examples/openvino/README.md`) # Silero VAD in OpenVINO The stock `silero_vad.onnx` does not load in OpenVINO. `read_model` fails with: ``` OpConversionFailure: Model wasn't fully converted. -- Conv-16: While validating ONNX node '': Check 'data.get_partial_shape().rank().is_static()' failed The input data tensor's rank has to be known (static) ``` The ONNX frontend cannot infer static ranks for the Conv ops (the STFT) inside the If subgraphs. There is one top level If that switches between the 8 kHz and 16 kHz paths, plus 7 nested If nodes per branch left behind by TorchScript tracing, so folding the sample rate constant alone is not enough. None of those conditions depend on the audio itself, only on the sample rate and tensor shapes, so for a fixed configuration they can all be resolved offline. `convert.py` inlines every If by evaluating its condition empirically with onnxruntime, removes the dead branch including its weights (the file drops from 2.27 MB to 1.23 MB), folds Identity nodes (recent OpenVINO serializes them as opset16::Identity, which older CPU plugins cannot execute) and fixes static shapes. The result loads and compiles in OpenVINO directly, as ONNX or as exported IR. ## Usage ``` pip install numpy onnx onnxruntime openvino python convert.py # writes silero_vad_16k.onnx python convert.py --sr 8000 # writes silero_vad_8k.onnx python convert.py --save-ir # also exports OpenVINO IR python verify.py silero_vad_16k.onnx ../../tests/data/test.wav "../c++/aepyx.wav" python verify.py silero_vad_8k.onnx --sr 8000 "../c++/aepyx_8k.wav" ``` `convert.py` defaults to the model shipped in this repo. `verify.py` compares the converted model against the stock one with chained state, on synthetic audio and on any 16 bit mono wav files you pass, and checks that the resulting speech segmentation is identical. It exits nonzero if anything is off. ## Set inference precision to f32 On CPUs with bf16 support (AMX or AVX512 BF16) the OpenVINO CPU plugin defaults to bf16 inference. For this model that is not a harmless accuracy tradeoff: the per chunk error compounds through the recurrent state until the speech segmentation actually changes. On real audio the default gave max abs diffs up to 3e-1 against the stock model, while f32 gives 2e-6. Always compile with: ```python compiled = core.compile_model(model, "CPU", {"INFERENCE_PRECISION_HINT": "f32"}) ``` or `ov::hint::inference_precision(ov::element::f32)` in C++. The model is tiny, f32 costs nothing. ## Inference I/O contract of the converted model: `input` f32 [1, 576] at 16 kHz (64 context samples followed by 512 new ones; [1, 288] as 32 plus 256 at 8 kHz), `state` f32 [2, 1, 128], outputs `output` f32 [1, 1] and `stateN` f32 [2, 1, 128]. Chain `stateN` back into `state`, keep the last 64 (or 32) input samples as the next context, and reset both to zeros when a new stream starts. Same protocol as the OnnxWrapper in this repo. ```python import numpy as np import openvino as ov req = ov.Core().compile_model("silero_vad_16k.onnx", "CPU", {"INFERENCE_PRECISION_HINT": "f32"}).create_infer_request() state = np.zeros((2, 1, 128), np.float32) ctx = np.zeros((1, 64), np.float32) for chunk in stream_of_512_sample_chunks: x = np.concatenate([ctx, chunk[None]], axis=1) res = req.infer({"input": x, "state": state}) prob, state, ctx = res["output"][0, 0], res["stateN"], x[:, -64:] ``` ## Validation Converted vs stock, chained state throughout. In onnxruntime the converted model is bit exact (max abs diff 0.0) for both sample rates, so the transformation is the mathematical identity for the chosen rate. In OpenVINO with f32: at 16 kHz, max abs diff 2.3e-06 over `tests/data/test.wav` plus `examples/c++/aepyx.wav` (229 s, 7161 chunks) with identical segmentation (29 of 29 and 65 of 65 segments). At 8 kHz, max abs diff 4.5e-06 over `examples/c++/aepyx_8k.wav` with identical segmentation (79 of 79). Reproduced on Linux and Windows across two OpenVINO versions. Source model sha256 `1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3`. ## Caveats Each converted model is single sample rate (the `sr` input is removed) with batch fixed to 1. Some of the inlined guards may be batch dependent, so batch 1 is validated exhaustively and larger batches are not. IR files are version sensitive: regenerate the IR with your target OpenVINO version, or load the cleaned ONNX directly with `read_model`. ## FAQ You may find these answers useful. > The OpenVINO model has to be generated separately for each major version? No. The only version sensitive artifact is the IR (the xml plus bin pair). The cleaned ONNX that convert.py produces is a plain ONNX file and OpenVINO loads it directly with read_model, so it works across OV versions the same way the stock model works across onnxruntime versions. That's why the PR ships the conversion script instead of binaries: anyone can regenerate the cleaned ONNX from the model already in this repo in a few seconds. If you prefer to publish a converted artifact in the repo, I can add the cleaned ONNX as well. That should not require frequent updates. For IR files, I would recommend generating them locally with the target OpenVINO version, since that avoids version-compatibility edge cases. > Is the OpenVINO model backwards of forwards compatible? The IR is forward compatible but not backward compatible. An IR generated with an older OpenVINO keeps loading on newer runtimes (the IR format has been stable since 2022.1), but an IR generated with a newer OpenVINO can fail on older runtimes. I hit exactly that while testing this: an IR saved with 2026.2 contained opset16::Identity, which an older CPU plugin could parse but not execute. That's the reason convert.py folds Identity nodes out of the graph, and why the README says to regenerate the IR locally or just load the ONNX. > Does it depend on CPU model? The files don't, they are hardware agnostic and compilation for the target happens at load time. The one CPU dependent behavior is the default inference precision: on bf16 capable CPUs (AMX or AVX512 BF16, mostly server Xeons) the CPU plugin silently defaults to bf16, and for this model the error compounds through the recurrent state until the segmentation changes. With INFERENCE_PRECISION_HINT set to f32 as the README says, results match onnxruntime to about 1e-6 on every CPU I tested. > It works only with Intel CPUs? Not only. The CPU plugin dispatches on instruction set features rather than vendor, so it runs on AMD x86-64 as well (Intel is what gets validated and tuned), and ARM64 is officially supported too, Apple Silicon included. The Intel exclusive parts are the accelerator plugins (integrated GPU and NPU), which this example doesn't touch. --- ### Overview (`examples/pyaudio-streaming/README.md`) # Pyaudio Streaming Example This example notebook shows how micophone audio fetched by pyaudio can be processed with Silero-VAD. It has been designed as a low-level example for binary real-time streaming using only the prediction of the model, processing the binary data and plotting the speech probabilities at the end to visualize it. Currently, the notebook consits of two examples: - One that records audio of a predefined length from the microphone, process it with Silero-VAD, and plots it afterwards. - The other one plots the speech probabilities in real-time (using jupyterplot) and records the audio until you press enter. This example does not work in google colab! For local usage only. ## Example Video for the Real-Time Visualization https://user-images.githubusercontent.com/8079748/117580455-4622dd00-b0f8-11eb-858d-e6368ed4eada.mp4 --- ### Overview (`examples/rust-example/README.md`) # Stream example in Rust Made after [C++ stream example](https://github.com/snakers4/silero-vad/tree/master/examples/cpp) ## Dependencies - To build Rust crate `ort` you need `cc` installed. ## Usage Just ``` cargo run ``` If you run example outside of this repo adjust environment variable ``` SILERO_MODEL_PATH=/path/to/silero_vad.onnx cargo run ``` If you need to test against other wav file, not `recorder.wav`, specify it as the first argument ``` cargo run -- /path/to/audio/file.wav ``` --- ### Overview (`examples/rust-wav-processing-with-wavekat-vad/README.md`) # Rust example using wavekat-vad Uses [wavekat-vad](https://github.com/wavekat/wavekat-vad), a Rust library that provides a unified interface for multiple VAD backends including Silero VAD. The Silero ONNX model is downloaded and embedded in the binary at compile time — no manual model setup needed. ## Features - Automatic resampling from any sample rate to 16kHz - `FrameAdapter` handles frame buffering (feed any chunk size, get correctly sized frames) - Works with any WAV file format (mono/stereo, any sample rate) ## Usage ```sh cargo run -- /path/to/audio.wav ``` Sample output: ``` File: audio.wav (16000Hz, 1ch, 16bit) Duration: 3.50s (56000 samples at 16000Hz) Silero VAD — frame: 512 samples (32ms) 0ms 0.012 20ms 0.008 40ms 0.245 60ms 0.876 ################################### SPEECH 80ms 0.923 #################################### SPEECH ... ``` --- --- METRICS --- - Files Extracted: 15 - Estimated Token Budget: ~10292 tokens - Canonical Reference: https://codewiki.google/github.com/snakers4/silero-vad