{"owner":"xorbitsai","repo":"inference","hasSkills":true,"totalSkillsCount":3,"totalTokensCount":4131,"categories":["plugin-manifest","root-instruction","claude-rule"],"hasMcp":false,"mcpConfig":null,"found":["xinference/thirdparty/fish_speech/tools/sensevoice/README.md","AGENTS.md","CLAUDE.md"],"skills":{"xinference/thirdparty/fish_speech/tools/sensevoice/README.md":"# FunASR Command Line Interface\n\nThis tool provides a command-line interface for separating vocals from instrumental tracks, converting videos to audio, and performing speech-to-text transcription on the resulting audio files.\n\n## Requirements\n\n- Python >= 3.10\n- PyTorch <= 2.3.1\n- ffmpeg, pydub, audio-separator[gpu].\n\n## Installation\n\nInstall the required packages:\n\n```bash\npip install -e .[stable]\n```\n\nMake sure you have `ffmpeg` installed and available in your `PATH`.\n\n## Usage\n\n### Basic Usage\n\nTo run the tool with default settings:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir <audio_directory> --save-dir <output_directory>\n```\n\n## Options\n\n|          Option           |                                  Description                                  |\n| :-----------------------: | :---------------------------------------------------------------------------: |\n|        --audio-dir        |                  Directory containing audio or video files.                   |\n|        --save-dir         |                   Directory to save processed audio files.                    |\n|         --device          |         Device to use for processing. Options: cuda (default) or cpu.         |\n|        --language         |                Language of the transcription. Default is auto.                |\n| --max_single_segment_time | Maximum duration of a single audio segment in milliseconds. Default is 20000. |\n|          --punc           |                        Enable punctuation prediction.                         |\n|         --denoise         |                  Enable noise reduction (vocal separation).                   |\n\n## Example\n\nTo process audio files in the directory `path/to/audio` and save the output to `path/to/output`, with punctuation and noise reduction enabled:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir path/to/audio --save-dir path/to/output --punc --denoise\n```\n\n## Additional Notes\n\n- The tool supports `both audio and video files`. Videos will be converted to audio automatically.\n- If the `--denoise` option is used, the tool will perform vocal separation to isolate the vocals from the instrumental tracks.\n- The script will automatically create necessary directories in the `--save-dir`.\n\n## Troubleshooting\n\nIf you encounter any issues, make sure all dependencies are correctly installed and configured. For more detailed troubleshooting, refer to the documentation of each dependency.\n","AGENTS.md":"# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n","CLAUDE.md":"# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n"},"files":{"xinference/thirdparty/fish_speech/tools/sensevoice/README.md":"# FunASR Command Line Interface\n\nThis tool provides a command-line interface for separating vocals from instrumental tracks, converting videos to audio, and performing speech-to-text transcription on the resulting audio files.\n\n## Requirements\n\n- Python >= 3.10\n- PyTorch <= 2.3.1\n- ffmpeg, pydub, audio-separator[gpu].\n\n## Installation\n\nInstall the required packages:\n\n```bash\npip install -e .[stable]\n```\n\nMake sure you have `ffmpeg` installed and available in your `PATH`.\n\n## Usage\n\n### Basic Usage\n\nTo run the tool with default settings:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir <audio_directory> --save-dir <output_directory>\n```\n\n## Options\n\n|          Option           |                                  Description                                  |\n| :-----------------------: | :---------------------------------------------------------------------------: |\n|        --audio-dir        |                  Directory containing audio or video files.                   |\n|        --save-dir         |                   Directory to save processed audio files.                    |\n|         --device          |         Device to use for processing. Options: cuda (default) or cpu.         |\n|        --language         |                Language of the transcription. Default is auto.                |\n| --max_single_segment_time | Maximum duration of a single audio segment in milliseconds. Default is 20000. |\n|          --punc           |                        Enable punctuation prediction.                         |\n|         --denoise         |                  Enable noise reduction (vocal separation).                   |\n\n## Example\n\nTo process audio files in the directory `path/to/audio` and save the output to `path/to/output`, with punctuation and noise reduction enabled:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir path/to/audio --save-dir path/to/output --punc --denoise\n```\n\n## Additional Notes\n\n- The tool supports `both audio and video files`. Videos will be converted to audio automatically.\n- If the `--denoise` option is used, the tool will perform vocal separation to isolate the vocals from the instrumental tracks.\n- The script will automatically create necessary directories in the `--save-dir`.\n\n## Troubleshooting\n\nIf you encounter any issues, make sure all dependencies are correctly installed and configured. For more detailed troubleshooting, refer to the documentation of each dependency.\n","AGENTS.md":"# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n","CLAUDE.md":"# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","rawUrl":"https://raw.githubusercontent.com/xorbitsai/inference/HEAD/AGENTS.md","title":"AI Agent Protocol & Instructions","category":"root-instruction","format":"markdown","content":"# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n","isInternal":false,"tokens":1828,"sizeBytes":7899},{"name":"CLAUDE.md","path":"CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/xorbitsai/inference/HEAD/CLAUDE.md","title":"Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":"# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n","isInternal":false,"tokens":1828,"sizeBytes":7899},{"name":"README.md","path":"xinference/thirdparty/fish_speech/tools/sensevoice/README.md","rawUrl":"https://raw.githubusercontent.com/xorbitsai/inference/HEAD/xinference/thirdparty/fish_speech/tools/sensevoice/README.md","title":"sensevoice Documentation","category":"plugin-manifest","format":"markdown","content":"# FunASR Command Line Interface\n\nThis tool provides a command-line interface for separating vocals from instrumental tracks, converting videos to audio, and performing speech-to-text transcription on the resulting audio files.\n\n## Requirements\n\n- Python >= 3.10\n- PyTorch <= 2.3.1\n- ffmpeg, pydub, audio-separator[gpu].\n\n## Installation\n\nInstall the required packages:\n\n```bash\npip install -e .[stable]\n```\n\nMake sure you have `ffmpeg` installed and available in your `PATH`.\n\n## Usage\n\n### Basic Usage\n\nTo run the tool with default settings:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir <audio_directory> --save-dir <output_directory>\n```\n\n## Options\n\n|          Option           |                                  Description                                  |\n| :-----------------------: | :---------------------------------------------------------------------------: |\n|        --audio-dir        |                  Directory containing audio or video files.                   |\n|        --save-dir         |                   Directory to save processed audio files.                    |\n|         --device          |         Device to use for processing. Options: cuda (default) or cpu.         |\n|        --language         |                Language of the transcription. Default is auto.                |\n| --max_single_segment_time | Maximum duration of a single audio segment in milliseconds. Default is 20000. |\n|          --punc           |                        Enable punctuation prediction.                         |\n|         --denoise         |                  Enable noise reduction (vocal separation).                   |\n\n## Example\n\nTo process audio files in the directory `path/to/audio` and save the output to `path/to/output`, with punctuation and noise reduction enabled:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir path/to/audio --save-dir path/to/output --punc --denoise\n```\n\n## Additional Notes\n\n- The tool supports `both audio and video files`. Videos will be converted to audio automatically.\n- If the `--denoise` option is used, the tool will perform vocal separation to isolate the vocals from the instrumental tracks.\n- The script will automatically create necessary directories in the `--save-dir`.\n\n## Troubleshooting\n\nIf you encounter any issues, make sure all dependencies are correctly installed and configured. For more detailed troubleshooting, refer to the documentation of each dependency.\n","isInternal":false,"tokens":475,"sizeBytes":2457}],"systemPromptSnippet":"<yakaai_skills repo=\"xorbitsai/inference\">\n<!-- File: AGENTS.md (Tokens: ~1828 | Category: root-instruction) -->\n# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n\n<!-- File: CLAUDE.md (Tokens: ~1828 | Category: claude-rule) -->\n# AI Agent Guidance\n\nGuidance for AI coding agents working in this repository.\n\n## Project Overview\n\nXinference is a Python model-serving project for language, embedding, rerank,\nimage, video, audio, and multimodal models. It exposes CLI commands, a Python\nclient, REST/OpenAI-compatible APIs, an xoscar-based distributed runtime, and a\nReact Web UI.\n\nPrimary package and entry points:\n\n- `xinference/`: Python package.\n- `xinference/deploy/cmdline.py`: CLI entry points for `xinference`,\n  `xinference-local`, `xinference-supervisor`, and `xinference-worker`.\n- `xinference/core/`: supervisor/worker runtime and actor orchestration.\n- `xinference/model/`: model families, engines, built-in model specs, and model\n  tests.\n- `xinference/api/`: API server and OpenAI-compatible routes.\n- `xinference/client/`: sync and async Python clients.\n- `frontend/`: Next.js Web UI.\n- `doc/source/`: Sphinx documentation.\n- `.github/workflows/python.yaml`: main lint and test CI.\n\n## Working Rules\n\n- Prefer small, focused changes that match the current module's style.\n- Preserve backward compatibility. If a breaking change is unavoidable, document\n  the reason and add deprecation behavior where practical.\n- Do not edit `xinference/thirdparty/` unless the task explicitly concerns\n  vendored code.\n- Avoid broad refactors while fixing a localized bug.\n- Keep public API behavior, request/response schemas, model registration names,\n  and CLI flags stable unless the task requires changing them.\n- Add or update tests for behavior changes. For model-runtime changes, prefer\n  tests close to the affected model family under `xinference/model/**/tests/`.\n- Use type hints for new Python code when practical; this project encourages\n  PEP 484 style annotations.\n- Treat docs and examples as user-facing API. Keep command examples accurate.\n\n## Environment Setup\n\nRecommended local setup:\n\n```bash\nconda create --name xinf python=3.12 nodejs\nconda activate xinf\npip install -e \".[dev]\"\n```\n\nNotes:\n\n- Wheel, sdist, and editable builds (including `pip install -e .`) run the\n  Web UI build through the in-tree build backend (`build_backend.py` /\n  `build_web.py`) unless `NO_WEB_UI=1` is set.\n- For Python 3.12 and newer, CI installs `setuptools<82`; use the same pin if\n  packaging or editable installs fail.\n- The project supports Python 3.10 through 3.14 in CI.\n- Optional model engines have extras in `pyproject.toml`, such as\n  `transformers`, `vllm`, `mlx`, `embedding`, `rerank`, `image`, `video`, and\n  `audio`.\n\n## Formatting and Linting\n\nPython formatting and checks are managed through pre-commit:\n\n```bash\npip install pre-commit\npre-commit run --files <modified-files>\n```\n\nFor a branch-wide check against upstream main:\n\n```bash\npre-commit run --from-ref=upstream/main --to-ref=HEAD --all-files\n```\n\nConfigured hooks include Black, end-of-file/trailing-whitespace checks, Ruff,\nisort, mypy with missing imports ignored, and codespell. Configuration lives in\n`.pre-commit-config.yaml` and `pyproject.toml`.\n\nThe `test-platform-coverage` hook also checks changed tests against the CI\nplatform scopes. Include tests in `PLATFORM_TEST_PATHS` or `LINUX_TEST_PATHS` in\n`.github/workflows/python.yaml`, use the existing macOS MLX scope, or declare\n`# CI: linux` for portable/Linux-only regressions. Local checks discover Git\nchanges automatically and honor `--from-ref` / `--to-ref`; CI supplies the PR's\nbase and head SHAs to check the same scope as the workflow tests.\n\n## Python Tests\n\nRun focused tests first:\n\n```bash\npytest -vv path/to/test_file.py\n```\n\nThe broad CI-style non-GPU test command is approximately:\n\n```bash\npytest --timeout=3000 -W ignore::PendingDeprecationWarning -vv \\\n  --cov-config=pyproject.toml --cov-report=xml --cov=xinference \\\n  --ignore xinference/core/tests/test_continuous_batching.py \\\n  --ignore xinference/model/image/tests/test_stable_diffusion.py \\\n  --ignore xinference/model/image/tests/test_got_ocr2.py \\\n  --ignore xinference/model/audio/tests \\\n  --ignore xinference/model/embedding/tests/test_integrated_embedding.py \\\n  --ignore xinference/model/llm/transformers/tests/test_tensorizer.py \\\n  --ignore xinference/model/llm/tests/test_llm_model.py \\\n  --ignore xinference/model/llm/vllm \\\n  --ignore xinference/model/llm/sglang \\\n  --ignore xinference/client/tests/test_client.py \\\n  --ignore xinference/client/tests/test_async_client.py \\\n  --ignore xinference/model/llm/mlx \\\n  xinference\n```\n\nUse narrower commands for daily development. Many model tests require large\ndependencies, GPU, Metal, network access, or model downloads.\n\n## Frontend\n\nThe Web UI is under `frontend` and is a Next.js app (React, TypeScript,\nTailwind CSS), built as a static export and served by the Python backend from\n`xinference/ui/web/dist`.\n\nCommon commands:\n\n```bash\ncd frontend\nnpm ci\nnpm run dev\nnpm run build\nnpx eslint .\n```\n\nUse `npm run format` only when you intentionally want Prettier writes across\nthe frontend tree.\n\n## Documentation\n\nDocumentation source is in `doc/source`.\n\nCommon docs dependencies are included in the `doc` extra:\n\n```bash\npip install -e \".[doc]\"\ncd doc\nmake html\n```\n\nWhen changing CLI behavior, API behavior, deployment behavior, or model support,\nupdate the relevant documentation pages in `doc/source`.\n\nNew or changed English documentation must include corresponding gettext updates\nfor every existing locale under `doc/source/locale/*/LC_MESSAGES/`. Keep PO\nchanges scoped to the affected messages, validate every modified catalog with\n`msgfmt --check --check-format`, compile them with\n`python doc/build_i18n.py --all`, and build the changed page in each maintained\nlanguage to verify the rendered translations.\n\nBuilt-in model documentation under `doc/source/models/builtin/` is generated by\n`doc/source/gen_docs.py`; do not edit those generated files by hand. After\nchanging a built-in model registry or `model_spec.json`, run the generator from\n`doc/source` and commit any resulting documentation changes:\n\n```bash\ncd doc/source\npython gen_docs.py\n```\n\nThe PR workflow in `.github/workflows/pr_auto_run_gen_docs.yaml` only pushes\ngenerated documentation for eligible same-repository `chore/models-sync/*`\nbranches. For other branches and fork PRs, run the generator explicitly rather\nthan assuming that the workflow will update the PR.\n\n## Model and Runtime Conventions\n\n- Keep model-family logic inside the relevant `xinference/model/<family>/`\n  package.\n- Keep built-in model metadata changes close to existing specs and tests.\n- Be careful with lazy imports and optional dependencies. Import heavyweight\n  model libraries only where needed so unrelated installs still work.\n- Preserve platform guards for Linux-only, CUDA-only, and macOS Metal/MLX paths.\n- For distributed runtime changes, consider both local mode and\n  supervisor/worker mode.\n- For OpenAI-compatible behavior, verify request/response fields and streaming\n  behavior against existing API and client tests.\n\n## CI Expectations\n\nThe main CI workflow:\n\n- Runs `pre-commit run --all-files`.\n- Runs UI `npm ci`, `npx eslint .`, and Prettier check.\n- Tests Python 3.10 through 3.14: every supported version on Linux, and the\n  minimum and maximum supported versions on macOS and Windows.\n- Has special GPU and macOS Metal jobs for model-specific paths.\n\nBefore marking a change done, run the smallest meaningful validation command\nthat covers the behavior you changed, and mention any broader checks that were\nnot run because of environment cost or missing hardware.\n\n## Git and Review Hygiene\n\n- Keep commits scoped to the requested change.\n- Do not rewrite or revert user changes in an existing worktree unless asked.\n- If the active checkout is busy or on an unrelated branch, use a separate\n  worktree and a semantic branch name such as `fix/...`, `feat/...`, or\n  `docs/...`.\n- In PR reviews, inspect current GitHub review threads before adding duplicate\n  comments.\n\n<!-- File: xinference/thirdparty/fish_speech/tools/sensevoice/README.md (Tokens: ~475 | Category: plugin-manifest) -->\n# FunASR Command Line Interface\n\nThis tool provides a command-line interface for separating vocals from instrumental tracks, converting videos to audio, and performing speech-to-text transcription on the resulting audio files.\n\n## Requirements\n\n- Python >= 3.10\n- PyTorch <= 2.3.1\n- ffmpeg, pydub, audio-separator[gpu].\n\n## Installation\n\nInstall the required packages:\n\n```bash\npip install -e .[stable]\n```\n\nMake sure you have `ffmpeg` installed and available in your `PATH`.\n\n## Usage\n\n### Basic Usage\n\nTo run the tool with default settings:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir <audio_directory> --save-dir <output_directory>\n```\n\n## Options\n\n|          Option           |                                  Description                                  |\n| :-----------------------: | :---------------------------------------------------------------------------: |\n|        --audio-dir        |                  Directory containing audio or video files.                   |\n|        --save-dir         |                   Directory to save processed audio files.                    |\n|         --device          |         Device to use for processing. Options: cuda (default) or cpu.         |\n|        --language         |                Language of the transcription. Default is auto.                |\n| --max_single_segment_time | Maximum duration of a single audio segment in milliseconds. Default is 20000. |\n|          --punc           |                        Enable punctuation prediction.                         |\n|         --denoise         |                  Enable noise reduction (vocal separation).                   |\n\n## Example\n\nTo process audio files in the directory `path/to/audio` and save the output to `path/to/output`, with punctuation and noise reduction enabled:\n\n```bash\npython tools/sensevoice/fun_asr.py --audio-dir path/to/audio --save-dir path/to/output --punc --denoise\n```\n\n## Additional Notes\n\n- The tool supports `both audio and video files`. Videos will be converted to audio automatically.\n- If the `--denoise` option is used, the tool will perform vocal separation to isolate the vocals from the instrumental tracks.\n- The script will automatically create necessary directories in the `--save-dir`.\n\n## Troubleshooting\n\nIf you encounter any issues, make sure all dependencies are correctly installed and configured. For more detailed troubleshooting, refer to the documentation of each dependency.\n</yakaai_skills>"}