# Technical Documentation: vllm-project/semantic-router > ℹ️ **Provenance:** Hybrid Fusion: `vllm-project/semantic-router` + `vllm-project/vllm-project.github.io` · [CodeWiki Reference](https://codewiki.google/github.com/vllm-project/semantic-router) · Recency: Active (< 180 days) ## 1. Project Overview & Quickstart (vllm-project/semantic-router) --- ## About vLLM Semantic Router is a programmable routing layer for building Mixture-of-Models systems across heterogeneous LLM infrastructure. It evaluates request signals, user preferences, and application policies to select—or compose—the right model path for each request. Use it to improve quality, cost, latency, privacy, and safety without hard-coding routing logic into applications. | Dimension | Fragmented today | With vLLM SR | | --- | --- | --- | | **Models** | Models specialize in different work. | Compose personalized model paths. | | **Compute** | GPUs, accelerators, edge, and cloud coexist. | Route across heterogeneous compute. | | **Location** | Inference spans edge, private, and cloud. | Keep data within its boundaries. | | **Preference** | "Best" changes by user and workload. | Make every preference executable. | [Explore how it works →](https://vllm-sr.ai/docs/intro/) ## Getting Started ### Install ```bash curl -fsSL https://vllm-sr.ai/install.sh | bash ``` For platform notes, detailed setup options, and troubleshooting, see the **[Installation Guide](https://vllm-sr.ai/docs/installation/)**. Online playground credentials - Username: `love@vllm-sr.ai` - Password: `vllm-sr` ## Latest News - [2026/07/21] New Blog: [Beyond a Single Model: Building Mixture-of-Models Systems with vLLM Semantic Router](https://vllm.ai/blog/2026-07-21-vllm-sr-new-chapter-mom) - [2026/06/29] New Blog: [Micro-Agent: Beat Frontier Models with Collaboration inside Model API](https://vllm.ai/blog/2026-06-29-micro-agent-frontier-models) - [2026/06/16] New Blog: [Beyond One Model: Fusion in vLLM Semantic Router](https://vllm.ai/blog/2026-06-16-vllm-sr-fusion-api) - [2026/06/05] v0.3 Released: [vLLM Semantic Router v0.3 Themis: From Signals to Stateful Production Routing](https://vllm.ai/blog/2026-06-05-v0.3-vllm-sr-themis-release) Earlier announcements - [2026/03/24] Vision Paper Released: [The Workload-Router-Pool Architecture for LLM Inference Optimization](https://vllm-sr.ai/vision-paper) - [2026/03/10] v0.2 Released: [vLLM Semantic Router v0.2 Athena Release](https://vllm.ai/blog/v0.2-vllm-sr-athena-release) - [2026/02/27] White Paper Released: [Signal Driven Decision Routing for Mixture-of-Modality Models](https://vllm-sr.ai/white-paper/) - [2026/01/05] Iris v0.1 Released: [vLLM Semantic Router v0.1 Iris: The First Major Release](https://blog.vllm.ai/2026/01/05/vllm-sr-iris.html) - [2025/12/16] Collaboration: [AMD × vLLM Semantic Router: Building the System Intelligence Together](https://blog.vllm.ai/2025/12/16/vllm-sr-amd.html) - [2025/12/15] New Blog: [Token-Level Truth: Real-Time Hallucination Detection for Production LLMs](https://blog.vllm.ai/2025/12/14/halugate.html) - [2025/11/19] New Blog: [Signal-Decision Driven Architecture: Reshaping Semantic Routing at Scale](https://blog.vllm.ai/2025/11/19/signal-decision.html) - [2025/11/03] Paper Published: [Category-Aware Semantic Caching for Heterogeneous LLM Workloads](https://arxiv.org/abs/2510.26835) - [2025/10/27] New Blog: [Scaling Semantic Routing with Extensible LoRA](https://blog.vllm.ai/2025/10/27/semantic-router-modular.html) - [2025/10/12] Paper Accepted: [When to Reason: Semantic Router for vLLM](https://arxiv.org/abs/2510.08731) - [2025/10/08] Collaboration: vLLM Semantic Router with [vLLM Production Stack](https://github.com/vllm-project/production-stack) Team. - [2025/09/01] Released the project: [vLLM Semantic Router: Next Phase in LLM inference](https://blog.vllm.ai/2025/09/11/semantic-router.html). More announcements are available on the **[Blog](https://vllm-sr.ai/blog/)** and **[Publications](https://vllm-sr.ai/publications/)** pages. ## Community For questions, feedback, or to contribute, please join the `#semantic-router` channel in vLLM Slack. ### Community Meetings We host community meetings on the first and third Tuesday of each month to sync with contributors across different time zones: - **First Tuesday of the month**: 9:00-10:00 AM EST (accommodates US EST, EU, and Asia Pacific contributors) - [Zoom Link](https://us05web.zoom.us/j/84122485631?pwd=BB88v03mMNLVHn60YzVk4PihuqBV9d.1) - [Google Calendar Invite](https://us05web.zoom.us/meeting/tZAsdeuspj4sGdVraOOR4UaXSstrH2jjPYFq/calendar/google/add?meetingMasterEventId=4jjzUKSLSLiBHtIKZpGc3g) - [ics file](https://drive.google.com/file/d/15wO8cg0ZjNxdr8OtGiZyAgkSS8_Wry0J/view?usp=sharing) - **Third Tuesday of the month**: 1:00-2:00 PM EST (accommodates US EST and California contributors) - [Zoom Link](https://us06web.zoom.us/j/86871492845?pwd=LcTtXm9gtGu23JeWqXxbnLLCCvbumB.1) - [Google Calendar Invite](https://us05web.zoom.us/meeting/tZIlcOispzkiHtH2dlkWlLym68bEqvuf3MU5/calendar/google/add?meetingMasterEventId=PqWz2vk7TOCszPXqconGAA) - [ics file](https://drive.google.com/file/d/1T54mwYpXXoV9QfR76I56BFBPNbykSsTw/view?usp=sharing) - Meeting recordings: [YouTube](https://www.youtube.com/@vLLMSemanticRouter/videos) ## Contributing If you want to contribute, start with **[CONTRIBUTING.md](CONTRIBUTING.md)**. For repository-native development workflow and validation commands, use **[AGENTS.md](AGENTS.md)** as the entrypoint and **[tools/agent/docs/README.md](tools/agent/docs/README.md)** as the canonical index. ## Citation If you find Semantic Router helpful in your research or projects, please consider citing it: ``` @misc{semanticrouter2025, title={vLLM Semantic Router}, author={vLLM Semantic Router Team}, year={2025}, howpublished={\url{https://github.com/vllm-project/semantic-router}}, } ``` ## Sponsors We are grateful to our sponsors who support us: --- [**AMD**](https://www.amd.com) provides us with GPU resources and [ROCm™](https://www.amd.com/en/products/software/rocm.html) software for training and researching frontier router models, enhancing E2E testing, and building the online models playground. --- ## 2. Official Technical Reference & Guides (vllm-project/vllm-project.github.io) # vLLM blog source ## Local development 1. Install `jekyll` and `bundler` by running `gem install jekyll bundler`. Ask ChatGPT for help if you encounter any issues. 2. `bundle install` to install the necessary gems. 3. `rm ./Gemfile.lock` if you meet gem error. 5. `bundle exec jekyll server` to compile and start the server. To add a new blogpost, please refer to `_posts/2023-06-20-vllm.md` as an example. Some notes: - Google Doc can be saved as markdown format which will make your life easier. - Note that the blogpost will only show when its date is in the past. - Add `summary` front matter for SEO. Keep it concise, specific, and 240 characters or fewer. - Put figures under `assets/figures/yourblogname/`. - Make a pull request. The blog is automatically built and deployed by GitHub Actions when `main` is pushed to. ## LaTeX Math The blog supports LaTeX math via [MathJax](https://docs.mathjax.org/en/latest/index.html). It can be enabled by adding `math: true` to the document frontmatter. It has been configured to support the standard LaTeX style math notation, i.e.: ```latex $ inline math $ ``` ```latex $$ math block $$ ``` ## GitHub Flavored Admonitions The blog supports GitHub flavored admonitions via [jekyll-gfm-admonition](https://github.com/Helveg/jekyll-gfm-admonitions). It supports the following syntax: ```markdown > [!NOTE] > Highlights information that users should take into account, even when skimming. > And supports multi-line text. > [!TIP] > Optional information to help a user be more successful. > [!IMPORTANT] > Crucial information necessary for users to succeed. > [!WARNING] > Critical content demanding immediate > user attention due to potential risks. > [!CAUTION] > Negative potential consequences of an action. > Opportunity to provide more context. ``` ## Theme customization The theme we are using is [Minima](https://github.com/jekyll/minima). If you need to customise anything from this theme, see [Overriding theme defaults](https://jekyllrb.com/docs/themes/#overriding-theme-defaults). --- METRICS --- - Files Extracted: 2 - Estimated Token Budget: ~2025 tokens - Recency Window: Active (< 180 days) - Canonical Reference: https://codewiki.google/github.com/vllm-project/semantic-router