semantic-router (Agent Skills)

# vLLM Semantic Router Agent Entry

This file is the short entrypoint for coding agents. The detailed human-readable system of record lives in [tools/agent/docs/README.md](tools/agent/docs/README.md). The executable rule layer lives in [tools/agent/repo-manifest.yaml](tools/agent/repo-manifest.yaml), [tools/agent/test-domain-registry.yaml](tools/agent/test-domain-registry.yaml), [tools/agent/task-matrix.yaml](tools/agent/task-matrix.yaml), [tools/agent/skill-registry.yaml](tools/agent/skill-registry.yaml), [tools/agent/structure-rules.yaml](tools/agent/structure-rules.yaml), [tools/agent/maintainer-policy.yaml](tools/agent/maintainer-policy.yaml), and [tools/make/agent.mk](tools/make/agent.mk).

vLLM Semantic Router is an Envoy ExtProc request router for LLM inference. It
resolves a request-facing entrypoint to an isolated recipe, evaluates that
recipe's signals and projections, applies its decision and algorithm, then
invokes the selected backend and recipe-scoped plugins.

## Read First

1. [tools/agent/docs/README.md](tools/agent/docs/README.md)
2. [tools/agent/docs/repo-map.md](tools/agent/docs/repo-map.md)
3. [tools/agent/docs/environments.md](tools/agent/docs/environments.md)
4. [tools/agent/docs/change-surfaces.md](tools/agent/docs/change-surfaces.md)
5. `make agent-report ENV=cpu|amd CHANGED_FILES="..."`

## Native Discovery vs Routed Context

- Root startup should always discover this [AGENTS.md](AGENTS.md) entrypoint and the thin repo-native bridge at [.agents/skills/harness/SKILL.md](.agents/skills/harness/SKILL.md).
- Full task routing, primary-skill resolution, local-rule surfacing, loop-mode guidance, and validation planning still come from `make agent-report ENV=cpu|amd CHANGED_FILES="..."`.
- `tools/agent/**` remains the canonical harness source; `.agents/skills/**` is only a discovery bridge.

If you need real AMD model deployment details instead of the minimal smoke path, also read [website/docs/installation/amd-rocm.md](website/docs/installation/amd-rocm.md) and [config/recipes/balance/config.yaml](config/recipes/balance/config.yaml).

## Repository Map

- `src/semantic-router/`: Go router, Envoy ExtProc server, routing runtime, and APIs
- `src/vllm-sr/`: Python CLI and local stack orchestration
- `config/`: canonical config, reusable fragments, runtime examples, and complete recipes
- `candle-binding/`, `ml-binding/`, `nlp-binding/`, `onnx-binding/`: inference bindings
- `dashboard/`: React frontend and Go management backend
- `deploy/`: artifacts that create or configure deployment targets
- `e2e/`: end-to-end framework and profiles
- `tools/`: build, development, release, security, smoke, and agent tooling
- `website/`: the only public documentation tree

The root contains only repository-wide contracts, community metadata, and
tool-mandated entrypoints. The executable allowlist is in
`tools/agent/structure-rules.yaml`; do not add root catch-all files.

## Supported Environments

- `cpu-local`: `make vllm-sr-dev`, then `vllm-sr serve --image-pull-policy never`
- `amd-local`: `make vllm-sr-dev VLLM_SR_PLATFORM=amd`, then `vllm-sr serve --image-pull-policy never --platform amd`
- `nvidia-local`: `VLLM_SR_PLATFORM=nvidia make vllm-sr-build`, then `vllm-sr serve --platform nvidia --config <recipe>` (selects the CUDA image + flips `use_cpu` to false, at parity with `--platform amd`; see [tools/agent/docs/nvidia-local.md](tools/agent/docs/nvidia-local.md))
- `ci-k8s`: `make e2e-test`

## Non-Negotiable Rules

- Use the local image flow for local-dev behavior. Do not invent another serve path.
- Start from one project-level primary skill. Cross-cutting guidance belongs in change surfaces, canonical docs, or maintainer support skills.
- Run the smallest relevant gate first: `make agent-validate`, `make agent-lint`, `make agent-ci-gate`, then `make agent-feature-gate`.
- Use `make agent-pr-gate` when you need a repo-native local reproduction of the baseline PR requirements.
- Drive the active task to its reported completion boundary: fix failures and rerun the applicable gates until the current change or subtask is done, and do not hand off on the first failing run.
- Treat docs-only and website-only edits as lightweight unless the task matrix says otherwise.
- Contributor workflow, issue or PR intake rules, and maintainer label taxonomy live in `CONTRIBUTING.md`, `.github/PULL_REQUEST_TEMPLATE.md`, `.github/ISSUE_TEMPLATE/**`, and `.prowlabels.yaml`; commits intended for PRs must use `git commit -s`.
- Keep commits reviewable and logically coherent. Each commit must build and lint, describe why the change exists, and avoid unrelated drive-by cleanup.
- Keep PR blast radius aligned with the requested behavior; couple subsystems only when their contracts change together.
- Maintainer release, issue, PR, stale-work, and daily-board workflows live in [tools/agent/docs/maintainer-ops.md](tools/agent/docs/maintainer-ops.md) and write local state only under `.agent-harness/maintainer/` unless an explicit reviewed apply step mutates GitHub.
- Behavior-visible routing, startup, config, Docker, CLI, or API changes need E2E updates unless the change is a pure refactor.
- If the work needs multiple resumable loops across sessions or contributors, use the indexed current execution plans under [tools/agent/docs/plans/README.md](tools/agent/docs/plans/README.md) instead of ad hoc task notes. Historical plans are not kept in the current tree.
- If the desired architecture and the current implementation still diverge after your change, add or update the durable debt entry indexed from [tools/agent/docs/tech-debt/README.md](tools/agent/docs/tech-debt/README.md) instead of leaving the gap only in chat or PR text.
- Keep modules narrow: one main responsibility per file, small orchestrators plus helpers, interfaces only at seams.
- Legacy hotspots are debt, not precedent. Touched hotspot files must not grow in responsibility; prefer extraction-first edits.
- Read the nearest local `AGENTS.md` before editing hotspot trees under `src/semantic-router/pkg/config/`, `src/semantic-router/pkg/extproc/`, `src/vllm-sr/cli/`, `src/fleet-sim/fleet_sim/optimizer/`, `deploy/operator/api/v1alpha1/`, `deploy/operator/controllers/`, `dashboard/frontend/src/`, `dashboard/frontend/src/pages/`, `dashboard/frontend/src/components/`, and `dashboard/backend/handlers/`.

## Canonical Commands

- Harness: `make agent-bootstrap`, `make agent-validate`, `make agent-report ENV=cpu|amd CHANGED_FILES="..."`, `make agent-lint`, `make agent-ci-gate`, `make agent-pr-gate`
- Build and core: `make test-and-build-local`, `make test-semantic-router`, `make test-binding`, `make check-go-mod-tidy`
- Classifiers: `make test-category-classifier`, `make test-pii-classifier`, `make test-jailbreak-classifier`
- Runtime and E2E: `make agent-dev ENV=cpu|amd`, `make agent-serve-local ENV=cpu|amd`, `make agent-feature-gate ENV=cpu|amd CHANGED_FILES="..."`, `make agent-e2e-affected CHANGED_FILES="..."`
- Full lint: `pre-commit run --all-files`

## Rule Layers

- Entry and navigation: [tools/agent/docs/README.md](tools/agent/docs/README.md), [tools/agent/docs/governance.md](tools/agent/docs/governance.md)
- Architecture and boundaries: [tools/agent/docs/architecture-guardrails.md](tools/agent/docs/architecture-guardrails.md), nearest local `AGENTS.md`
- Testing and done criteria: [tools/agent/docs/feature-complete-checklist.md](tools/agent/docs/feature-complete-checklist.md)
- Executable contract: [tools/agent/repo-manifest.yaml](tools/agent/repo-manifest.yaml), [tools/agent/test-domain-registry.yaml](tools/agent/test-domain-registry.yaml), [tools/agent/task-matrix.yaml](tools/agent/task-matrix.yaml), [tools/agent/skill-registry.yaml](tools/agent/skill-registry.yaml), [tools/agent/e2e-profile-map.yaml](tools/agent/e2e-profile-map.yaml), [tools/agent/structure-rules.yaml](tools/agent/structure-rules.yaml)
- Maintainer ops: [tools/agent/docs/maintainer-ops.md](tools/agent/docs/maintainer-ops.md), [tools/agent/maintainer-policy.yaml](tools/agent/maintainer-policy.yaml)

Temporary working notes can exist when needed, but they are not part of the canonical harness unless promoted into the docs or executable rule layer above.

# GitHub Copilot Instructions

This repository uses an agent harness. Treat reviews and code suggestions as part of that harness instead of as isolated code edits.

## Source of Truth Order

When reviewing or generating changes, follow the repository rule layers in this order:

1. `AGENTS.md`
2. `tools/agent/docs/README.md`
3. relevant docs under `tools/agent/docs/*`
4. executable rule sources usnder `tools/agent/*`, `tools/make/agent.mk`, and `.github/workflows/*`
5. nearest local `AGENTS.md` for hotspot directories

Interpret them this way:

- `AGENTS.md` is a short entrypoint, not the full handbook.
- `tools/agent/docs/*` is the human-readable system of record.
- manifests, scripts, Make targets, and CI workflows are the executable rule layer.
- local `AGENTS.md` files are narrow hotspot supplements, not alternate truth.

Do not invent a second source of truth in reviews or code suggestions.

## Review Priorities

When reviewing a change, first check whether it follows the repository harness:

- the change starts from the correct project-level skill and impacted surfaces
- code, docs, and executable rules stay aligned when the change affects more than one layer
- contributor-facing wrappers such as `README.md`, `.github/PULL_REQUEST_TEMPLATE.md`, issue templates, and this file stay aligned with the canonical harness
- repeated governance requirements are enforced mechanically when possible instead of only described in prose

Then review the code change itself for bugs, regressions, missing validation, and maintainability issues.

## Harness-Specific Things to Check

- For harness changes, verify alignment with:
  - `tools/agent/docs/governance.md`
  - `tools/agent/docs/testing-strategy.md`
  - `tools/agent/docs/feature-complete-checklist.md`
  - `tools/agent/docs/skill-catalog.md`
  - `tools/agent/repo-manifest.yaml`
  - `tools/agent/task-matrix.yaml`
  - `tools/agent/skill-registry.yaml`
- For hotspot changes, verify the nearest local `AGENTS.md` was respected.
- For long-horizon work, verify active loop execution lives under `tools/agent/docs/plans/*.md` instead of ad hoc notes.
- For durable unresolved code/spec mismatches, verify the gap is promoted to an indexed debt entry under `tools/agent/docs/tech-debt/`.
- For durable governance or architecture decisions, verify they are recorded in `tools/agent/docs/adr/` when needed.

## Architecture and Modularity Checks

Review changes against `tools/agent/docs/architecture-guardrails.md` and `tools/agent/docs/module-boundaries.md`.

Flag issues when a change:

- grows a legacy hotspot without reducing responsibility elsewhere
- adds unrelated responsibilities into the same file
- uses large manager/god-object patterns where extraction or composition should be used
- introduces interfaces away from true seams
- ignores the repository layer model:
  - `signal`: extract information from request/response text
  - `decision`: boolean control logic using signals and conditions
  - `algorithm`: per-decision model selection
  - `plugin`: post-decision processing
  - `global`: intentionally cross-cutting behavior

## Validation Expectations

Review whether the change matches the repository validation ladder in `tools/agent/docs/testing-strategy.md`.

Flag missing validation when:

- behavior-visible routing, startup, config, Docker, CLI, or API changes do not update/add E2E unless the change is clearly a pure refactor
- harness-only changes do not update the relevant source-of-truth docs or executable rules
- contributor-interface changes drift away from the canonical harness entrypoints

## Commit Trajectory and PR Scoping

When generating or suggesting changes, structure commits as a reviewable trajectory:

1. Separate concerns into distinct commits (refactor, logic, tests).
2. Each commit must compile and pass lint — never break the tree mid-sequence.
3. Commit messages describe the "why", not just the "what".
4. Sign off all commits with `-s` (DCO required).

PR scoping expectations:

- Narrow the blast radius — touch only files necessary for the change.
- One subsystem per PR when possible; don't mix unrelated cleanups into a feature PR.
- PR titles use module-aligned prefixes: `[Router]`, `[Dashboard]`, `[Docs]`, `[CI/Build]`, `[Operator]`, etc.
- Include a Test Plan section describing validation steps and outcomes.
- Behavior-visible changes need E2E test updates unless the change is a pure refactor.

## Review Output Style

Prioritize findings about:

1. bugs or behavioral regressions
2. harness violations or source-of-truth drift
3. missing validation or E2E coverage
4. architecture, modularity, and hotspot-ratchet violations

Keep findings concrete, file-specific, and actionable.

If there are no findings, say so explicitly and briefly note any residual risks or validation gaps.