# Technical Documentation: Mesh-LLM/flash-moe

> ℹ️ **Provenance:** In-Tree Documentation (3 Chapters) · [CodeWiki Reference](https://codewiki.google/github.com/Mesh-LLM/flash-moe) · Recency: Active (< 180 days)

## File: README.md

# flash-moe

`flash-moe` is an external mesh-llm plugin for connecting mesh-llm to a
Flash-MoE OpenAI-compatible HTTP server.

The plugin is intentionally decoupled from mesh-llm internals. mesh-llm only
launches the plugin through the plugin API, passes `[[plugin]].url` as
`MESH_LLM_PLUGIN_URL`, and consumes the plugin-declared inference endpoint.

## Install

```bash
mesh-llm plugins install flash-moe
```

You can also install directly from GitHub:

```bash
mesh-llm plugins install Mesh-LLM/flash-moe
```

## Existing Endpoint Mode

If Flash-MoE is already running, attach the endpoint:

```toml
[[plugin]]
name = "flash-moe"
url = "http://127.0.0.1:8000/v1"
```

## Managed Process Mode

To let the plugin supervise a local Flash-MoE `infer` process, pass the backend
command and backend args to the plugin:

```toml
[[plugin]]
name = "flash-moe"
args = [
  "--backend-command", "/opt/flash-moe/metal_infer/infer",
  "--",
  "--model", "/models/qwen3.5-397b/model.gguf",
  "--weights", "/models/qwen3.5-397b/experts.bin",
  "--manifest", "/models/qwen3.5-397b/manifest.json",
  "--vocab", "/models/qwen3.5-397b/vocab.json"
]
```

The plugin allocates a local port and appends:

```text
--serve <port>
```

Do not pass `--serve` in backend args.

## Build

```bash
cargo build
```

## Release Archives

GitHub releases package archives using the same contract as
`Mesh-LLM/blackboard`:

- `flash-moe-<target>.tar.gz` or `.zip`
- `flash-moe-<version>-<target>.tar.gz` or `.zip`

Each archive contains:

- `flash-moe/flash-moe`
- `flash-moe/plugin.toml`

---

## File: .github/workflows/ci.yml

name: CI

on:
  pull_request:
  push:
    branches:
      - main

concurrency:
  group: ci-${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  rust:
    name: Rust
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Install Rust
        uses: dtolnay/rust-toolchain@stable
        with:
          components: clippy, rustfmt

      - name: Cache Rust
        uses: Swatinem/rust-cache@v2

      - name: Check formatting
        run: cargo fmt --all -- --check

      - name: Clippy
        run: cargo clippy --all-targets --locked -- -D warnings

      - name: Test
        run: cargo test --locked

      - name: Build
        run: cargo build --locked

---

## File: .github/workflows/release.yml

name: Release

on:
  push:
    tags:
      - "v*"
  workflow_dispatch:
    inputs:
      version:
        required: true
        type: string

permissions:
  contents: write

jobs:
  build:
    name: Build ${{ matrix.name }}
    runs-on: ${{ matrix.os }}
    strategy:
      fail-fast: false
      matrix:
        include:
          - name: linux-x86_64
            os: ubuntu-latest
            binary: flash-moe
            target: x86_64-unknown-linux-gnu
            archive_ext: tar.gz
            package: tar
          - name: macos-aarch64
            os: macos-14
            binary: flash-moe
            target: aarch64-apple-darwin
            archive_ext: tar.gz
            package: tar
          - name: windows-x86_64
            os: windows-latest
            binary: flash-moe.exe
            target: x86_64-pc-windows-msvc
            archive_ext: zip
            package: zip

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Install Rust
        uses: dtolnay/rust-toolchain@stable

      - name: Cache Rust
        uses: Swatinem/rust-cache@v2

      - name: Build release binary
        run: cargo build --release --locked

      - name: Package Unix archive
        if: matrix.package == 'tar'
        shell: bash
        run: |
          set -euo pipefail
          version="${{ github.event.inputs.version || github.ref_name }}"
          version="${version#v}"
          stable_archive="flash-moe-${{ matrix.target }}.${{ matrix.archive_ext }}"
          versioned_archive="flash-moe-${version}-${{ matrix.target }}.${{ matrix.archive_ext }}"
          mkdir -p dist/flash-moe
          cp "target/release/${{ matrix.binary }}" dist/flash-moe/flash-moe
          cp plugin.toml dist/flash-moe/plugin.toml
          tar -czf "$stable_archive" -C dist flash-moe
          cp "$stable_archive" "$versioned_archive"

      - name: Package Windows archive
        if: matrix.package == 'zip'
        shell: pwsh
        run: |
          $version = "${{ github.event.inputs.version || github.ref_name }}".TrimStart("v")
          $stableArchive = "flash-moe-${{ matrix.target }}.${{ matrix.archive_ext }}"
          $versionedArchive = "flash-moe-$version-${{ matrix.target }}.${{ matrix.archive_ext }}"
          New-Item -ItemType Directory -Force -Path dist/flash-moe | Out-Null
          Copy-Item "target/release/${{ matrix.binary }}" "dist/flash-moe/flash-moe.exe"
          Copy-Item plugin.toml "dist/flash-moe/plugin.toml"
          Compress-Archive -Path "dist/flash-moe" -DestinationPath $stableArchive
          Copy-Item $stableArchive $versionedArchive

      - name: Upload artifact
        uses: actions/upload-artifact@v4
        with:
          name: flash-moe-${{ matrix.target }}
          path: |
            flash-moe-*.tar.gz
            flash-moe-*.zip
          if-no-files-found: error

  publish:
    name: Publish GitHub release
    needs: build
    runs-on: ubuntu-latest
    steps:
      - name: Download artifacts
        uses: actions/download-artifact@v4
        with:
          path: artifacts
          merge-multiple: true

      - name: Publish release
        uses: softprops/action-gh-release@v2
        with:
          tag_name: ${{ github.event.inputs.version || github.ref_name }}
          name: ${{ github.event.inputs.version || github.ref_name }}
          files: artifacts/*
          fail_on_unmatched_files: true
          overwrite_files: true

--- METRICS ---
- Files Extracted: 3
- Estimated Token Budget: ~1541 tokens
- Recency Window: Active (< 180 days)
- Canonical Reference: https://codewiki.google/github.com/Mesh-LLM/flash-moe