{"owner":"OpenBMB","repo":"MiniCPM","hasSkills":true,"totalSkillsCount":18,"totalTokensCount":19427,"categories":["anthropic-skill"],"hasMcp":false,"mcpConfig":null,"found":["skills/minicpm5-deploy-ollama/SKILL.md","skills/minicpm5-deploy-litert/SKILL.md","skills/minicpm5-deploy-mlx/SKILL.md","skills/minicpm5-deploy-lmstudio/SKILL.md","skills/minicpm5-deploy-llama-cpp/SKILL.md","skills/minicpm5-deploy-arclight/SKILL.md","skills/minicpm5-deploy-sglang/SKILL.md","skills/minicpm5-deploy-transformers/SKILL.md","skills/minicpm5-finetune-gguf-lora/SKILL.md","skills/minicpm5-deploy-vllm-ascend/SKILL.md","skills/minicpm5-deploy-vllm/SKILL.md","skills/minicpm5-deploy/SKILL.md","skills/minicpm5-finetune-llamafactory/SKILL.md","skills/minicpm5-finetune-ms-swift/SKILL.md","skills/minicpm5-finetune-trl/SKILL.md","skills/minicpm5-finetune-unsloth/SKILL.md","skills/minicpm5-finetune-xtuner/SKILL.md","skills/minicpm5-finetune/SKILL.md"],"skills":{"skills/minicpm5-deploy-ollama/SKILL.md":"---\nname: minicpm5-deploy-ollama\ndescription: Run MiniCPM5-1B or MiniCPM5-2B via Ollama on macOS / Linux laptop using the released GGUF. Use when the user wants \"ollama run\", \"ollama pull\", a Modelfile-driven setup, or one-line laptop deployment.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with Ollama\n\nOne-binary, no-Python laptop deployment. Consumes the released GGUF.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install Ollama (once)\n\n```bash\nbrew install ollama                                # macOS\n# or:\ncurl -fsSL https://ollama.com/install.sh | sh      # Linux\n\nOLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve &\n```\n\n### 2. Download the GGUF + write Modelfile\n\n```bash\nmkdir -p ~/${MODEL_NAME} && cd ~/${MODEL_NAME}\n\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n\ncat > Modelfile <<EOF\nFROM ./MiniCPM5-2B-${QUANT}.gguf\n\n# MiniCPM5 basic chat template\nTEMPLATE \"\"\"{{- if .Messages -}}\n{{- range .Messages -}}\n<|im_start|>{{ .Role }}\n{{ .Content }}<|im_end|>\n{{ end -}}\n<|im_start|>assistant\n{{ end -}}\"\"\"\n\nPARAMETER stop \"<|im_end|>\"\nPARAMETER stop \"</s>\"\n\n# Defaults tuned for think mode\nPARAMETER temperature 1.0\nPARAMETER top_p 0.95\nPARAMETER num_ctx 8192\nEOF\n```\n\n### 3. Create + run\n\n```bash\nollama create ${MODEL_NAME} -f Modelfile\nollama run ${MODEL_NAME}\n```\n\n### 4. Validate via OpenAI-compatible API\n\n```bash\ncurl http://localhost:11434/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply. \n\n## MiniCPM5-1B Think vs No-think\n\nA MiniCPM5-1B Modelfile configured with `temperature=0.7, top_p=0.95` defaults to **no-think**. To switch a single conversation to **think** mode, override the sampling params at request time:\n\n```bash\nollama run minicpm5-1b --temperature 0.9 --top-p 0.95\n```\n\nOr bake it into a separate model tag by raising the temperature line (top_p stays 0.95):\n\n```Modelfile\nPARAMETER temperature 0.9\nPARAMETER top_p 0.95\n```\n\nThen `ollama create minicpm5-1b-think -f Modelfile.think`.\n\nOllama 0.24 does **not** directly evaluate the GGUF-embedded Jinja chat template. It maps recognized Jinja templates to built-in Go templates, so MiniCPM5 uses the explicit Go `TEMPLATE` block above. To enter the MiniCPM5-2B think path, use raw mode and prepend `<think>\\n` manually:\n\n```bash\ncurl http://localhost:11434/api/generate -d '{\n    \"model\": \"minicpm5-2b\",\n    \"raw\": true,\n    \"prompt\": \"<|im_start|>user\\n鸡兔同笼…<|im_end|>\\n<|im_start|>assistant\\n<think>\\n\",\n    \"options\": {\"temperature\": 1.0, \"top_p\": 0.95}\n}'\n```\n\n## Common pitfalls\n\n- **`Error: invalid file magic`**: corrupted download. Re-run `huggingface-cli download`.\n\n## When NOT to use\n\n- Highest throughput on Mac → `minicpm5-deploy-mlx` (Q4 build)\n- GUI experience → `minicpm5-deploy-lmstudio`\n- NVIDIA GPU production → `minicpm5-deploy-vllm`\n\n## Reference\n\n[`docs/deployment/ollama.md`](../../docs/deployment/ollama.md)\n","skills/minicpm5-deploy-litert/SKILL.md":"---\nname: minicpm5-deploy-litert\ndescription: Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime — the litert-lm CLI or its OpenAI-compatible server on a desktop, the Kotlin API or the AI Edge Gallery app on Android, the same .litertlm bundle on CPU or GPU. Use when the user says \"LiteRT\", \"LiteRT-LM\", \"litertlm\", \".litertlm\", \"Android\", \"Edge Gallery\", \"on-device app\", or wants one bundle for phones and desktops.\n---\n\n# Deploy MiniCPM5-2B / MiniCPM5-1B with LiteRT-LM (Android / iOS / desktop / IoT)\n\n[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM), Google's on-device runtime built on [LiteRT](https://github.com/google-ai-edge/litert) (formerly TensorFlow Lite). One `.litertlm` bundle, CPU or GPU, through a CLI, an OpenAI-compatible server, Python, Kotlin and Swift. The bundles are hosted in `litert-community`. Human-readable reference: [`docs/deployment/litert.md`](../../docs/deployment/litert.md).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `LITERT_REPO` | `litert-community/MiniCPM5-2B` or `litert-community/MiniCPM5-1B` | `litert-community/MiniCPM5-2B` |\n| `FILE` | `MiniCPM5-2B_int4.litertlm` (1.55 GB, the phone file) or `MiniCPM5-2B_int8.litertlm` (2.60 GB, reasoning that has to finish); 1B: `minicpm_wi4b32_wi8_afp32.litertlm` (CPU) or `minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm` (GPU) | `MiniCPM5-2B_int4.litertlm` |\n| `BACKEND` | `cpu` or `gpu` | `cpu` |\n| `THINKING` | `true` (reason first) or `false` (direct answer) | `false` |\n| `PROMPT` | `1+1=?` | `1+1=?` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\nuv tool install litert-lm\n```\n\n### 2A. Run a pre-converted bundle (desktop CLI)\n\n```bash\nlitert-lm run --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" \\\n    --backend \"${BACKEND}\" --thinking \"${THINKING:-false}\" --prompt \"${PROMPT}\"\n```\n\nThe first run downloads the file into `~/.litert-lm/cache/huggingface/<repo>/`; later runs skip the download. With thinking on, the reasoning prints between `[thought]` and `[/thought]` and the answer follows; `--thinking-budget 2048` caps the reasoning (4096 for math); `THINKING=true` makes the model reason first. Sampling as OpenBMB recommends: `--top-k 40 --top-p 0.95 --temperature 1.0` (the CLI's default top-k is 1 = greedy, so temperature alone changes nothing).\n\n### 2B. Android\n\n- **No code**: the [AI Edge Gallery](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery) app, Model manager → **+** → **Import from HF** → paste the file's Hugging Face link.\n- **Your own app**: `implementation(\"com.google.ai.edge.litertlm:litertlm-android:0.17.0\")` from Google Maven, the `libOpenCL.so` `<uses-native-library>` entries in the manifest for the GPU, then:\n\n```kotlin\nval engine = Engine(EngineConfig(modelPath = path, backend = Backend.GPU(), cacheDir = context.cacheDir.absolutePath))\nengine.initialize()\nengine.createConversation(ConversationConfig(maxOutputToken = 1024)).use { conversation ->\n    val reply = conversation.sendMessage(\"1+1=?\")\n    println(reply); println(reply.channels[\"thought\"])\n}\n```\n\n`ConversationConfig(thinkingConfig = ThinkingConfig(enableThinking = false))` turns thinking off. Full snippet, manifest and the Galaxy S26 check: the cookbook.\n\n### 2C. OpenAI-compatible server (desktop)\n\n```bash\nlitert-lm import --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" minicpm5-2b\nlitert-lm serve --host 127.0.0.1 --port 9379\ncurl http://127.0.0.1:9379/v1/chat/completions -H \"Content-Type: application/json\" \\\n    -d '{\"model\": \"minicpm5-2b\", \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}], \"reasoning_effort\": \"none\"}'\n```\n\n`reasoning_effort` `none` = direct answer, any other value = thinking on; the cap field is `max_completion_tokens`; `top_k` is accepted alongside `temperature` / `top_p` / `seed`; `\"stream\": true` streams ([server guide](https://developers.google.com/edge/litert-lm/cli/openai_server)).\n\n### 3. Validate\n\nThe reply contains `2` for `1+1=?`: with `--thinking false` it is the whole reply; with thinking on it follows `[/thought]`. Through the server (2C), the same check is HTTP 200 with `choices[0].message.content` containing `2`, and `GET /v1/models` lists `minicpm5-2b`.\n\n## Common pitfalls\n\n- **Thinking on the int4 file** runs long chains: pass `--thinking false` for direct answers, or use the int8 file when the reasoning has to complete.\n- **int8 on iOS needs the increased-memory-limit entitlement** (2.33 GB weight section; the default entitlements do not map it). int4's 1.28 GB main section is under the limit (checked on iPhone 17 Pro, GPU and CPU).\n- **First run per backend is slow**: an XNNPACK weight cache is written beside the file on the CPU, kernels are compiled on the GPU (`--cache no` to skip the disk cache).\n- **Android GPU without the manifest entries**: `initialize()` succeeds, the first message fails with `Can not find OpenCL library on this device`.\n- **Temperature without top-k does nothing**: pass `--top-k` above 1 together with `--top-p` / `--temperature` (the default is greedy).\n- **Scripts**: the CLI reads stdin; redirect it (`< /dev/null`) when running from a job with an open pipe.\n\n## Reference\n\n[`docs/deployment/litert.md`](../../docs/deployment/litert.md)\n","skills/minicpm5-deploy-mlx/SKILL.md":"---\nname: minicpm5-deploy-mlx\ndescription: Run MiniCPM5-1B or MiniCPM5-2B natively on Apple Silicon with Apple's MLX framework. Use when the user has an Apple Silicon Mac and asks for \"MLX\", \"mlx_lm\", \"mlx_lm.convert\", \"mlx_lm.generate\", or wants the fastest path on Apple Silicon.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with MLX (Apple Silicon)\n\nApple's on-device tensor framework. Highest throughput on M-series. Stays inside one Python process — no separate server, no `llama.cpp` build chain.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MLX_REPO` | `openbmb/MiniCPM5-2B-MLX` (pre-converted 4-bit affine) | required; `openbmb/MiniCPM5-1B-MLX` also works |\n| OR `HF_REPO` + `QUANT` | `openbmb/MiniCPM5-2B`, `4bit` or `bf16` | for local conversion |\n| `MAX_TOKENS` | `200` | `200` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"mlx-lm>=0.31\" \"gguf\"\n```\n\n### 2A. Use a pre-converted MLX repo (recommended when available)\n\n```bash\nmlx_lm.generate --model \"${MLX_REPO}\" \\\n    --prompt \"<|im_start|>user\n1+1=?<|im_end|>\n<|im_start|>assistant\n\" \\\n    --max-tokens ${MAX_TOKENS} --temp 1.0 --top-p 0.95\n```\n\n### 2B. Convert from a HF checkpoint locally (advanced)\n\nUse `mlx_lm.convert` only if you have a self-trained HF fp16 checkpoint:\n\n```bash\nHF=/path/to/your-fp16-hf\n\n# Convert: bf16\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-bf16\n\n# Convert: 4-bit (smaller / faster)\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-q4 -q --q-bits 4\n```\n\nThen run as in 2A.\n\n### 3. Validate\n\nThe reply should contain `\"2\"` for `1+1=?`.\n\n## OpenAI-compatible server (mlx-lm)\n\n```bash\nmlx_lm.server --model \"${MLX_REPO}\" --host 127.0.0.1 --port 8000\n\ncurl http://127.0.0.1:8000/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"default\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\n## Common pitfalls\n\n- **Slow first generate**: MLX JIT-compiles kernels on first call (~5-10 s); subsequent calls hit the warm cache.\n- **Model runs past `<|im_end|>`**: only happens on `mlx-lm < 0.31` (older versions ignore multi-id `eos_token_id` lists). Upgrade, or pass `--extra-eos-token \"<|im_end|>\"` as a manual override — `<|im_end|>` is token id 130073 and is already listed in `generation_config.json` on 0.31+.\n\n## When NOT to use\n\n- Not on Apple Silicon → `minicpm5-deploy-llama-cpp` (CPU/CUDA) or `minicpm5-deploy-vllm` (CUDA)\n- Want a desktop GUI → `minicpm5-deploy-lmstudio` (LM Studio bundles an MLX runtime)\n- Want one-line CLI run → `minicpm5-deploy-ollama`\n\n## Reference\n\n[`docs/deployment/mlx.md`](../../docs/deployment/mlx.md)\n","skills/minicpm5-deploy-lmstudio/SKILL.md":"---\nname: minicpm5-deploy-lmstudio\ndescription: Run MiniCPM5-1B or MiniCPM5-2B in LM Studio (desktop GUI) using either the GGUF runtime (cross-platform) or the MLX runtime (Apple Silicon, faster). Includes OpenAI-compatible local server. Use when the user mentions \"LM Studio\", desktop GUI inference, \"lms\" CLI, or wants a no-code chat UI for MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with LM Studio\n\nDesktop GUI + OpenAI-compatible local server. On Apple Silicon ships **two runtimes**:\n\n| Runtime | Format | When to use |\n| --- | --- | --- |\n| **GGUF** (llama.cpp engine) | F16 / Q8_0 / Q4_K_M | cross-platform, same artifact as Ollama; Q4_K_M build |\n| **MLX** (Apple Silicon only) | bf16 / 4-bit | ~60 % faster, automatic think/answer split via `reasoning_content`; Q4 build |\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| Runtime | `gguf` or `mlx` | `mlx` on Apple Silicon, `gguf` elsewhere |\n| `QUANT` | `Q4_K_M` (GGUF) or `4bit` (MLX) | `Q4_K_M` / `4bit` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install LM Studio + complete onboarding\n\n```bash\nbrew install --cask lm-studio\nopen -a \"LM Studio\"     # accept EULA + pick model source\n```\n\n> ⚠️ The first launch MUST be GUI — `lms` (CLI) refuses with `Cannot find LM Studio installation` until LM Studio has run interactively at least once.\n\n### 2A. GGUF runtime path\n\n```bash\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-${QUANT}.gguf \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF/\n\nLMS=\"/Applications/LM Studio.app/Contents/Resources/app/.webpack/lms\"\n\"$LMS\" server start                              # binds 127.0.0.1:1234\n\"$LMS\" load minicpm5-2b --gpu max --context-length 8192 -y\n\"$LMS\" ps                                        # verify the model is loaded\n```\n\n### 2B. MLX runtime path (Apple Silicon, recommended on Mac)\n\nThe MLX runtime needs an MLX-format checkpoint. The published MiniCPM5-2B MLX repo is [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) (4-bit affine); there is no separate `-bf16` / `-4bit` variant. Either drop that one in as-is, or convert locally from `openbmb/MiniCPM5-2B` (see `minicpm5-deploy-mlx`). Then:\n\n```bash\n# Option A — use the official pre-quantized repo\nhuggingface-cli download openbmb/MiniCPM5-2B-MLX \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX\n\n# Option B — drop a locally converted directory in\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}\ncp -r ./minicpm5-mlx-${QUANT}/* ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}/\n\n\"$LMS\" server start\n\"$LMS\" load minicpm5-2b-mlx${QUANT:+-${QUANT}} --gpu max -y\n```\n\n### 3. Validate\n\n```bash\ncurl http://127.0.0.1:1234/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nFor the MLX runtime, a think prompt produces output split into `message.reasoning_content` (the `<think>` block) and `message.content` (the final answer) automatically — that's an MLX-runtime feature, not a model setting.\n\n## Think vs nothink (MiniCPM5-1B)\n\nLM Studio 0.4.13's chat-completion endpoint does **not** propagate `chat_template_kwargs.enable_thinking` to the GGUF runtime. Instead:\n\n- **Default = think mode** for both runtimes.\n- **For nothink** with the GGUF runtime, prepend the closing think block manually:\n  ```json\n  \"messages\": [\n    {\"role\":\"user\",\"content\":\"1+1=?\"},\n    {\"role\":\"assistant\",\"content\":\"<think>\\n\\n</think>\\n\\n\"}\n  ]\n  ```\n  and the model continues from there.\n- **MLX runtime**: think/answer are auto-split, you don't need to do anything.\n\n## Common pitfalls\n\n- **MLX runtime not available**: only on Apple Silicon. On Intel Mac / Windows / Linux LM Studio, only the GGUF runtime works.\n\n## When NOT to use\n\n- Just want CLI / scripted runs → `minicpm5-deploy-ollama` is leaner\n- Production server → `minicpm5-deploy-vllm`\n- No GUI desired → `minicpm5-deploy-llama-cpp` (`llama-server`)\n\n## Reference\n\n[`docs/deployment/lmstudio.md`](../../docs/deployment/lmstudio.md)\n","skills/minicpm5-deploy-llama-cpp/SKILL.md":"---\nname: minicpm5-deploy-llama-cpp\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for \"llama.cpp\", \"llama-cli\", \"llama-server\", \"GGUF\", or has no Python available.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with llama.cpp\n\nCPU / edge / consumer-GPU deployment via the released GGUF artifacts. The artifacts work directly with vanilla `llama.cpp` and every downstream runtime (Ollama / LM Studio / `llama-cpp-python`).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `NGL` | `99` (all layers on GPU) / `0` (CPU only) | `99` if NVIDIA GPU, else `0` |\n| `CTX` | `8192` (default) up to `131072` (128 K) | `8192` |\n\n## Steps\n\n### 1. Install llama.cpp\n\n```bash\n# macOS\nbrew install llama.cpp\n\n# Linux / cross-platform: pre-built binary\ncurl -fsSL https://github.com/ggerganov/llama.cpp/releases/latest/download/llama-cli-linux.tar.gz | tar -xz\n# OR build from source:\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git && cd llama.cpp\nmkdir build && cd build\ncmake .. -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release    # CPU-only: omit GGML_CUDA=ON\ncmake --build . --config Release -j $(nproc) --target llama-cli llama-server\n```\n\n### 2. Download the GGUF\n\n```bash\nmkdir -p ~/minicpm5 && cd ~/minicpm5\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n```\n\n### 3a. Interactive chat (CLI)\n\n```bash\nllama-cli -m MiniCPM5-2B-${QUANT}.gguf \\\n    -n 2048 --temp 1.0 --top-p 0.95 --min-p 0.0 -ngl ${NGL} -c ${CTX}\n```\n\n### 3b. OpenAI-compatible HTTP server\n\n```bash\nllama-server -m MiniCPM5-2B-${QUANT}.gguf \\\n    --port 8080 -ngl ${NGL} -c ${CTX} --jinja\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:8080/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"min_p\": 0.0, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nIn llama.cpp, the default `min_p=0.05` can lead to repetitive output: it filters out tokens whose probability is below 5% of the highest-probability token, potentially discarding the exact tokens needed to break out of a repetition loop. To prevent this, we set `min_p=0.0`.\n\n## Sampling defaults\n\n| Mode | `--temp` | `--top-p` | `--min-p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | 1.0 | 0.95 | 0.0 |\n| MiniCPM5-1B Think | 0.9 | 0.95 | 0.0 |\n| MiniCPM5-1B No-think | 0.7 | 0.95 | 0.0 |\n\n## Choosing a quant\n\n| Quant | Disk | Quality |\n| --- | --- | --- |\n| F16 | 5.04 GB | reference |\n| Q8_0 | 2.68 GB | ~indistinguishable from F16 |\n| **Q4_K_M** | **1.56 GB** | small drop, ideal for laptops |\n\n## Common pitfalls\n\n- **Slow on CPU + large context**: drop `-c 131072` to `-c 8192` if you don't need 128 K.\n\n## Building your own GGUF (advanced)\n\nIf you've trained your own MiniCPM5-1B or MiniCPM5-2B variant, build a GGUF with:\n\n```bash\npython convert_hf_to_gguf.py /path/to/your-fp16-hf --outfile out/F16.gguf --outtype f16\nllama-quantize out/F16.gguf out/Q4_K_M.gguf Q4_K_M\n```\n\nTrained a **LoRA adapter** (not a full model) and want to apply it at runtime with `--lora` instead of baking it in? Convert it to a GGUF adapter — see **`minicpm5-finetune-gguf-lora`**.\n\n## When NOT to use\n\n- NVIDIA GPU + want OpenAI-compatible serving → `minicpm5-deploy-vllm`\n- Apple Silicon native → `minicpm5-deploy-mlx` is faster\n- Just want one-line desktop run → `minicpm5-deploy-ollama`\n- Want a desktop GUI → `minicpm5-deploy-lmstudio`\n\n## Reference\n\n[`docs/deployment/llama_cpp.md`](../../docs/deployment/llama_cpp.md)\n","skills/minicpm5-deploy-arclight/SKILL.md":"# Deploy ArcLight from Source (CPU)\n\nArcLight is a lightweight C/C++ LLM inference framework for unified-memory systems. The recommended path is to build from source, then run a GGUF model with `al-gen`, `al-chat`, or `al-ppl`.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL` | `/path/to/MiniCPM5-2B-Q4_0.gguf` | required |\n| `PROMPT` | `\"Hello!\"` | `\"Hello!\"` |\n| `THREADS` | `4` | choose for the target CPU |\n| `NUMA_MODE` | `none` or `tp` | `none` for first run |\n| `NODES` | `1`, `2`, `4` | `1` with `NUMA_MODE=none` |\n| `MAX_GEN` | `256` | `256` |\n\n## Steps\n\n### 1. Build from source\n\n```bash\ngit clone https://github.com/OpenBMB/ArcLight.git\ncd ArcLight\n\ncmake -B build -DARCLIGHT_BACKEND=AUTO\ncmake --build build --config Release -j 32\n```\n\nUse `ARCLIGHT_BACKEND=AUTO` by default. Set it explicitly only when needed:\n\n- `X86`: force the x86 backend\n- `NEON`: force the ARM NEON backend\n- `NONE`: build without architecture-specific backend code\n\n### 2. Prepare a GGUF model\n\nArcLight uses GGUF checkpoints from `llama.cpp`. The `nnml` backend only loads `f32 / f16 / q4_0 / q8_0 / q6_K / q8_K` tensor types — **`Q4_K_M` is not supported**.\n\nSupported model families: MiniCPM5-2B, Qwen3, Llama2.\n\nFor first validation use the released `Q8_0` (`openbmb/MiniCPM5-2B-GGUF`), or quantize an unreleased `Q4_0` yourself from the F16:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-F16.gguf --local-dir .\nllama-quantize ./MiniCPM5-2B-F16.gguf ./MiniCPM5-2B-Q4_0.gguf Q4_0\n```\n\n### 3A. One-shot generation\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\n### 3B. Interactive chat\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\nTo seed the first turn:\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n### 3C. Perplexity\n\n```bash\n./build/al-ppl \\\n    --model \"${MODEL}\" \\\n    --prompt \"Good morning, Miss Lee!\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n## Many-core CPU / NUMA\n\nStart with single-node mode for correctness:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\nUse cross-NUMA tensor parallelism on many-core machines:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa tp --nodes ${NODES} \\\n    --threads ${THREADS}\n```\n\nRules:\n\n- `--numa none` requires `--nodes 1`.\n- `--numa tp` requires `--nodes N` where `N > 1`.\n- In the current version, `NODES` should be a power of 2.\n- Choose `THREADS` so it can be evenly divided across `NODES`.\n- `--numa pp` is reserved for future pipeline parallelism and is not implemented.\n\n## Optional memory buffers\n\nIf allocation fails or the model is larger, pass manual buffer sizes:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --w_gb 4 --a_gb 8 --kv_gb 2 --work_gb 2\n```\n\nMeaning:\n\n- `--w_gb`: weight buffer size\n- `--a_gb`: activation buffer size\n- `--kv_gb`: KV cache buffer size\n- `--work_gb`: temporary workspace size\n\nIncrease `--kv_gb` for longer `--max_length`. Increase `--w_gb` for larger models.\n\n## Validate\n\nRun:\n\n```bash\nMODEL=/path/to/MiniCPM5-2B-Q4_0.gguf\nTHREADS=4\n\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"1+1=?\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_gen 64\n```\n\nThe reply should contain `2` or a short explanation that evaluates to `2`.\n\n## Common pitfalls\n\n- **Program aborts in single-node mode**: use `--numa none --nodes 1`.\n- **Tensor parallel mode fails**: use `--numa tp --nodes N` with `N > 1`; in this version, `N` should be a power of 2.\n- **Pipeline parallelism fails**: `--numa pp` is not implemented yet.\n- **Model loading fails**: verify the checkpoint is GGUF and from a supported model family.\n- **Out of memory**: increase `--w_gb`, `--a_gb`, `--kv_gb`, or `--work_gb`; longer contexts usually need a larger KV cache.\n- **Poor CPU throughput**: check `--threads`, NUMA layout, and thread-core bindings; use `--print_binding 1 --print_perf 1` for diagnostics.\n\n## When NOT to use\n\n- Need CUDA server inference -> use a GPU-oriented runtime instead.\n- Need Apple Silicon MLX inference -> use an MLX deployment path instead.\n- Need a desktop GUI -> use a GUI runtime that supports GGUF models.\n- Need pipeline parallelism -> wait for ArcLight `--numa pp` support.\n\n## Reference\n\n- [`Arclight`](https://github.com/OpenBMB/ArcLight)\n- [llama.cpp GGUF documentation](https://github.com/ggml-org/llama.cpp)\n","skills/minicpm5-deploy-sglang/SKILL.md":"---\nname: minicpm5-deploy-sglang\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for \"SGLang\", \"RadixAttention\", \"prefix cache\", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with SGLang\n\nOpenAI-compatible server with RadixAttention prefix cache. Best fit for **tool calling**, batched eval pipelines, and high-concurrency serving.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B`| required; |\n| `PORT` | `30000` | `30000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TOOL_PARSER` | `minicpm5` | `minicpm5`; use `auto` if you want template detection |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"sglang[srt]>=0.5.16\"          # latest, requires CUDA 13.x driver\n# pip install \"sglang==0.5.6.post3\"        # fallback for CUDA 12.x driver hosts\n```\n\n**For tool calling, install from `main`** — the `minicpm5` parser ([PR #25600](https://github.com/sgl-project/sglang/pull/25600), merged 2026-05-22) is not in any pip release yet (`v0.5.12.post1` was branched earlier). Plain chat works on the pip release; only `--tool-call-parser minicpm5` needs `main`:\n\n```bash\npip install \"git+https://github.com/sgl-project/sglang.git@main#subdirectory=python\"\n```\n\n### 2. Recommended runtime env vars\n\n```bash\nexport VLLM_WORKER_MULTIPROC_METHOD=spawn\nexport SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1\nexport SGLANG_DISABLE_CUDNN_CHECK=1\n```\n\n### 3. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT}\n```\n\nWait for `The server is fired up and ready to roll!` .\n\n**Speculative decoding (DSpark)**: we also release [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark), a DSpark draft model trained for MiniCPM5-2B. Enable it in SGLang to accelerate decoding while keeping the target model's outputs unchanged:\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT} \\\n    --speculative-algorithm DSPARK \\\n    --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \\\n    --speculative-dspark-block-size 7\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains `\"2\"`.\n\n## Tool calling\n\nSGLang includes a native MiniCPM5 XML tool-call parser. Keep `TOOL_PARSER=minicpm5` (or `auto`) when launching, then send an OpenAI-style tools request:\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n        \"tools\": [{\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get current weather for a city\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\"city\": {\"type\": \"string\"}},\n                    \"required\": [\"city\"]\n                }\n            }\n        }],\n        \"tool_choice\": \"auto\",\n        \"temperature\": 1.0, \"top_p\": 0.95,\n        \"max_tokens\": 256\n    }'\n```\n\n## Offline / batched (Engine API)\n\n```python\nimport sglang as sgl\n\nllm = sgl.Engine(model_path=\"${MODEL_PATH}\", tp_size=1, mem_fraction_static=0.8, context_length=131072)\n\noutputs = llm.generate(\n    [\"用一句话解释什么是 GQA。\"],\n    sampling_params={\"temperature\": 1.0, \"top_p\": 0.95, \"max_new_tokens\": 1024,\n                     \"skip_special_tokens\": False},\n)\nprint(outputs)\n```\n\n## Common pitfalls\n\n- **`GLIBCXX_3.4.31 not found`** (only if you hit it — not universal): conda Python ships an older `libstdc++` than the SGLang wheel was built against. Force-load the system one: `LD_PRELOAD=/lib/x86_64-linux-gnu/libstdc++.so.6 python -m sglang.launch_server ...`\n\n## When NOT to use\n\n- Production serving without batch-eval needs → `minicpm5-deploy-vllm` is simpler\n- One-shot Python → `minicpm5-deploy-transformers`\n- No NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n\n## Reference\n\n[`docs/deployment/sglang.md`](../../docs/deployment/sglang.md)\n","skills/minicpm5-deploy-transformers/SKILL.md":"---\nname: minicpm5-deploy-transformers\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with Hugging Face Transformers for one-shot Python generation on GPU (bfloat16) or CPU (float32). Use when the user wants a quick Python script, no server, no extra deps, or asks for \"Transformers\", \"AutoModelForCausalLM\", \"model.generate\" with MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with HF Transformers\n\nOne-shot Python generation. No server. Works on a single GPU (bfloat16) or CPU only (fp32).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or local dir | required; `openbmb/MiniCPM5-1B` also works |\n| `MODE` | `think` or `nothink` (`nothink` is 1B-only) | `think` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install -U \"transformers>=5.6,<6\" \"torch>=2.11\" accelerate     # latest (CUDA 13.x driver hosts)\n# pip install -U \"transformers==4.57.3\" \"torch==2.7.1\" accelerate  # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Run\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_path = \"${MODEL_PATH}\"      # ← replace\ntok = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_path,\n    torch_dtype=torch.bfloat16,    # CPU users: torch.float32 + device_map=\"cpu\"\n    device_map=\"auto\",\n).eval()\n\nmessages = [{\"role\": \"user\", \"content\": \"用一句话解释什么是 GQA。\"}]\ninputs = tok.apply_chat_template(\n    messages,\n    add_generation_prompt=True,\n    enable_thinking=True,\n    return_tensors=\"pt\",\n    return_dict=True,\n).to(model.device)\n\nwith torch.no_grad():\n    out = model.generate(\n        **inputs,\n        max_new_tokens=1024,\n        do_sample=True,\n        temperature=1.0,\n        top_p=0.95,\n    )\n\nprompt_len = inputs[\"input_ids\"].shape[-1]\nprint(tok.decode(out[0][prompt_len:], skip_special_tokens=True))\n```\n\nFor CPU only: change `torch_dtype=torch.float32, device_map=\"cpu\"`. Keep `enable_thinking=True` and `temperature=1.0` for MiniCPM5-2B; use `enable_thinking=False` and `temperature=0.7` only for MiniCPM5-1B No-think mode.\n\n## Sampling defaults\n\n| Mode | `enable_thinking` | `temperature` | `top_p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | `True` | 1.0 | 0.95 |\n| MiniCPM5-1B Think | `True` | 0.9 | 0.95 |\n| MiniCPM5-1B No-think | `False` | 0.7 | 0.95 |\n\n## Validate\n\nA coherent answer to `1+1=?` (e.g. `\"2\"` or `\"答案是 2\"`).\n\n## LoRA inference\n\n```python\nfrom peft import PeftModel\nbase = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=\"auto\")\nmodel = PeftModel.from_pretrained(base, \"/path/to/adapter\").eval()\n```\n\nAdapters from any of the `minicpm5-finetune-*` skills load directly with no surgery.\n\n## When NOT to use\n\n- Need an OpenAI-compatible HTTP server → `minicpm5-deploy-vllm` or `minicpm5-deploy-sglang`\n- Apple Silicon → `minicpm5-deploy-mlx` is faster\n- CPU-only or low-VRAM laptop → `minicpm5-deploy-llama-cpp` with Q4_K_M is faster\n\n## Reference\n\n[`docs/deployment/transformers.md`](../../docs/deployment/transformers.md)\n","skills/minicpm5-finetune-gguf-lora/SKILL.md":"---\nname: minicpm5-finetune-gguf-lora\ndescription: Fine-tune MiniCPM5-1B into a LoRA adapter and convert it to a GGUF adapter that loads directly into llama.cpp / llama-server and the MiniCPM Desk Pet app's custom-LoRA upload. Use when the user wants \"GGUF LoRA\", \"convert LoRA to GGUF\", \"convert_lora_to_gguf\", a custom persona/skin for the desktop pet, \"桌宠自定义 LoRA\", \"上传 LoRA 到桌宠\", or asks how to take a trained adapter and run it on a GGUF base.\n---\n\n# Fine-tune MiniCPM5-1B → GGUF LoRA adapter\n\nThe framework skills (`minicpm5-finetune-*`) all emit a **PEFT adapter** (`adapter_model.safetensors` + `adapter_config.json`). But `llama.cpp` / `llama-server` — and the **MiniCPM Desk Pet** app's custom-LoRA upload — load a **GGUF LoRA adapter** (`--lora some-adapter.gguf`). This skill is the bridge: train a PEFT LoRA, then convert it to GGUF and run/upload it.\n\n> This is the path you want when the base model is served as GGUF (Desk Pet, Ollama, LM Studio, plain `llama-server`). If you serve the fp16 HF base with vLLM / transformers, you don't need GGUF — load the PEFT adapter directly.\n\n## Pipeline overview\n\n```\n  train (any minicpm5-finetune-* skill)        this skill\n ┌────────────────────────────────────┐   ┌──────────────────────────────────┐\n BASE (fp16 HF) ─► adapter_model.safetensors ─► convert_lora_to_gguf.py ─► adapter.gguf\n                   adapter_config.json                                          │\n                                                                  llama-server --lora / Desk Pet upload\n```\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `ADAPTER_DIR` | `./runs/minicpm5_unsloth/adapter_final` | required — a PEFT dir with `adapter_config.json` + `adapter_model.safetensors` |\n| `BASE_MODEL` | `openbmb/MiniCPM5-1B` (HF id) **or** a local fp16 HF dir | required — must be the **same base** the adapter was trained on |\n| `OUTTYPE` | `f16` (recommended) / `q8_0` / `bf16` / `f32` | `f16` |\n| `OUT_GGUF` | `./minicpm5-mylora.gguf` | `<ADAPTER_DIR>/adapter_model.f16.gguf` |\n\n> **Don't have an adapter yet?** First run a training skill — start from the router **`minicpm5-finetune`** (or go straight to **`minicpm5-finetune-unsloth`** for single-GPU LoRA). Come back here with `ADAPTER_DIR` pointing at its output.\n\n## Steps\n\n### 1. Get llama.cpp (has the converter)\n\n```bash\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git\ncd llama.cpp\npip install -r requirements.txt          # converter deps: torch, safetensors, gguf, transformers\n```\n\n`convert_lora_to_gguf.py` lives at the repo root. You only need the Python script for conversion; build the C++ binaries (step 4) only if you want to test locally.\n\n### 2. Convert the PEFT adapter → GGUF\n\n```bash\npython convert_lora_to_gguf.py \"$ADAPTER_DIR\" \\\n    --base \"$BASE_MODEL\" \\\n    --outtype f16 \\\n    --outfile \"$OUT_GGUF\"\n```\n\n- `--base` only needs the base model's **config** (`config.json`, `tokenizer.json`) — not the weights. A local fp16 HF dir or an HF id both work.\n- If the base is only on the Hub, use `--base-model-id openbmb/MiniCPM5-1B` instead of `--base` (it pulls just the config).\n- Output is a single `.gguf` of roughly the same size as the input `.safetensors` (a 22 MB r=16 adapter → ~22 MB GGUF).\n\n> 🔑 **The #1 gotcha — `base_model_name_or_path` points at the training machine.**\n> PEFT writes the absolute path of the base used during training into `adapter_config.json`\n> (e.g. `/user/.../MiniCPM5-models-fixed/official`). On any other machine the converter\n> can't find it. **Always pass `--base` (or `--base-model-id`) explicitly** to override it —\n> don't rely on whatever is baked into the config. The base you pass MUST match the one you\n> trained on, or the adapter math is meaningless.\n\n### 3. (Optional) Sanity-check the GGUF metadata\n\n```bash\npython -c \"import gguf,sys; r=gguf.GGUFReader('$OUT_GGUF'); print('tensors:', len(r.tensors)); print('arch:', r.get_field('general.architecture').parts[-1].tobytes().decode() if r.get_field('general.architecture') else '?')\"\n```\n\nExpect `general.type = adapter` and a nonzero tensor count. MiniCPM5-1B is a **Llama-architecture** model, so the converter treats it as `llama` — this is correct, not an error.\n\n### 4. Test with llama-server before shipping\n\nYou need a GGUF **base** model too (the adapter is applied on top of it). Grab the released base:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-1B-GGUF MiniCPM5-1B-Q8_0.gguf --local-dir .\n```\n\nThen:\n\n```bash\n# CLI\nllama-cli -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" \\\n    -p \"你好\" -n 128 --temp 0.7 --top-p 0.95\n\n# OpenAI-compatible server\nllama-server -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" --port 8080 --jinja\n```\n\nIf the persona/behavior you trained shows up, the GGUF is good. If output is identical to the base, the adapter didn't load (check the `--lora` path and that `--base` matched in step 2).\n\n> **Quant compatibility:** a GGUF LoRA built from an fp16 adapter applies fine on top of a\n> quantized base (Q8_0 / Q4_K_M). You do **not** need a separate adapter per base quant.\n\n### 5. Use it in MiniCPM Desk Pet\n\nThe Desk Pet app loads exactly this kind of file. The fp16 GGUF you just built is upload-ready:\n\n1. Open **Settings → MiniCPM → 适配器 (LoRA)**.\n2. Click **上传** (Upload), pick your `.gguf` — the app copies it into its\n   `<userData>/adapters/uploads/` directory and registers it.\n3. Give it a display name + comma-separated aliases (used so you can switch personas by\n   voice/chat, e.g. \"换成 XX\").\n4. Select it in the adapter list to activate. The sidecar reloads `llama-server` with your\n   `--lora`.\n\nConstraints the app enforces (match them or the upload is rejected):\n\n- **Must be a single `.gguf` file** (the GGUF LoRA from step 2 — *not* the `.safetensors`,\n  *not* a merged full model).\n- It is applied on top of whatever MiniCPM5-1B GGUF base the app already runs, so **train\n  against the MiniCPM5-1B base**, not some other model.\n\n## Common pitfalls\n\n- **`FileNotFoundError` / `can't load base model config`** during conversion → the\n  `adapter_config.json` base path doesn't exist locally. Fix with `--base <local-dir>` or\n  `--base-model-id openbmb/MiniCPM5-1B` (step 2 gotcha).\n- **Adapter loads but output is unchanged** → base mismatch (trained on base A, applied on\n  base B), or the persona needs its **system prompt** too — a LoRA biases style but the\n  system prompt still matters. In Desk Pet the persona's system prompt is wired to the\n  adapter; standalone `llama-cli` users must pass it themselves.\n- **`KeyError` on an unknown tensor / target module** → the adapter targeted modules the\n  converter doesn't map. Stick to the standard attention+MLP projections\n  (`q/k/v/o_proj`, `gate/up/down_proj`) when training (all `minicpm5-finetune-*` skills\n  already default to these).\n- **Uploaded `.gguf` but Desk Pet shows nothing** → you uploaded the merged full model or a\n  base GGUF by mistake. A LoRA adapter GGUF is small (tens of MB) and has\n  `general.type = adapter` (step 3).\n\n## When NOT to use\n\n- Serving the fp16 HF base with **vLLM / transformers / SGLang** → load the PEFT adapter\n  directly, skip GGUF. See `minicpm5-deploy-vllm` / `-transformers` / `-sglang`.\n- You want to **bake the LoRA into the weights** (no runtime `--lora`) → merge first\n  (`model.save_pretrained_merged(...)` or `peft ... merge_and_unload()`), then convert the\n  **merged fp16 model** with `convert_hf_to_gguf.py` (see the \"Building your own GGUF\"\n  section of `minicpm5-deploy-llama-cpp`). That produces a standalone base GGUF, not an\n  adapter.\n\n## Reference\n\n- Training frameworks: **`minicpm5-finetune`** (router) and its sub-skills.\n- Running GGUF: **`minicpm5-deploy-llama-cpp`**.\n- Converter: `convert_lora_to_gguf.py` in [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp).\n","skills/minicpm5-deploy-vllm-ascend/SKILL.md":"---\nname: minicpm5-deploy-vllm-ascend\ndescription: Deploy MiniCPM5-2B with vLLM on Huawei Ascend NPU using vLLM-Ascend. Use when the user mentions vLLM-Ascend, Ascend NPU, Huawei Ascend, CANN, torch_npu, davinci devices, or wants an OpenAI-compatible MiniCPM5 server on Ascend hardware.\n---\n\n# Deploy MiniCPM5-2B with vLLM-Ascend\n\nUse this skill for the BF16 / FP16 `openbmb/MiniCPM5-2B` checkpoint on Huawei Ascend NPU. `vllm-ascend` is installed alongside a matching vLLM release and supplies the Ascend backend.\n\n## Required input\n\n| Variable | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required |\n| `PORT` | `8000` | `8000` |\n| `ASCEND_RT_VISIBLE_DEVICES` | `0` | `0` |\n| `CTX_LEN` | `32768` | `32768`; the native maximum is `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TP_SIZE` | `1` | number of visible devices |\n\nAccept either the Hugging Face id or a local directory containing the MiniCPM5-2B model files.\n\n## Prerequisites\n\nUse a compatible Linux Ascend environment with CANN, PyTorch, and TorchNPU installed. Keep the vLLM and vLLM-Ascend versions matched. The basic serving recipe below follows the tested `vllm==0.18.0` / `vllm-ascend==0.18.0` pair.\n\nDevice visibility is controlled by `ASCEND_RT_VISIBLE_DEVICES`; `CUDA_VISIBLE_DEVICES` does not select Ascend devices.\n\n## Install\n\nThe prebuilt image is the simplest option for a compatible Ascend host. This is a minimal single-device example; adjust device nodes and driver mounts for the host hardware and image documentation.\n\n```bash\ndocker run -it --net=host --shm-size=1g \\\n    --device /dev/davinci0 --device /dev/davinci_manager \\\n    --device /dev/devmm_svm --device /dev/hisi_hdc \\\n    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \\\n    -v /usr/local/dcmi:/usr/local/dcmi \\\n    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \\\n    quay.io/ascend/vllm-ascend:v0.18.0\n```\n\nFor an existing CANN environment:\n\n```bash\npip install vllm==0.18.0 vllm-ascend==0.18.0 \\\n    --extra-index-url https://download.pytorch.org/whl/cpu\n```\n\nFor built-in MiniCPM5 tool-call parsing, use the matching `vllm==0.23.0` / `vllm-ascend==0.23.0` stack described in [Tool calling](#tool-calling). The parser responses in the cookbook were produced with the upstream parser file on an older vLLM and a compatibility shim because the 0.23.0 stack was not available on the test hardware.\n\n## Prepare the environment\n\nRun the CANN and NNAL setup scripts before starting vLLM. Change the paths if those components are installed elsewhere.\n\n```bash\nsource /usr/local/Ascend/ascend-toolkit/set_env.sh\nsource /usr/local/Ascend/nnal/atb/set_env.sh\n\nexport ASCEND_RT_VISIBLE_DEVICES=${ASCEND_RT_VISIBLE_DEVICES:-0}\nexport LD_LIBRARY_PATH=$(python3 -c \"import torch,os;print(os.path.dirname(torch.__file__))\")/lib:\\\n$(python3 -c \"import torch_npu,os;print(os.path.dirname(torch_npu.__file__))\")/lib:$LD_LIBRARY_PATH\n```\n\nCheck the driver and Python packages before loading the model:\n\n```bash\nnpu-smi info\npython3 -c \"import torch, torch_npu; print(torch_npu.npu.is_available(), torch_npu.npu.device_count())\"\n```\n\nIf `npu-smi` reports a missing driver library, add the corresponding driver `lib64` directories to `LD_LIBRARY_PATH` according to the host installation.\n\n## Launch the server\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --port ${PORT:-8000}\n```\n\nSet `TP_SIZE` to the number of devices in `ASCEND_RT_VISIBLE_DEVICES` when using tensor parallelism.\n\n## Validate chat completions\n\nMiniCPM5-2B is think-only. Use `temperature=1.0`, `top_p=0.95`, and `enable_thinking=true`:\n\n```bash\ncurl http://localhost:${PORT:-8000}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0,\n        \"top_p\": 0.95,\n        \"max_tokens\": 256,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpect HTTP 200 and an answer containing `2` in `choices[0].message.content`. Leave enough `max_tokens` for the model's reasoning text.\n\n## Tool calling\n\nMiniCPM5-2B emits XML-style tool calls. This section requires matching vLLM and vLLM-Ascend `0.23.0`; that release includes the `minicpm5` parser in the OpenAI server layer, so no parser plugin file is needed:\n\n```bash\npip install vllm==0.23.0\npip install vllm-ascend==0.23.0 \\\n    --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi\n```\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --enable-auto-tool-choice \\\n    --tool-call-parser minicpm5 \\\n    --port ${PORT:-8000}\n```\n\nWhen sending `tools`, use both server flags above and the same 2B sampling settings:\n\n```json\n{\n  \"model\": \"MiniCPM5-2B\",\n  \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n  \"tools\": [{\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_weather\",\n      \"description\": \"Get current weather for a city\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\"city\": {\"type\": \"string\"}},\n        \"required\": [\"city\"]\n      }\n    }\n  }],\n  \"tool_choice\": \"auto\",\n  \"temperature\": 1.0,\n  \"top_p\": 0.95,\n  \"max_tokens\": 512,\n  \"chat_template_kwargs\": {\"enable_thinking\": true}\n}\n```\n\n## Common failures\n\n- `torch_npu` cannot import: source the CANN/NNAL environment and check that the installed PyTorch, TorchNPU, CANN, vLLM, and vLLM-Ascend versions match the supported stack.\n- No devices are visible: check `npu-smi info`, `ASCEND_RT_VISIBLE_DEVICES`, and the Docker device/driver mounts. Do not use `CUDA_VISIBLE_DEVICES` for this backend.\n- HBM allocation or 128K startup failure: lower `--max-model-len`, then lower `--gpu-memory-utilization`.\n- Tensor-parallel initialization failure: make `--tensor-parallel-size` match the number of visible Ascend devices.\n- Tool requests fail with an auto-tool-choice error: launch with both `--enable-auto-tool-choice` and `--tool-call-parser minicpm5`.\n\n## Reference\n\nSee [`docs/deployment/vllm-ascend.md`](../../docs/deployment/vllm-ascend.md) for the human-readable cookbook and the [official vLLM-Ascend installation guide](https://docs.vllm.ai/projects/ascend/en/v0.23.0/installation.html) for hardware-specific setup.\n","skills/minicpm5-deploy-vllm/SKILL.md":"---\nname: minicpm5-deploy-vllm\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for \"vLLM\", \"OpenAI server\", \"REST API for MiniCPM5\", or \"production deployment\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with vLLM\n\nOpenAI-compatible server for the BF16 / FP16 MiniCPM5-1B or MiniCPM5-2B checkpoint.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` also works |\n| `PORT` | `8000` | `8000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072`; lower if VRAM tight |\n| `MEM_FRAC` | `0.85` | `0.85`; lower on shared GPUs |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"vllm>=0.21\"          # latest (CUDA 13.x driver hosts)\n# pip install \"vllm==0.10.1.1\"    # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} vllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN} \\\n    --gpu-memory-utilization ${MEM_FRAC} \\\n    --port ${PORT}\n```\n\nWait for `Application startup complete` in the log.\n\n### 3. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains a coherent answer.\n\n## Sampling defaults\n\n```json\n{\"temperature\": 1.0, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-2B Think\n{\"temperature\": 0.9, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-1B Think\n{\"temperature\": 0.7, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": false}}   // MiniCPM5-1B No-think\n```\n\n## Common pitfalls\n\n- **`(free / total) < MEM_FRAC` hard error**: lower `--gpu-memory-utilization` (e.g. 0.5 on a shared GPU).\n- **OOM at startup with 128 K**: drop `--max-model-len` to 32768 or 8192.\n\n## Tool calling (plugin)\n\nThe vLLM-side MiniCPM5 XML parser ([PR #43175](https://github.com/vllm-project/vllm/pull/43175)) merged to `main` on 2026-05-27 but is **not in any pip release yet** (`v0.22.0` was cut before the merge). Use the bridge plugin shipped at `tool_parsers/minicpm5xml_tool_parser.py` in this repo:\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 --max-model-len ${CTX_LEN} --port ${PORT} \\\n    --enable-auto-tool-choice \\\n    --tool-parser-plugin /path/to/MiniCPM/tool_parsers/minicpm5xml_tool_parser.py \\\n    --tool-call-parser minicpm5\n```\n\nDrop `--tool-parser-plugin` once vLLM ships a release containing the parser natively.\n\n## When NOT to use\n\n- One-shot Python script → `minicpm5-deploy-transformers`\n- Apple Silicon / no NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n- High-concurrency batch eval w/ prefix cache or tool calling → `minicpm5-deploy-sglang`\n\n## Reference\n\n[`docs/deployment/vllm.md`](../../docs/deployment/vllm.md)\n","skills/minicpm5-deploy/SKILL.md":"---\nname: minicpm5-deploy\ndescription: Pick the right inference backend for a MiniCPM5-1B or MiniCPM5-2B checkpoint and route to a backend-specific cookbook skill. Use when the user wants to deploy / serve / chat-with / benchmark a MiniCPM5 model and has not yet committed to a specific engine, or when they say \"deploy MiniCPM5\", \"run MiniCPM5\", \"serve MiniCPM5\", \"MiniCPM5 推理\", \"部署 MiniCPM5\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B — backend router\n\nYou're being asked to deploy / serve / chat-with a MiniCPM5-1B or MiniCPM5-2B checkpoint. Your job is to **pick exactly one backend skill below** based on the user's hardware, format, and goal, then **invoke that skill** rather than improvising.\n\n## 1. Required input from the user\n\nBefore picking a backend, you MUST know:\n\n| Variable | Example | Where to ask |\n| --- | --- | --- |\n| `MODEL_PATH` | HF id `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B` **or** a local path | \"Which checkpoint? HF id or local path?\" |\n| Hardware | NVIDIA GPU / Apple Silicon / CPU only | infer from context, otherwise ask |\n| Goal | \"interactive chat\" / \"OpenAI server\" / \"Python script\" / \"benchmark\" | infer from context |\n\n### Available checkpoints on Hugging Face\n\n| Variant | HF repo | Use with |\n| --- | --- | --- |\n| **HF fp16 (recommended)** | [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) or [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) | `transformers` / `vllm` (no `--quantization`) / `vllm-ascend` / `sglang` / any `minicpm5-finetune-*` |\n| GGUF F16 / Q8_0 / Q4_K_M | [`openbmb/MiniCPM5-2B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) or [`openbmb/MiniCPM5-1B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) | `minicpm5-deploy-llama-cpp` / `-ollama` / `-lmstudio` |\n| MLX (Apple Silicon) | [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) or [`openbmb/MiniCPM5-1B-MLX`](https://huggingface.co/openbmb/MiniCPM5-1B-MLX) | `minicpm5-deploy-mlx` |\n| LiteRT-LM `.litertlm` (Android / iOS / desktop / IoT, CPU + GPU) | [`litert-community/MiniCPM5-2B`](https://huggingface.co/litert-community/MiniCPM5-2B) or [`litert-community/MiniCPM5-1B`](https://huggingface.co/litert-community/MiniCPM5-1B) | `minicpm5-deploy-litert` |\n\nIf the user has a local copy, accept any directory path that contains `config.json` and `model.safetensors` (or the equivalent GGUF / MLX layout).\n\n## 2. Decision matrix — pick exactly one\n\n| User says / wants | Hardware | Format | → Skill to invoke |\n| --- | --- | --- | --- |\n| \"Quick Python script\" / \"one-shot generation\" / \"no server\" | any GPU or CPU | HF safetensors | **`minicpm5-deploy-transformers`** |\n| \"OpenAI server\" / \"production serving\" / \"high QPS\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-vllm`** |\n| \"vLLM-Ascend\" / \"Ascend NPU\" / \"CANN\" / \"torch_npu\" | Huawei Ascend NPU | HF safetensors | **`minicpm5-deploy-vllm-ascend`** |\n| \"RadixAttention\" / \"prefix cache\" / \"batched eval\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-sglang`** |\n| \"GGUF\" / \"llama.cpp\" / \"llama-cli\" / \"CPU only\" | any CPU + optional GPU | GGUF | **`minicpm5-deploy-llama-cpp`** |\n| \"Ollama\" / \"ollama run\" / \"Modelfile\" | macOS / Linux laptop | GGUF | **`minicpm5-deploy-ollama`** |\n| \"LM Studio\" / \"desktop GUI\" | macOS / Windows / Linux | GGUF or MLX | **`minicpm5-deploy-lmstudio`** |\n| \"MLX\" / \"Apple Silicon native\" / \"fastest on Mac\" | Apple Silicon | MLX | **`minicpm5-deploy-mlx`** |\n| \"Android\" / \"on-device app\" / \"Edge Gallery\" / \"LiteRT\" / \"LiteRT-LM\" / \"litertlm\" / \"Raspberry Pi\" | Android phone, iPhone, desktop or IoT board (CPU or GPU) | LiteRT-LM `.litertlm` | **`minicpm5-deploy-litert`** |\n\nIf the user **has not specified** any of the above and asks \"how do I run this?\":\n\n- **CUDA box, want fastest server**: pick `minicpm5-deploy-vllm`.\n- **Ascend NPU, want an OpenAI-compatible server**: pick `minicpm5-deploy-vllm-ascend`.\n- **CUDA box, want minimal Python**: pick `minicpm5-deploy-transformers`.\n- **Apple Silicon laptop**: pick `minicpm5-deploy-ollama` (easiest) or `minicpm5-deploy-mlx` (fastest).\n- **Android phone, or an on-device app (Android / iOS)**: pick `minicpm5-deploy-litert`.\n- **CPU only / Windows / low-VRAM**: pick `minicpm5-deploy-llama-cpp` (Q4_K_M).\n\n## 3. Invocation contract\n\nOnce you've picked a backend skill, **invoke that skill with `MODEL_PATH` set**. Do NOT inline the backend's commands here — each backend has its own pitfalls (mandatory flags, env-var pins, install order) that the dedicated skill handles. The user explicitly does NOT want you to \"improvise\" — read the picked sub-skill in full first.\n\n## 4. Sanity check after deploy\n\nWhichever backend you pick, after launch run this universal sanity check:\n\n```bash\n# Replace localhost:PORT with the backend's actual port (default in each sub-skill)\ncurl http://localhost:PORT/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: HTTP 200 with `choices[0].message.content` containing `\"2\"`.\n\n`minicpm5-deploy-litert` also serves this endpoint: after `litert-lm import … minicpm5-2b`, `litert-lm serve` listens on port 9379 ([guide](https://developers.google.com/edge/litert-lm/cli/openai_server)); use `\"model\": \"minicpm5-2b\"`, add `\"reasoning_effort\": \"none\"` for a direct answer, and cap with `max_completion_tokens`.\n\n## 5. Known cross-backend pitfalls\n\nThese are common to multiple backends — surface to the user up front:\n\n- **Think vs no-think**: MiniCPM5-2B uses Think mode with `temperature=1.0, top_p=0.95`. MiniCPM5-1B also supports No-think with `enable_thinking=false` + `temperature=0.7, top_p=0.95`.\n- **128 K context**: `max_position_embeddings=131072`, `rope_theta=5e6`, **no rope-scaling**. Pass `--max-model-len 131072` (vLLM) / `--context-length 131072` (SGLang) / `-c 131072` (llama.cpp) to use the full window. Lower if VRAM is tight.\n- **Untied lm_head**: `tie_word_embeddings=false`. Tools that assume the Llama tied default (e.g. `mlx_lm.convert` < 0.31) will silently drop `lm_head` → output collapses to random tokens. The MLX skill bakes in the fix.\n\n## 6. Don't reinvent: link to the cookbook\n\nEach sub-skill is paired with a one-page cookbook in [`docs/deployment/`](../../docs/deployment/). The skill is the machine-readable shortcut; the cookbook is the human-readable reference. Both are kept in sync.\n","skills/minicpm5-finetune-llamafactory/SKILL.md":"# minicpm5-finetune-llamafactory\n\nPath: `skills/minicpm5-finetune-llamafactory/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-llamafactory/SKILL.md)","skills/minicpm5-finetune-ms-swift/SKILL.md":"# minicpm5-finetune-ms-swift\n\nPath: `skills/minicpm5-finetune-ms-swift/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-ms-swift/SKILL.md)","skills/minicpm5-finetune-trl/SKILL.md":"# minicpm5-finetune-trl\n\nPath: `skills/minicpm5-finetune-trl/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-trl/SKILL.md)","skills/minicpm5-finetune-unsloth/SKILL.md":"# minicpm5-finetune-unsloth\n\nPath: `skills/minicpm5-finetune-unsloth/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-unsloth/SKILL.md)","skills/minicpm5-finetune-xtuner/SKILL.md":"# minicpm5-finetune-xtuner\n\nPath: `skills/minicpm5-finetune-xtuner/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-xtuner/SKILL.md)","skills/minicpm5-finetune/SKILL.md":"# minicpm5-finetune\n\nPath: `skills/minicpm5-finetune/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune/SKILL.md)"},"files":{"skills/minicpm5-deploy-ollama/SKILL.md":"---\nname: minicpm5-deploy-ollama\ndescription: Run MiniCPM5-1B or MiniCPM5-2B via Ollama on macOS / Linux laptop using the released GGUF. Use when the user wants \"ollama run\", \"ollama pull\", a Modelfile-driven setup, or one-line laptop deployment.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with Ollama\n\nOne-binary, no-Python laptop deployment. Consumes the released GGUF.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install Ollama (once)\n\n```bash\nbrew install ollama                                # macOS\n# or:\ncurl -fsSL https://ollama.com/install.sh | sh      # Linux\n\nOLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve &\n```\n\n### 2. Download the GGUF + write Modelfile\n\n```bash\nmkdir -p ~/${MODEL_NAME} && cd ~/${MODEL_NAME}\n\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n\ncat > Modelfile <<EOF\nFROM ./MiniCPM5-2B-${QUANT}.gguf\n\n# MiniCPM5 basic chat template\nTEMPLATE \"\"\"{{- if .Messages -}}\n{{- range .Messages -}}\n<|im_start|>{{ .Role }}\n{{ .Content }}<|im_end|>\n{{ end -}}\n<|im_start|>assistant\n{{ end -}}\"\"\"\n\nPARAMETER stop \"<|im_end|>\"\nPARAMETER stop \"</s>\"\n\n# Defaults tuned for think mode\nPARAMETER temperature 1.0\nPARAMETER top_p 0.95\nPARAMETER num_ctx 8192\nEOF\n```\n\n### 3. Create + run\n\n```bash\nollama create ${MODEL_NAME} -f Modelfile\nollama run ${MODEL_NAME}\n```\n\n### 4. Validate via OpenAI-compatible API\n\n```bash\ncurl http://localhost:11434/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply. \n\n## MiniCPM5-1B Think vs No-think\n\nA MiniCPM5-1B Modelfile configured with `temperature=0.7, top_p=0.95` defaults to **no-think**. To switch a single conversation to **think** mode, override the sampling params at request time:\n\n```bash\nollama run minicpm5-1b --temperature 0.9 --top-p 0.95\n```\n\nOr bake it into a separate model tag by raising the temperature line (top_p stays 0.95):\n\n```Modelfile\nPARAMETER temperature 0.9\nPARAMETER top_p 0.95\n```\n\nThen `ollama create minicpm5-1b-think -f Modelfile.think`.\n\nOllama 0.24 does **not** directly evaluate the GGUF-embedded Jinja chat template. It maps recognized Jinja templates to built-in Go templates, so MiniCPM5 uses the explicit Go `TEMPLATE` block above. To enter the MiniCPM5-2B think path, use raw mode and prepend `<think>\\n` manually:\n\n```bash\ncurl http://localhost:11434/api/generate -d '{\n    \"model\": \"minicpm5-2b\",\n    \"raw\": true,\n    \"prompt\": \"<|im_start|>user\\n鸡兔同笼…<|im_end|>\\n<|im_start|>assistant\\n<think>\\n\",\n    \"options\": {\"temperature\": 1.0, \"top_p\": 0.95}\n}'\n```\n\n## Common pitfalls\n\n- **`Error: invalid file magic`**: corrupted download. Re-run `huggingface-cli download`.\n\n## When NOT to use\n\n- Highest throughput on Mac → `minicpm5-deploy-mlx` (Q4 build)\n- GUI experience → `minicpm5-deploy-lmstudio`\n- NVIDIA GPU production → `minicpm5-deploy-vllm`\n\n## Reference\n\n[`docs/deployment/ollama.md`](../../docs/deployment/ollama.md)\n","skills/minicpm5-deploy-litert/SKILL.md":"---\nname: minicpm5-deploy-litert\ndescription: Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime — the litert-lm CLI or its OpenAI-compatible server on a desktop, the Kotlin API or the AI Edge Gallery app on Android, the same .litertlm bundle on CPU or GPU. Use when the user says \"LiteRT\", \"LiteRT-LM\", \"litertlm\", \".litertlm\", \"Android\", \"Edge Gallery\", \"on-device app\", or wants one bundle for phones and desktops.\n---\n\n# Deploy MiniCPM5-2B / MiniCPM5-1B with LiteRT-LM (Android / iOS / desktop / IoT)\n\n[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM), Google's on-device runtime built on [LiteRT](https://github.com/google-ai-edge/litert) (formerly TensorFlow Lite). One `.litertlm` bundle, CPU or GPU, through a CLI, an OpenAI-compatible server, Python, Kotlin and Swift. The bundles are hosted in `litert-community`. Human-readable reference: [`docs/deployment/litert.md`](../../docs/deployment/litert.md).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `LITERT_REPO` | `litert-community/MiniCPM5-2B` or `litert-community/MiniCPM5-1B` | `litert-community/MiniCPM5-2B` |\n| `FILE` | `MiniCPM5-2B_int4.litertlm` (1.55 GB, the phone file) or `MiniCPM5-2B_int8.litertlm` (2.60 GB, reasoning that has to finish); 1B: `minicpm_wi4b32_wi8_afp32.litertlm` (CPU) or `minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm` (GPU) | `MiniCPM5-2B_int4.litertlm` |\n| `BACKEND` | `cpu` or `gpu` | `cpu` |\n| `THINKING` | `true` (reason first) or `false` (direct answer) | `false` |\n| `PROMPT` | `1+1=?` | `1+1=?` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\nuv tool install litert-lm\n```\n\n### 2A. Run a pre-converted bundle (desktop CLI)\n\n```bash\nlitert-lm run --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" \\\n    --backend \"${BACKEND}\" --thinking \"${THINKING:-false}\" --prompt \"${PROMPT}\"\n```\n\nThe first run downloads the file into `~/.litert-lm/cache/huggingface/<repo>/`; later runs skip the download. With thinking on, the reasoning prints between `[thought]` and `[/thought]` and the answer follows; `--thinking-budget 2048` caps the reasoning (4096 for math); `THINKING=true` makes the model reason first. Sampling as OpenBMB recommends: `--top-k 40 --top-p 0.95 --temperature 1.0` (the CLI's default top-k is 1 = greedy, so temperature alone changes nothing).\n\n### 2B. Android\n\n- **No code**: the [AI Edge Gallery](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery) app, Model manager → **+** → **Import from HF** → paste the file's Hugging Face link.\n- **Your own app**: `implementation(\"com.google.ai.edge.litertlm:litertlm-android:0.17.0\")` from Google Maven, the `libOpenCL.so` `<uses-native-library>` entries in the manifest for the GPU, then:\n\n```kotlin\nval engine = Engine(EngineConfig(modelPath = path, backend = Backend.GPU(), cacheDir = context.cacheDir.absolutePath))\nengine.initialize()\nengine.createConversation(ConversationConfig(maxOutputToken = 1024)).use { conversation ->\n    val reply = conversation.sendMessage(\"1+1=?\")\n    println(reply); println(reply.channels[\"thought\"])\n}\n```\n\n`ConversationConfig(thinkingConfig = ThinkingConfig(enableThinking = false))` turns thinking off. Full snippet, manifest and the Galaxy S26 check: the cookbook.\n\n### 2C. OpenAI-compatible server (desktop)\n\n```bash\nlitert-lm import --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" minicpm5-2b\nlitert-lm serve --host 127.0.0.1 --port 9379\ncurl http://127.0.0.1:9379/v1/chat/completions -H \"Content-Type: application/json\" \\\n    -d '{\"model\": \"minicpm5-2b\", \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}], \"reasoning_effort\": \"none\"}'\n```\n\n`reasoning_effort` `none` = direct answer, any other value = thinking on; the cap field is `max_completion_tokens`; `top_k` is accepted alongside `temperature` / `top_p` / `seed`; `\"stream\": true` streams ([server guide](https://developers.google.com/edge/litert-lm/cli/openai_server)).\n\n### 3. Validate\n\nThe reply contains `2` for `1+1=?`: with `--thinking false` it is the whole reply; with thinking on it follows `[/thought]`. Through the server (2C), the same check is HTTP 200 with `choices[0].message.content` containing `2`, and `GET /v1/models` lists `minicpm5-2b`.\n\n## Common pitfalls\n\n- **Thinking on the int4 file** runs long chains: pass `--thinking false` for direct answers, or use the int8 file when the reasoning has to complete.\n- **int8 on iOS needs the increased-memory-limit entitlement** (2.33 GB weight section; the default entitlements do not map it). int4's 1.28 GB main section is under the limit (checked on iPhone 17 Pro, GPU and CPU).\n- **First run per backend is slow**: an XNNPACK weight cache is written beside the file on the CPU, kernels are compiled on the GPU (`--cache no` to skip the disk cache).\n- **Android GPU without the manifest entries**: `initialize()` succeeds, the first message fails with `Can not find OpenCL library on this device`.\n- **Temperature without top-k does nothing**: pass `--top-k` above 1 together with `--top-p` / `--temperature` (the default is greedy).\n- **Scripts**: the CLI reads stdin; redirect it (`< /dev/null`) when running from a job with an open pipe.\n\n## Reference\n\n[`docs/deployment/litert.md`](../../docs/deployment/litert.md)\n","skills/minicpm5-deploy-mlx/SKILL.md":"---\nname: minicpm5-deploy-mlx\ndescription: Run MiniCPM5-1B or MiniCPM5-2B natively on Apple Silicon with Apple's MLX framework. Use when the user has an Apple Silicon Mac and asks for \"MLX\", \"mlx_lm\", \"mlx_lm.convert\", \"mlx_lm.generate\", or wants the fastest path on Apple Silicon.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with MLX (Apple Silicon)\n\nApple's on-device tensor framework. Highest throughput on M-series. Stays inside one Python process — no separate server, no `llama.cpp` build chain.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MLX_REPO` | `openbmb/MiniCPM5-2B-MLX` (pre-converted 4-bit affine) | required; `openbmb/MiniCPM5-1B-MLX` also works |\n| OR `HF_REPO` + `QUANT` | `openbmb/MiniCPM5-2B`, `4bit` or `bf16` | for local conversion |\n| `MAX_TOKENS` | `200` | `200` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"mlx-lm>=0.31\" \"gguf\"\n```\n\n### 2A. Use a pre-converted MLX repo (recommended when available)\n\n```bash\nmlx_lm.generate --model \"${MLX_REPO}\" \\\n    --prompt \"<|im_start|>user\n1+1=?<|im_end|>\n<|im_start|>assistant\n\" \\\n    --max-tokens ${MAX_TOKENS} --temp 1.0 --top-p 0.95\n```\n\n### 2B. Convert from a HF checkpoint locally (advanced)\n\nUse `mlx_lm.convert` only if you have a self-trained HF fp16 checkpoint:\n\n```bash\nHF=/path/to/your-fp16-hf\n\n# Convert: bf16\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-bf16\n\n# Convert: 4-bit (smaller / faster)\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-q4 -q --q-bits 4\n```\n\nThen run as in 2A.\n\n### 3. Validate\n\nThe reply should contain `\"2\"` for `1+1=?`.\n\n## OpenAI-compatible server (mlx-lm)\n\n```bash\nmlx_lm.server --model \"${MLX_REPO}\" --host 127.0.0.1 --port 8000\n\ncurl http://127.0.0.1:8000/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"default\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\n## Common pitfalls\n\n- **Slow first generate**: MLX JIT-compiles kernels on first call (~5-10 s); subsequent calls hit the warm cache.\n- **Model runs past `<|im_end|>`**: only happens on `mlx-lm < 0.31` (older versions ignore multi-id `eos_token_id` lists). Upgrade, or pass `--extra-eos-token \"<|im_end|>\"` as a manual override — `<|im_end|>` is token id 130073 and is already listed in `generation_config.json` on 0.31+.\n\n## When NOT to use\n\n- Not on Apple Silicon → `minicpm5-deploy-llama-cpp` (CPU/CUDA) or `minicpm5-deploy-vllm` (CUDA)\n- Want a desktop GUI → `minicpm5-deploy-lmstudio` (LM Studio bundles an MLX runtime)\n- Want one-line CLI run → `minicpm5-deploy-ollama`\n\n## Reference\n\n[`docs/deployment/mlx.md`](../../docs/deployment/mlx.md)\n","skills/minicpm5-deploy-lmstudio/SKILL.md":"---\nname: minicpm5-deploy-lmstudio\ndescription: Run MiniCPM5-1B or MiniCPM5-2B in LM Studio (desktop GUI) using either the GGUF runtime (cross-platform) or the MLX runtime (Apple Silicon, faster). Includes OpenAI-compatible local server. Use when the user mentions \"LM Studio\", desktop GUI inference, \"lms\" CLI, or wants a no-code chat UI for MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with LM Studio\n\nDesktop GUI + OpenAI-compatible local server. On Apple Silicon ships **two runtimes**:\n\n| Runtime | Format | When to use |\n| --- | --- | --- |\n| **GGUF** (llama.cpp engine) | F16 / Q8_0 / Q4_K_M | cross-platform, same artifact as Ollama; Q4_K_M build |\n| **MLX** (Apple Silicon only) | bf16 / 4-bit | ~60 % faster, automatic think/answer split via `reasoning_content`; Q4 build |\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| Runtime | `gguf` or `mlx` | `mlx` on Apple Silicon, `gguf` elsewhere |\n| `QUANT` | `Q4_K_M` (GGUF) or `4bit` (MLX) | `Q4_K_M` / `4bit` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install LM Studio + complete onboarding\n\n```bash\nbrew install --cask lm-studio\nopen -a \"LM Studio\"     # accept EULA + pick model source\n```\n\n> ⚠️ The first launch MUST be GUI — `lms` (CLI) refuses with `Cannot find LM Studio installation` until LM Studio has run interactively at least once.\n\n### 2A. GGUF runtime path\n\n```bash\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-${QUANT}.gguf \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF/\n\nLMS=\"/Applications/LM Studio.app/Contents/Resources/app/.webpack/lms\"\n\"$LMS\" server start                              # binds 127.0.0.1:1234\n\"$LMS\" load minicpm5-2b --gpu max --context-length 8192 -y\n\"$LMS\" ps                                        # verify the model is loaded\n```\n\n### 2B. MLX runtime path (Apple Silicon, recommended on Mac)\n\nThe MLX runtime needs an MLX-format checkpoint. The published MiniCPM5-2B MLX repo is [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) (4-bit affine); there is no separate `-bf16` / `-4bit` variant. Either drop that one in as-is, or convert locally from `openbmb/MiniCPM5-2B` (see `minicpm5-deploy-mlx`). Then:\n\n```bash\n# Option A — use the official pre-quantized repo\nhuggingface-cli download openbmb/MiniCPM5-2B-MLX \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX\n\n# Option B — drop a locally converted directory in\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}\ncp -r ./minicpm5-mlx-${QUANT}/* ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}/\n\n\"$LMS\" server start\n\"$LMS\" load minicpm5-2b-mlx${QUANT:+-${QUANT}} --gpu max -y\n```\n\n### 3. Validate\n\n```bash\ncurl http://127.0.0.1:1234/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nFor the MLX runtime, a think prompt produces output split into `message.reasoning_content` (the `<think>` block) and `message.content` (the final answer) automatically — that's an MLX-runtime feature, not a model setting.\n\n## Think vs nothink (MiniCPM5-1B)\n\nLM Studio 0.4.13's chat-completion endpoint does **not** propagate `chat_template_kwargs.enable_thinking` to the GGUF runtime. Instead:\n\n- **Default = think mode** for both runtimes.\n- **For nothink** with the GGUF runtime, prepend the closing think block manually:\n  ```json\n  \"messages\": [\n    {\"role\":\"user\",\"content\":\"1+1=?\"},\n    {\"role\":\"assistant\",\"content\":\"<think>\\n\\n</think>\\n\\n\"}\n  ]\n  ```\n  and the model continues from there.\n- **MLX runtime**: think/answer are auto-split, you don't need to do anything.\n\n## Common pitfalls\n\n- **MLX runtime not available**: only on Apple Silicon. On Intel Mac / Windows / Linux LM Studio, only the GGUF runtime works.\n\n## When NOT to use\n\n- Just want CLI / scripted runs → `minicpm5-deploy-ollama` is leaner\n- Production server → `minicpm5-deploy-vllm`\n- No GUI desired → `minicpm5-deploy-llama-cpp` (`llama-server`)\n\n## Reference\n\n[`docs/deployment/lmstudio.md`](../../docs/deployment/lmstudio.md)\n","skills/minicpm5-deploy-llama-cpp/SKILL.md":"---\nname: minicpm5-deploy-llama-cpp\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for \"llama.cpp\", \"llama-cli\", \"llama-server\", \"GGUF\", or has no Python available.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with llama.cpp\n\nCPU / edge / consumer-GPU deployment via the released GGUF artifacts. The artifacts work directly with vanilla `llama.cpp` and every downstream runtime (Ollama / LM Studio / `llama-cpp-python`).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `NGL` | `99` (all layers on GPU) / `0` (CPU only) | `99` if NVIDIA GPU, else `0` |\n| `CTX` | `8192` (default) up to `131072` (128 K) | `8192` |\n\n## Steps\n\n### 1. Install llama.cpp\n\n```bash\n# macOS\nbrew install llama.cpp\n\n# Linux / cross-platform: pre-built binary\ncurl -fsSL https://github.com/ggerganov/llama.cpp/releases/latest/download/llama-cli-linux.tar.gz | tar -xz\n# OR build from source:\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git && cd llama.cpp\nmkdir build && cd build\ncmake .. -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release    # CPU-only: omit GGML_CUDA=ON\ncmake --build . --config Release -j $(nproc) --target llama-cli llama-server\n```\n\n### 2. Download the GGUF\n\n```bash\nmkdir -p ~/minicpm5 && cd ~/minicpm5\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n```\n\n### 3a. Interactive chat (CLI)\n\n```bash\nllama-cli -m MiniCPM5-2B-${QUANT}.gguf \\\n    -n 2048 --temp 1.0 --top-p 0.95 --min-p 0.0 -ngl ${NGL} -c ${CTX}\n```\n\n### 3b. OpenAI-compatible HTTP server\n\n```bash\nllama-server -m MiniCPM5-2B-${QUANT}.gguf \\\n    --port 8080 -ngl ${NGL} -c ${CTX} --jinja\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:8080/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"min_p\": 0.0, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nIn llama.cpp, the default `min_p=0.05` can lead to repetitive output: it filters out tokens whose probability is below 5% of the highest-probability token, potentially discarding the exact tokens needed to break out of a repetition loop. To prevent this, we set `min_p=0.0`.\n\n## Sampling defaults\n\n| Mode | `--temp` | `--top-p` | `--min-p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | 1.0 | 0.95 | 0.0 |\n| MiniCPM5-1B Think | 0.9 | 0.95 | 0.0 |\n| MiniCPM5-1B No-think | 0.7 | 0.95 | 0.0 |\n\n## Choosing a quant\n\n| Quant | Disk | Quality |\n| --- | --- | --- |\n| F16 | 5.04 GB | reference |\n| Q8_0 | 2.68 GB | ~indistinguishable from F16 |\n| **Q4_K_M** | **1.56 GB** | small drop, ideal for laptops |\n\n## Common pitfalls\n\n- **Slow on CPU + large context**: drop `-c 131072` to `-c 8192` if you don't need 128 K.\n\n## Building your own GGUF (advanced)\n\nIf you've trained your own MiniCPM5-1B or MiniCPM5-2B variant, build a GGUF with:\n\n```bash\npython convert_hf_to_gguf.py /path/to/your-fp16-hf --outfile out/F16.gguf --outtype f16\nllama-quantize out/F16.gguf out/Q4_K_M.gguf Q4_K_M\n```\n\nTrained a **LoRA adapter** (not a full model) and want to apply it at runtime with `--lora` instead of baking it in? Convert it to a GGUF adapter — see **`minicpm5-finetune-gguf-lora`**.\n\n## When NOT to use\n\n- NVIDIA GPU + want OpenAI-compatible serving → `minicpm5-deploy-vllm`\n- Apple Silicon native → `minicpm5-deploy-mlx` is faster\n- Just want one-line desktop run → `minicpm5-deploy-ollama`\n- Want a desktop GUI → `minicpm5-deploy-lmstudio`\n\n## Reference\n\n[`docs/deployment/llama_cpp.md`](../../docs/deployment/llama_cpp.md)\n","skills/minicpm5-deploy-arclight/SKILL.md":"# Deploy ArcLight from Source (CPU)\n\nArcLight is a lightweight C/C++ LLM inference framework for unified-memory systems. The recommended path is to build from source, then run a GGUF model with `al-gen`, `al-chat`, or `al-ppl`.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL` | `/path/to/MiniCPM5-2B-Q4_0.gguf` | required |\n| `PROMPT` | `\"Hello!\"` | `\"Hello!\"` |\n| `THREADS` | `4` | choose for the target CPU |\n| `NUMA_MODE` | `none` or `tp` | `none` for first run |\n| `NODES` | `1`, `2`, `4` | `1` with `NUMA_MODE=none` |\n| `MAX_GEN` | `256` | `256` |\n\n## Steps\n\n### 1. Build from source\n\n```bash\ngit clone https://github.com/OpenBMB/ArcLight.git\ncd ArcLight\n\ncmake -B build -DARCLIGHT_BACKEND=AUTO\ncmake --build build --config Release -j 32\n```\n\nUse `ARCLIGHT_BACKEND=AUTO` by default. Set it explicitly only when needed:\n\n- `X86`: force the x86 backend\n- `NEON`: force the ARM NEON backend\n- `NONE`: build without architecture-specific backend code\n\n### 2. Prepare a GGUF model\n\nArcLight uses GGUF checkpoints from `llama.cpp`. The `nnml` backend only loads `f32 / f16 / q4_0 / q8_0 / q6_K / q8_K` tensor types — **`Q4_K_M` is not supported**.\n\nSupported model families: MiniCPM5-2B, Qwen3, Llama2.\n\nFor first validation use the released `Q8_0` (`openbmb/MiniCPM5-2B-GGUF`), or quantize an unreleased `Q4_0` yourself from the F16:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-F16.gguf --local-dir .\nllama-quantize ./MiniCPM5-2B-F16.gguf ./MiniCPM5-2B-Q4_0.gguf Q4_0\n```\n\n### 3A. One-shot generation\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\n### 3B. Interactive chat\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\nTo seed the first turn:\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n### 3C. Perplexity\n\n```bash\n./build/al-ppl \\\n    --model \"${MODEL}\" \\\n    --prompt \"Good morning, Miss Lee!\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n## Many-core CPU / NUMA\n\nStart with single-node mode for correctness:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\nUse cross-NUMA tensor parallelism on many-core machines:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa tp --nodes ${NODES} \\\n    --threads ${THREADS}\n```\n\nRules:\n\n- `--numa none` requires `--nodes 1`.\n- `--numa tp` requires `--nodes N` where `N > 1`.\n- In the current version, `NODES` should be a power of 2.\n- Choose `THREADS` so it can be evenly divided across `NODES`.\n- `--numa pp` is reserved for future pipeline parallelism and is not implemented.\n\n## Optional memory buffers\n\nIf allocation fails or the model is larger, pass manual buffer sizes:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --w_gb 4 --a_gb 8 --kv_gb 2 --work_gb 2\n```\n\nMeaning:\n\n- `--w_gb`: weight buffer size\n- `--a_gb`: activation buffer size\n- `--kv_gb`: KV cache buffer size\n- `--work_gb`: temporary workspace size\n\nIncrease `--kv_gb` for longer `--max_length`. Increase `--w_gb` for larger models.\n\n## Validate\n\nRun:\n\n```bash\nMODEL=/path/to/MiniCPM5-2B-Q4_0.gguf\nTHREADS=4\n\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"1+1=?\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_gen 64\n```\n\nThe reply should contain `2` or a short explanation that evaluates to `2`.\n\n## Common pitfalls\n\n- **Program aborts in single-node mode**: use `--numa none --nodes 1`.\n- **Tensor parallel mode fails**: use `--numa tp --nodes N` with `N > 1`; in this version, `N` should be a power of 2.\n- **Pipeline parallelism fails**: `--numa pp` is not implemented yet.\n- **Model loading fails**: verify the checkpoint is GGUF and from a supported model family.\n- **Out of memory**: increase `--w_gb`, `--a_gb`, `--kv_gb`, or `--work_gb`; longer contexts usually need a larger KV cache.\n- **Poor CPU throughput**: check `--threads`, NUMA layout, and thread-core bindings; use `--print_binding 1 --print_perf 1` for diagnostics.\n\n## When NOT to use\n\n- Need CUDA server inference -> use a GPU-oriented runtime instead.\n- Need Apple Silicon MLX inference -> use an MLX deployment path instead.\n- Need a desktop GUI -> use a GUI runtime that supports GGUF models.\n- Need pipeline parallelism -> wait for ArcLight `--numa pp` support.\n\n## Reference\n\n- [`Arclight`](https://github.com/OpenBMB/ArcLight)\n- [llama.cpp GGUF documentation](https://github.com/ggml-org/llama.cpp)\n","skills/minicpm5-deploy-sglang/SKILL.md":"---\nname: minicpm5-deploy-sglang\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for \"SGLang\", \"RadixAttention\", \"prefix cache\", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with SGLang\n\nOpenAI-compatible server with RadixAttention prefix cache. Best fit for **tool calling**, batched eval pipelines, and high-concurrency serving.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B`| required; |\n| `PORT` | `30000` | `30000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TOOL_PARSER` | `minicpm5` | `minicpm5`; use `auto` if you want template detection |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"sglang[srt]>=0.5.16\"          # latest, requires CUDA 13.x driver\n# pip install \"sglang==0.5.6.post3\"        # fallback for CUDA 12.x driver hosts\n```\n\n**For tool calling, install from `main`** — the `minicpm5` parser ([PR #25600](https://github.com/sgl-project/sglang/pull/25600), merged 2026-05-22) is not in any pip release yet (`v0.5.12.post1` was branched earlier). Plain chat works on the pip release; only `--tool-call-parser minicpm5` needs `main`:\n\n```bash\npip install \"git+https://github.com/sgl-project/sglang.git@main#subdirectory=python\"\n```\n\n### 2. Recommended runtime env vars\n\n```bash\nexport VLLM_WORKER_MULTIPROC_METHOD=spawn\nexport SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1\nexport SGLANG_DISABLE_CUDNN_CHECK=1\n```\n\n### 3. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT}\n```\n\nWait for `The server is fired up and ready to roll!` .\n\n**Speculative decoding (DSpark)**: we also release [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark), a DSpark draft model trained for MiniCPM5-2B. Enable it in SGLang to accelerate decoding while keeping the target model's outputs unchanged:\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT} \\\n    --speculative-algorithm DSPARK \\\n    --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \\\n    --speculative-dspark-block-size 7\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains `\"2\"`.\n\n## Tool calling\n\nSGLang includes a native MiniCPM5 XML tool-call parser. Keep `TOOL_PARSER=minicpm5` (or `auto`) when launching, then send an OpenAI-style tools request:\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n        \"tools\": [{\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get current weather for a city\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\"city\": {\"type\": \"string\"}},\n                    \"required\": [\"city\"]\n                }\n            }\n        }],\n        \"tool_choice\": \"auto\",\n        \"temperature\": 1.0, \"top_p\": 0.95,\n        \"max_tokens\": 256\n    }'\n```\n\n## Offline / batched (Engine API)\n\n```python\nimport sglang as sgl\n\nllm = sgl.Engine(model_path=\"${MODEL_PATH}\", tp_size=1, mem_fraction_static=0.8, context_length=131072)\n\noutputs = llm.generate(\n    [\"用一句话解释什么是 GQA。\"],\n    sampling_params={\"temperature\": 1.0, \"top_p\": 0.95, \"max_new_tokens\": 1024,\n                     \"skip_special_tokens\": False},\n)\nprint(outputs)\n```\n\n## Common pitfalls\n\n- **`GLIBCXX_3.4.31 not found`** (only if you hit it — not universal): conda Python ships an older `libstdc++` than the SGLang wheel was built against. Force-load the system one: `LD_PRELOAD=/lib/x86_64-linux-gnu/libstdc++.so.6 python -m sglang.launch_server ...`\n\n## When NOT to use\n\n- Production serving without batch-eval needs → `minicpm5-deploy-vllm` is simpler\n- One-shot Python → `minicpm5-deploy-transformers`\n- No NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n\n## Reference\n\n[`docs/deployment/sglang.md`](../../docs/deployment/sglang.md)\n","skills/minicpm5-deploy-transformers/SKILL.md":"---\nname: minicpm5-deploy-transformers\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with Hugging Face Transformers for one-shot Python generation on GPU (bfloat16) or CPU (float32). Use when the user wants a quick Python script, no server, no extra deps, or asks for \"Transformers\", \"AutoModelForCausalLM\", \"model.generate\" with MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with HF Transformers\n\nOne-shot Python generation. No server. Works on a single GPU (bfloat16) or CPU only (fp32).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or local dir | required; `openbmb/MiniCPM5-1B` also works |\n| `MODE` | `think` or `nothink` (`nothink` is 1B-only) | `think` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install -U \"transformers>=5.6,<6\" \"torch>=2.11\" accelerate     # latest (CUDA 13.x driver hosts)\n# pip install -U \"transformers==4.57.3\" \"torch==2.7.1\" accelerate  # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Run\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_path = \"${MODEL_PATH}\"      # ← replace\ntok = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_path,\n    torch_dtype=torch.bfloat16,    # CPU users: torch.float32 + device_map=\"cpu\"\n    device_map=\"auto\",\n).eval()\n\nmessages = [{\"role\": \"user\", \"content\": \"用一句话解释什么是 GQA。\"}]\ninputs = tok.apply_chat_template(\n    messages,\n    add_generation_prompt=True,\n    enable_thinking=True,\n    return_tensors=\"pt\",\n    return_dict=True,\n).to(model.device)\n\nwith torch.no_grad():\n    out = model.generate(\n        **inputs,\n        max_new_tokens=1024,\n        do_sample=True,\n        temperature=1.0,\n        top_p=0.95,\n    )\n\nprompt_len = inputs[\"input_ids\"].shape[-1]\nprint(tok.decode(out[0][prompt_len:], skip_special_tokens=True))\n```\n\nFor CPU only: change `torch_dtype=torch.float32, device_map=\"cpu\"`. Keep `enable_thinking=True` and `temperature=1.0` for MiniCPM5-2B; use `enable_thinking=False` and `temperature=0.7` only for MiniCPM5-1B No-think mode.\n\n## Sampling defaults\n\n| Mode | `enable_thinking` | `temperature` | `top_p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | `True` | 1.0 | 0.95 |\n| MiniCPM5-1B Think | `True` | 0.9 | 0.95 |\n| MiniCPM5-1B No-think | `False` | 0.7 | 0.95 |\n\n## Validate\n\nA coherent answer to `1+1=?` (e.g. `\"2\"` or `\"答案是 2\"`).\n\n## LoRA inference\n\n```python\nfrom peft import PeftModel\nbase = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=\"auto\")\nmodel = PeftModel.from_pretrained(base, \"/path/to/adapter\").eval()\n```\n\nAdapters from any of the `minicpm5-finetune-*` skills load directly with no surgery.\n\n## When NOT to use\n\n- Need an OpenAI-compatible HTTP server → `minicpm5-deploy-vllm` or `minicpm5-deploy-sglang`\n- Apple Silicon → `minicpm5-deploy-mlx` is faster\n- CPU-only or low-VRAM laptop → `minicpm5-deploy-llama-cpp` with Q4_K_M is faster\n\n## Reference\n\n[`docs/deployment/transformers.md`](../../docs/deployment/transformers.md)\n","skills/minicpm5-finetune-gguf-lora/SKILL.md":"---\nname: minicpm5-finetune-gguf-lora\ndescription: Fine-tune MiniCPM5-1B into a LoRA adapter and convert it to a GGUF adapter that loads directly into llama.cpp / llama-server and the MiniCPM Desk Pet app's custom-LoRA upload. Use when the user wants \"GGUF LoRA\", \"convert LoRA to GGUF\", \"convert_lora_to_gguf\", a custom persona/skin for the desktop pet, \"桌宠自定义 LoRA\", \"上传 LoRA 到桌宠\", or asks how to take a trained adapter and run it on a GGUF base.\n---\n\n# Fine-tune MiniCPM5-1B → GGUF LoRA adapter\n\nThe framework skills (`minicpm5-finetune-*`) all emit a **PEFT adapter** (`adapter_model.safetensors` + `adapter_config.json`). But `llama.cpp` / `llama-server` — and the **MiniCPM Desk Pet** app's custom-LoRA upload — load a **GGUF LoRA adapter** (`--lora some-adapter.gguf`). This skill is the bridge: train a PEFT LoRA, then convert it to GGUF and run/upload it.\n\n> This is the path you want when the base model is served as GGUF (Desk Pet, Ollama, LM Studio, plain `llama-server`). If you serve the fp16 HF base with vLLM / transformers, you don't need GGUF — load the PEFT adapter directly.\n\n## Pipeline overview\n\n```\n  train (any minicpm5-finetune-* skill)        this skill\n ┌────────────────────────────────────┐   ┌──────────────────────────────────┐\n BASE (fp16 HF) ─► adapter_model.safetensors ─► convert_lora_to_gguf.py ─► adapter.gguf\n                   adapter_config.json                                          │\n                                                                  llama-server --lora / Desk Pet upload\n```\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `ADAPTER_DIR` | `./runs/minicpm5_unsloth/adapter_final` | required — a PEFT dir with `adapter_config.json` + `adapter_model.safetensors` |\n| `BASE_MODEL` | `openbmb/MiniCPM5-1B` (HF id) **or** a local fp16 HF dir | required — must be the **same base** the adapter was trained on |\n| `OUTTYPE` | `f16` (recommended) / `q8_0` / `bf16` / `f32` | `f16` |\n| `OUT_GGUF` | `./minicpm5-mylora.gguf` | `<ADAPTER_DIR>/adapter_model.f16.gguf` |\n\n> **Don't have an adapter yet?** First run a training skill — start from the router **`minicpm5-finetune`** (or go straight to **`minicpm5-finetune-unsloth`** for single-GPU LoRA). Come back here with `ADAPTER_DIR` pointing at its output.\n\n## Steps\n\n### 1. Get llama.cpp (has the converter)\n\n```bash\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git\ncd llama.cpp\npip install -r requirements.txt          # converter deps: torch, safetensors, gguf, transformers\n```\n\n`convert_lora_to_gguf.py` lives at the repo root. You only need the Python script for conversion; build the C++ binaries (step 4) only if you want to test locally.\n\n### 2. Convert the PEFT adapter → GGUF\n\n```bash\npython convert_lora_to_gguf.py \"$ADAPTER_DIR\" \\\n    --base \"$BASE_MODEL\" \\\n    --outtype f16 \\\n    --outfile \"$OUT_GGUF\"\n```\n\n- `--base` only needs the base model's **config** (`config.json`, `tokenizer.json`) — not the weights. A local fp16 HF dir or an HF id both work.\n- If the base is only on the Hub, use `--base-model-id openbmb/MiniCPM5-1B` instead of `--base` (it pulls just the config).\n- Output is a single `.gguf` of roughly the same size as the input `.safetensors` (a 22 MB r=16 adapter → ~22 MB GGUF).\n\n> 🔑 **The #1 gotcha — `base_model_name_or_path` points at the training machine.**\n> PEFT writes the absolute path of the base used during training into `adapter_config.json`\n> (e.g. `/user/.../MiniCPM5-models-fixed/official`). On any other machine the converter\n> can't find it. **Always pass `--base` (or `--base-model-id`) explicitly** to override it —\n> don't rely on whatever is baked into the config. The base you pass MUST match the one you\n> trained on, or the adapter math is meaningless.\n\n### 3. (Optional) Sanity-check the GGUF metadata\n\n```bash\npython -c \"import gguf,sys; r=gguf.GGUFReader('$OUT_GGUF'); print('tensors:', len(r.tensors)); print('arch:', r.get_field('general.architecture').parts[-1].tobytes().decode() if r.get_field('general.architecture') else '?')\"\n```\n\nExpect `general.type = adapter` and a nonzero tensor count. MiniCPM5-1B is a **Llama-architecture** model, so the converter treats it as `llama` — this is correct, not an error.\n\n### 4. Test with llama-server before shipping\n\nYou need a GGUF **base** model too (the adapter is applied on top of it). Grab the released base:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-1B-GGUF MiniCPM5-1B-Q8_0.gguf --local-dir .\n```\n\nThen:\n\n```bash\n# CLI\nllama-cli -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" \\\n    -p \"你好\" -n 128 --temp 0.7 --top-p 0.95\n\n# OpenAI-compatible server\nllama-server -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" --port 8080 --jinja\n```\n\nIf the persona/behavior you trained shows up, the GGUF is good. If output is identical to the base, the adapter didn't load (check the `--lora` path and that `--base` matched in step 2).\n\n> **Quant compatibility:** a GGUF LoRA built from an fp16 adapter applies fine on top of a\n> quantized base (Q8_0 / Q4_K_M). You do **not** need a separate adapter per base quant.\n\n### 5. Use it in MiniCPM Desk Pet\n\nThe Desk Pet app loads exactly this kind of file. The fp16 GGUF you just built is upload-ready:\n\n1. Open **Settings → MiniCPM → 适配器 (LoRA)**.\n2. Click **上传** (Upload), pick your `.gguf` — the app copies it into its\n   `<userData>/adapters/uploads/` directory and registers it.\n3. Give it a display name + comma-separated aliases (used so you can switch personas by\n   voice/chat, e.g. \"换成 XX\").\n4. Select it in the adapter list to activate. The sidecar reloads `llama-server` with your\n   `--lora`.\n\nConstraints the app enforces (match them or the upload is rejected):\n\n- **Must be a single `.gguf` file** (the GGUF LoRA from step 2 — *not* the `.safetensors`,\n  *not* a merged full model).\n- It is applied on top of whatever MiniCPM5-1B GGUF base the app already runs, so **train\n  against the MiniCPM5-1B base**, not some other model.\n\n## Common pitfalls\n\n- **`FileNotFoundError` / `can't load base model config`** during conversion → the\n  `adapter_config.json` base path doesn't exist locally. Fix with `--base <local-dir>` or\n  `--base-model-id openbmb/MiniCPM5-1B` (step 2 gotcha).\n- **Adapter loads but output is unchanged** → base mismatch (trained on base A, applied on\n  base B), or the persona needs its **system prompt** too — a LoRA biases style but the\n  system prompt still matters. In Desk Pet the persona's system prompt is wired to the\n  adapter; standalone `llama-cli` users must pass it themselves.\n- **`KeyError` on an unknown tensor / target module** → the adapter targeted modules the\n  converter doesn't map. Stick to the standard attention+MLP projections\n  (`q/k/v/o_proj`, `gate/up/down_proj`) when training (all `minicpm5-finetune-*` skills\n  already default to these).\n- **Uploaded `.gguf` but Desk Pet shows nothing** → you uploaded the merged full model or a\n  base GGUF by mistake. A LoRA adapter GGUF is small (tens of MB) and has\n  `general.type = adapter` (step 3).\n\n## When NOT to use\n\n- Serving the fp16 HF base with **vLLM / transformers / SGLang** → load the PEFT adapter\n  directly, skip GGUF. See `minicpm5-deploy-vllm` / `-transformers` / `-sglang`.\n- You want to **bake the LoRA into the weights** (no runtime `--lora`) → merge first\n  (`model.save_pretrained_merged(...)` or `peft ... merge_and_unload()`), then convert the\n  **merged fp16 model** with `convert_hf_to_gguf.py` (see the \"Building your own GGUF\"\n  section of `minicpm5-deploy-llama-cpp`). That produces a standalone base GGUF, not an\n  adapter.\n\n## Reference\n\n- Training frameworks: **`minicpm5-finetune`** (router) and its sub-skills.\n- Running GGUF: **`minicpm5-deploy-llama-cpp`**.\n- Converter: `convert_lora_to_gguf.py` in [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp).\n","skills/minicpm5-deploy-vllm-ascend/SKILL.md":"---\nname: minicpm5-deploy-vllm-ascend\ndescription: Deploy MiniCPM5-2B with vLLM on Huawei Ascend NPU using vLLM-Ascend. Use when the user mentions vLLM-Ascend, Ascend NPU, Huawei Ascend, CANN, torch_npu, davinci devices, or wants an OpenAI-compatible MiniCPM5 server on Ascend hardware.\n---\n\n# Deploy MiniCPM5-2B with vLLM-Ascend\n\nUse this skill for the BF16 / FP16 `openbmb/MiniCPM5-2B` checkpoint on Huawei Ascend NPU. `vllm-ascend` is installed alongside a matching vLLM release and supplies the Ascend backend.\n\n## Required input\n\n| Variable | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required |\n| `PORT` | `8000` | `8000` |\n| `ASCEND_RT_VISIBLE_DEVICES` | `0` | `0` |\n| `CTX_LEN` | `32768` | `32768`; the native maximum is `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TP_SIZE` | `1` | number of visible devices |\n\nAccept either the Hugging Face id or a local directory containing the MiniCPM5-2B model files.\n\n## Prerequisites\n\nUse a compatible Linux Ascend environment with CANN, PyTorch, and TorchNPU installed. Keep the vLLM and vLLM-Ascend versions matched. The basic serving recipe below follows the tested `vllm==0.18.0` / `vllm-ascend==0.18.0` pair.\n\nDevice visibility is controlled by `ASCEND_RT_VISIBLE_DEVICES`; `CUDA_VISIBLE_DEVICES` does not select Ascend devices.\n\n## Install\n\nThe prebuilt image is the simplest option for a compatible Ascend host. This is a minimal single-device example; adjust device nodes and driver mounts for the host hardware and image documentation.\n\n```bash\ndocker run -it --net=host --shm-size=1g \\\n    --device /dev/davinci0 --device /dev/davinci_manager \\\n    --device /dev/devmm_svm --device /dev/hisi_hdc \\\n    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \\\n    -v /usr/local/dcmi:/usr/local/dcmi \\\n    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \\\n    quay.io/ascend/vllm-ascend:v0.18.0\n```\n\nFor an existing CANN environment:\n\n```bash\npip install vllm==0.18.0 vllm-ascend==0.18.0 \\\n    --extra-index-url https://download.pytorch.org/whl/cpu\n```\n\nFor built-in MiniCPM5 tool-call parsing, use the matching `vllm==0.23.0` / `vllm-ascend==0.23.0` stack described in [Tool calling](#tool-calling). The parser responses in the cookbook were produced with the upstream parser file on an older vLLM and a compatibility shim because the 0.23.0 stack was not available on the test hardware.\n\n## Prepare the environment\n\nRun the CANN and NNAL setup scripts before starting vLLM. Change the paths if those components are installed elsewhere.\n\n```bash\nsource /usr/local/Ascend/ascend-toolkit/set_env.sh\nsource /usr/local/Ascend/nnal/atb/set_env.sh\n\nexport ASCEND_RT_VISIBLE_DEVICES=${ASCEND_RT_VISIBLE_DEVICES:-0}\nexport LD_LIBRARY_PATH=$(python3 -c \"import torch,os;print(os.path.dirname(torch.__file__))\")/lib:\\\n$(python3 -c \"import torch_npu,os;print(os.path.dirname(torch_npu.__file__))\")/lib:$LD_LIBRARY_PATH\n```\n\nCheck the driver and Python packages before loading the model:\n\n```bash\nnpu-smi info\npython3 -c \"import torch, torch_npu; print(torch_npu.npu.is_available(), torch_npu.npu.device_count())\"\n```\n\nIf `npu-smi` reports a missing driver library, add the corresponding driver `lib64` directories to `LD_LIBRARY_PATH` according to the host installation.\n\n## Launch the server\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --port ${PORT:-8000}\n```\n\nSet `TP_SIZE` to the number of devices in `ASCEND_RT_VISIBLE_DEVICES` when using tensor parallelism.\n\n## Validate chat completions\n\nMiniCPM5-2B is think-only. Use `temperature=1.0`, `top_p=0.95`, and `enable_thinking=true`:\n\n```bash\ncurl http://localhost:${PORT:-8000}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0,\n        \"top_p\": 0.95,\n        \"max_tokens\": 256,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpect HTTP 200 and an answer containing `2` in `choices[0].message.content`. Leave enough `max_tokens` for the model's reasoning text.\n\n## Tool calling\n\nMiniCPM5-2B emits XML-style tool calls. This section requires matching vLLM and vLLM-Ascend `0.23.0`; that release includes the `minicpm5` parser in the OpenAI server layer, so no parser plugin file is needed:\n\n```bash\npip install vllm==0.23.0\npip install vllm-ascend==0.23.0 \\\n    --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi\n```\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --enable-auto-tool-choice \\\n    --tool-call-parser minicpm5 \\\n    --port ${PORT:-8000}\n```\n\nWhen sending `tools`, use both server flags above and the same 2B sampling settings:\n\n```json\n{\n  \"model\": \"MiniCPM5-2B\",\n  \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n  \"tools\": [{\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_weather\",\n      \"description\": \"Get current weather for a city\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\"city\": {\"type\": \"string\"}},\n        \"required\": [\"city\"]\n      }\n    }\n  }],\n  \"tool_choice\": \"auto\",\n  \"temperature\": 1.0,\n  \"top_p\": 0.95,\n  \"max_tokens\": 512,\n  \"chat_template_kwargs\": {\"enable_thinking\": true}\n}\n```\n\n## Common failures\n\n- `torch_npu` cannot import: source the CANN/NNAL environment and check that the installed PyTorch, TorchNPU, CANN, vLLM, and vLLM-Ascend versions match the supported stack.\n- No devices are visible: check `npu-smi info`, `ASCEND_RT_VISIBLE_DEVICES`, and the Docker device/driver mounts. Do not use `CUDA_VISIBLE_DEVICES` for this backend.\n- HBM allocation or 128K startup failure: lower `--max-model-len`, then lower `--gpu-memory-utilization`.\n- Tensor-parallel initialization failure: make `--tensor-parallel-size` match the number of visible Ascend devices.\n- Tool requests fail with an auto-tool-choice error: launch with both `--enable-auto-tool-choice` and `--tool-call-parser minicpm5`.\n\n## Reference\n\nSee [`docs/deployment/vllm-ascend.md`](../../docs/deployment/vllm-ascend.md) for the human-readable cookbook and the [official vLLM-Ascend installation guide](https://docs.vllm.ai/projects/ascend/en/v0.23.0/installation.html) for hardware-specific setup.\n","skills/minicpm5-deploy-vllm/SKILL.md":"---\nname: minicpm5-deploy-vllm\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for \"vLLM\", \"OpenAI server\", \"REST API for MiniCPM5\", or \"production deployment\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with vLLM\n\nOpenAI-compatible server for the BF16 / FP16 MiniCPM5-1B or MiniCPM5-2B checkpoint.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` also works |\n| `PORT` | `8000` | `8000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072`; lower if VRAM tight |\n| `MEM_FRAC` | `0.85` | `0.85`; lower on shared GPUs |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"vllm>=0.21\"          # latest (CUDA 13.x driver hosts)\n# pip install \"vllm==0.10.1.1\"    # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} vllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN} \\\n    --gpu-memory-utilization ${MEM_FRAC} \\\n    --port ${PORT}\n```\n\nWait for `Application startup complete` in the log.\n\n### 3. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains a coherent answer.\n\n## Sampling defaults\n\n```json\n{\"temperature\": 1.0, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-2B Think\n{\"temperature\": 0.9, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-1B Think\n{\"temperature\": 0.7, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": false}}   // MiniCPM5-1B No-think\n```\n\n## Common pitfalls\n\n- **`(free / total) < MEM_FRAC` hard error**: lower `--gpu-memory-utilization` (e.g. 0.5 on a shared GPU).\n- **OOM at startup with 128 K**: drop `--max-model-len` to 32768 or 8192.\n\n## Tool calling (plugin)\n\nThe vLLM-side MiniCPM5 XML parser ([PR #43175](https://github.com/vllm-project/vllm/pull/43175)) merged to `main` on 2026-05-27 but is **not in any pip release yet** (`v0.22.0` was cut before the merge). Use the bridge plugin shipped at `tool_parsers/minicpm5xml_tool_parser.py` in this repo:\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 --max-model-len ${CTX_LEN} --port ${PORT} \\\n    --enable-auto-tool-choice \\\n    --tool-parser-plugin /path/to/MiniCPM/tool_parsers/minicpm5xml_tool_parser.py \\\n    --tool-call-parser minicpm5\n```\n\nDrop `--tool-parser-plugin` once vLLM ships a release containing the parser natively.\n\n## When NOT to use\n\n- One-shot Python script → `minicpm5-deploy-transformers`\n- Apple Silicon / no NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n- High-concurrency batch eval w/ prefix cache or tool calling → `minicpm5-deploy-sglang`\n\n## Reference\n\n[`docs/deployment/vllm.md`](../../docs/deployment/vllm.md)\n","skills/minicpm5-deploy/SKILL.md":"---\nname: minicpm5-deploy\ndescription: Pick the right inference backend for a MiniCPM5-1B or MiniCPM5-2B checkpoint and route to a backend-specific cookbook skill. Use when the user wants to deploy / serve / chat-with / benchmark a MiniCPM5 model and has not yet committed to a specific engine, or when they say \"deploy MiniCPM5\", \"run MiniCPM5\", \"serve MiniCPM5\", \"MiniCPM5 推理\", \"部署 MiniCPM5\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B — backend router\n\nYou're being asked to deploy / serve / chat-with a MiniCPM5-1B or MiniCPM5-2B checkpoint. Your job is to **pick exactly one backend skill below** based on the user's hardware, format, and goal, then **invoke that skill** rather than improvising.\n\n## 1. Required input from the user\n\nBefore picking a backend, you MUST know:\n\n| Variable | Example | Where to ask |\n| --- | --- | --- |\n| `MODEL_PATH` | HF id `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B` **or** a local path | \"Which checkpoint? HF id or local path?\" |\n| Hardware | NVIDIA GPU / Apple Silicon / CPU only | infer from context, otherwise ask |\n| Goal | \"interactive chat\" / \"OpenAI server\" / \"Python script\" / \"benchmark\" | infer from context |\n\n### Available checkpoints on Hugging Face\n\n| Variant | HF repo | Use with |\n| --- | --- | --- |\n| **HF fp16 (recommended)** | [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) or [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) | `transformers` / `vllm` (no `--quantization`) / `vllm-ascend` / `sglang` / any `minicpm5-finetune-*` |\n| GGUF F16 / Q8_0 / Q4_K_M | [`openbmb/MiniCPM5-2B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) or [`openbmb/MiniCPM5-1B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) | `minicpm5-deploy-llama-cpp` / `-ollama` / `-lmstudio` |\n| MLX (Apple Silicon) | [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) or [`openbmb/MiniCPM5-1B-MLX`](https://huggingface.co/openbmb/MiniCPM5-1B-MLX) | `minicpm5-deploy-mlx` |\n| LiteRT-LM `.litertlm` (Android / iOS / desktop / IoT, CPU + GPU) | [`litert-community/MiniCPM5-2B`](https://huggingface.co/litert-community/MiniCPM5-2B) or [`litert-community/MiniCPM5-1B`](https://huggingface.co/litert-community/MiniCPM5-1B) | `minicpm5-deploy-litert` |\n\nIf the user has a local copy, accept any directory path that contains `config.json` and `model.safetensors` (or the equivalent GGUF / MLX layout).\n\n## 2. Decision matrix — pick exactly one\n\n| User says / wants | Hardware | Format | → Skill to invoke |\n| --- | --- | --- | --- |\n| \"Quick Python script\" / \"one-shot generation\" / \"no server\" | any GPU or CPU | HF safetensors | **`minicpm5-deploy-transformers`** |\n| \"OpenAI server\" / \"production serving\" / \"high QPS\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-vllm`** |\n| \"vLLM-Ascend\" / \"Ascend NPU\" / \"CANN\" / \"torch_npu\" | Huawei Ascend NPU | HF safetensors | **`minicpm5-deploy-vllm-ascend`** |\n| \"RadixAttention\" / \"prefix cache\" / \"batched eval\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-sglang`** |\n| \"GGUF\" / \"llama.cpp\" / \"llama-cli\" / \"CPU only\" | any CPU + optional GPU | GGUF | **`minicpm5-deploy-llama-cpp`** |\n| \"Ollama\" / \"ollama run\" / \"Modelfile\" | macOS / Linux laptop | GGUF | **`minicpm5-deploy-ollama`** |\n| \"LM Studio\" / \"desktop GUI\" | macOS / Windows / Linux | GGUF or MLX | **`minicpm5-deploy-lmstudio`** |\n| \"MLX\" / \"Apple Silicon native\" / \"fastest on Mac\" | Apple Silicon | MLX | **`minicpm5-deploy-mlx`** |\n| \"Android\" / \"on-device app\" / \"Edge Gallery\" / \"LiteRT\" / \"LiteRT-LM\" / \"litertlm\" / \"Raspberry Pi\" | Android phone, iPhone, desktop or IoT board (CPU or GPU) | LiteRT-LM `.litertlm` | **`minicpm5-deploy-litert`** |\n\nIf the user **has not specified** any of the above and asks \"how do I run this?\":\n\n- **CUDA box, want fastest server**: pick `minicpm5-deploy-vllm`.\n- **Ascend NPU, want an OpenAI-compatible server**: pick `minicpm5-deploy-vllm-ascend`.\n- **CUDA box, want minimal Python**: pick `minicpm5-deploy-transformers`.\n- **Apple Silicon laptop**: pick `minicpm5-deploy-ollama` (easiest) or `minicpm5-deploy-mlx` (fastest).\n- **Android phone, or an on-device app (Android / iOS)**: pick `minicpm5-deploy-litert`.\n- **CPU only / Windows / low-VRAM**: pick `minicpm5-deploy-llama-cpp` (Q4_K_M).\n\n## 3. Invocation contract\n\nOnce you've picked a backend skill, **invoke that skill with `MODEL_PATH` set**. Do NOT inline the backend's commands here — each backend has its own pitfalls (mandatory flags, env-var pins, install order) that the dedicated skill handles. The user explicitly does NOT want you to \"improvise\" — read the picked sub-skill in full first.\n\n## 4. Sanity check after deploy\n\nWhichever backend you pick, after launch run this universal sanity check:\n\n```bash\n# Replace localhost:PORT with the backend's actual port (default in each sub-skill)\ncurl http://localhost:PORT/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: HTTP 200 with `choices[0].message.content` containing `\"2\"`.\n\n`minicpm5-deploy-litert` also serves this endpoint: after `litert-lm import … minicpm5-2b`, `litert-lm serve` listens on port 9379 ([guide](https://developers.google.com/edge/litert-lm/cli/openai_server)); use `\"model\": \"minicpm5-2b\"`, add `\"reasoning_effort\": \"none\"` for a direct answer, and cap with `max_completion_tokens`.\n\n## 5. Known cross-backend pitfalls\n\nThese are common to multiple backends — surface to the user up front:\n\n- **Think vs no-think**: MiniCPM5-2B uses Think mode with `temperature=1.0, top_p=0.95`. MiniCPM5-1B also supports No-think with `enable_thinking=false` + `temperature=0.7, top_p=0.95`.\n- **128 K context**: `max_position_embeddings=131072`, `rope_theta=5e6`, **no rope-scaling**. Pass `--max-model-len 131072` (vLLM) / `--context-length 131072` (SGLang) / `-c 131072` (llama.cpp) to use the full window. Lower if VRAM is tight.\n- **Untied lm_head**: `tie_word_embeddings=false`. Tools that assume the Llama tied default (e.g. `mlx_lm.convert` < 0.31) will silently drop `lm_head` → output collapses to random tokens. The MLX skill bakes in the fix.\n\n## 6. Don't reinvent: link to the cookbook\n\nEach sub-skill is paired with a one-page cookbook in [`docs/deployment/`](../../docs/deployment/). The skill is the machine-readable shortcut; the cookbook is the human-readable reference. Both are kept in sync.\n","skills/minicpm5-finetune-llamafactory/SKILL.md":"# minicpm5-finetune-llamafactory\n\nPath: `skills/minicpm5-finetune-llamafactory/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-llamafactory/SKILL.md)","skills/minicpm5-finetune-ms-swift/SKILL.md":"# minicpm5-finetune-ms-swift\n\nPath: `skills/minicpm5-finetune-ms-swift/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-ms-swift/SKILL.md)","skills/minicpm5-finetune-trl/SKILL.md":"# minicpm5-finetune-trl\n\nPath: `skills/minicpm5-finetune-trl/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-trl/SKILL.md)","skills/minicpm5-finetune-unsloth/SKILL.md":"# minicpm5-finetune-unsloth\n\nPath: `skills/minicpm5-finetune-unsloth/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-unsloth/SKILL.md)","skills/minicpm5-finetune-xtuner/SKILL.md":"# minicpm5-finetune-xtuner\n\nPath: `skills/minicpm5-finetune-xtuner/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-xtuner/SKILL.md)","skills/minicpm5-finetune/SKILL.md":"# minicpm5-finetune\n\nPath: `skills/minicpm5-finetune/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune/SKILL.md)"},"items":[{"name":"SKILL.md","path":"skills/minicpm5-deploy-arclight/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-arclight/SKILL.md","title":"minicpm5-deploy-arclight","category":"anthropic-skill","format":"markdown","content":"# Deploy ArcLight from Source (CPU)\n\nArcLight is a lightweight C/C++ LLM inference framework for unified-memory systems. The recommended path is to build from source, then run a GGUF model with `al-gen`, `al-chat`, or `al-ppl`.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL` | `/path/to/MiniCPM5-2B-Q4_0.gguf` | required |\n| `PROMPT` | `\"Hello!\"` | `\"Hello!\"` |\n| `THREADS` | `4` | choose for the target CPU |\n| `NUMA_MODE` | `none` or `tp` | `none` for first run |\n| `NODES` | `1`, `2`, `4` | `1` with `NUMA_MODE=none` |\n| `MAX_GEN` | `256` | `256` |\n\n## Steps\n\n### 1. Build from source\n\n```bash\ngit clone https://github.com/OpenBMB/ArcLight.git\ncd ArcLight\n\ncmake -B build -DARCLIGHT_BACKEND=AUTO\ncmake --build build --config Release -j 32\n```\n\nUse `ARCLIGHT_BACKEND=AUTO` by default. Set it explicitly only when needed:\n\n- `X86`: force the x86 backend\n- `NEON`: force the ARM NEON backend\n- `NONE`: build without architecture-specific backend code\n\n### 2. Prepare a GGUF model\n\nArcLight uses GGUF checkpoints from `llama.cpp`. The `nnml` backend only loads `f32 / f16 / q4_0 / q8_0 / q6_K / q8_K` tensor types — **`Q4_K_M` is not supported**.\n\nSupported model families: MiniCPM5-2B, Qwen3, Llama2.\n\nFor first validation use the released `Q8_0` (`openbmb/MiniCPM5-2B-GGUF`), or quantize an unreleased `Q4_0` yourself from the F16:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-F16.gguf --local-dir .\nllama-quantize ./MiniCPM5-2B-F16.gguf ./MiniCPM5-2B-Q4_0.gguf Q4_0\n```\n\n### 3A. One-shot generation\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\n### 3B. Interactive chat\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\nTo seed the first turn:\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n### 3C. Perplexity\n\n```bash\n./build/al-ppl \\\n    --model \"${MODEL}\" \\\n    --prompt \"Good morning, Miss Lee!\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n## Many-core CPU / NUMA\n\nStart with single-node mode for correctness:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\nUse cross-NUMA tensor parallelism on many-core machines:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa tp --nodes ${NODES} \\\n    --threads ${THREADS}\n```\n\nRules:\n\n- `--numa none` requires `--nodes 1`.\n- `--numa tp` requires `--nodes N` where `N > 1`.\n- In the current version, `NODES` should be a power of 2.\n- Choose `THREADS` so it can be evenly divided across `NODES`.\n- `--numa pp` is reserved for future pipeline parallelism and is not implemented.\n\n## Optional memory buffers\n\nIf allocation fails or the model is larger, pass manual buffer sizes:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --w_gb 4 --a_gb 8 --kv_gb 2 --work_gb 2\n```\n\nMeaning:\n\n- `--w_gb`: weight buffer size\n- `--a_gb`: activation buffer size\n- `--kv_gb`: KV cache buffer size\n- `--work_gb`: temporary workspace size\n\nIncrease `--kv_gb` for longer `--max_length`. Increase `--w_gb` for larger models.\n\n## Validate\n\nRun:\n\n```bash\nMODEL=/path/to/MiniCPM5-2B-Q4_0.gguf\nTHREADS=4\n\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"1+1=?\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_gen 64\n```\n\nThe reply should contain `2` or a short explanation that evaluates to `2`.\n\n## Common pitfalls\n\n- **Program aborts in single-node mode**: use `--numa none --nodes 1`.\n- **Tensor parallel mode fails**: use `--numa tp --nodes N` with `N > 1`; in this version, `N` should be a power of 2.\n- **Pipeline parallelism fails**: `--numa pp` is not implemented yet.\n- **Model loading fails**: verify the checkpoint is GGUF and from a supported model family.\n- **Out of memory**: increase `--w_gb`, `--a_gb`, `--kv_gb`, or `--work_gb`; longer contexts usually need a larger KV cache.\n- **Poor CPU throughput**: check `--threads`, NUMA layout, and thread-core bindings; use `--print_binding 1 --print_perf 1` for diagnostics.\n\n## When NOT to use\n\n- Need CUDA server inference -> use a GPU-oriented runtime instead.\n- Need Apple Silicon MLX inference -> use an MLX deployment path instead.\n- Need a desktop GUI -> use a GUI runtime that supports GGUF models.\n- Need pipeline parallelism -> wait for ArcLight `--numa pp` support.\n\n## Reference\n\n- [`Arclight`](https://github.com/OpenBMB/ArcLight)\n- [llama.cpp GGUF documentation](https://github.com/ggml-org/llama.cpp)\n","isInternal":false,"tokens":1497,"sizeBytes":4854},{"name":"SKILL.md","path":"skills/minicpm5-deploy-litert/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-litert/SKILL.md","title":"minicpm5-deploy-litert","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-litert\ndescription: Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime — the litert-lm CLI or its OpenAI-compatible server on a desktop, the Kotlin API or the AI Edge Gallery app on Android, the same .litertlm bundle on CPU or GPU. Use when the user says \"LiteRT\", \"LiteRT-LM\", \"litertlm\", \".litertlm\", \"Android\", \"Edge Gallery\", \"on-device app\", or wants one bundle for phones and desktops.\n---\n\n# Deploy MiniCPM5-2B / MiniCPM5-1B with LiteRT-LM (Android / iOS / desktop / IoT)\n\n[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM), Google's on-device runtime built on [LiteRT](https://github.com/google-ai-edge/litert) (formerly TensorFlow Lite). One `.litertlm` bundle, CPU or GPU, through a CLI, an OpenAI-compatible server, Python, Kotlin and Swift. The bundles are hosted in `litert-community`. Human-readable reference: [`docs/deployment/litert.md`](../../docs/deployment/litert.md).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `LITERT_REPO` | `litert-community/MiniCPM5-2B` or `litert-community/MiniCPM5-1B` | `litert-community/MiniCPM5-2B` |\n| `FILE` | `MiniCPM5-2B_int4.litertlm` (1.55 GB, the phone file) or `MiniCPM5-2B_int8.litertlm` (2.60 GB, reasoning that has to finish); 1B: `minicpm_wi4b32_wi8_afp32.litertlm` (CPU) or `minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm` (GPU) | `MiniCPM5-2B_int4.litertlm` |\n| `BACKEND` | `cpu` or `gpu` | `cpu` |\n| `THINKING` | `true` (reason first) or `false` (direct answer) | `false` |\n| `PROMPT` | `1+1=?` | `1+1=?` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\nuv tool install litert-lm\n```\n\n### 2A. Run a pre-converted bundle (desktop CLI)\n\n```bash\nlitert-lm run --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" \\\n    --backend \"${BACKEND}\" --thinking \"${THINKING:-false}\" --prompt \"${PROMPT}\"\n```\n\nThe first run downloads the file into `~/.litert-lm/cache/huggingface/<repo>/`; later runs skip the download. With thinking on, the reasoning prints between `[thought]` and `[/thought]` and the answer follows; `--thinking-budget 2048` caps the reasoning (4096 for math); `THINKING=true` makes the model reason first. Sampling as OpenBMB recommends: `--top-k 40 --top-p 0.95 --temperature 1.0` (the CLI's default top-k is 1 = greedy, so temperature alone changes nothing).\n\n### 2B. Android\n\n- **No code**: the [AI Edge Gallery](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery) app, Model manager → **+** → **Import from HF** → paste the file's Hugging Face link.\n- **Your own app**: `implementation(\"com.google.ai.edge.litertlm:litertlm-android:0.17.0\")` from Google Maven, the `libOpenCL.so` `<uses-native-library>` entries in the manifest for the GPU, then:\n\n```kotlin\nval engine = Engine(EngineConfig(modelPath = path, backend = Backend.GPU(), cacheDir = context.cacheDir.absolutePath))\nengine.initialize()\nengine.createConversation(ConversationConfig(maxOutputToken = 1024)).use { conversation ->\n    val reply = conversation.sendMessage(\"1+1=?\")\n    println(reply); println(reply.channels[\"thought\"])\n}\n```\n\n`ConversationConfig(thinkingConfig = ThinkingConfig(enableThinking = false))` turns thinking off. Full snippet, manifest and the Galaxy S26 check: the cookbook.\n\n### 2C. OpenAI-compatible server (desktop)\n\n```bash\nlitert-lm import --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" minicpm5-2b\nlitert-lm serve --host 127.0.0.1 --port 9379\ncurl http://127.0.0.1:9379/v1/chat/completions -H \"Content-Type: application/json\" \\\n    -d '{\"model\": \"minicpm5-2b\", \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}], \"reasoning_effort\": \"none\"}'\n```\n\n`reasoning_effort` `none` = direct answer, any other value = thinking on; the cap field is `max_completion_tokens`; `top_k` is accepted alongside `temperature` / `top_p` / `seed`; `\"stream\": true` streams ([server guide](https://developers.google.com/edge/litert-lm/cli/openai_server)).\n\n### 3. Validate\n\nThe reply contains `2` for `1+1=?`: with `--thinking false` it is the whole reply; with thinking on it follows `[/thought]`. Through the server (2C), the same check is HTTP 200 with `choices[0].message.content` containing `2`, and `GET /v1/models` lists `minicpm5-2b`.\n\n## Common pitfalls\n\n- **Thinking on the int4 file** runs long chains: pass `--thinking false` for direct answers, or use the int8 file when the reasoning has to complete.\n- **int8 on iOS needs the increased-memory-limit entitlement** (2.33 GB weight section; the default entitlements do not map it). int4's 1.28 GB main section is under the limit (checked on iPhone 17 Pro, GPU and CPU).\n- **First run per backend is slow**: an XNNPACK weight cache is written beside the file on the CPU, kernels are compiled on the GPU (`--cache no` to skip the disk cache).\n- **Android GPU without the manifest entries**: `initialize()` succeeds, the first message fails with `Can not find OpenCL library on this device`.\n- **Temperature without top-k does nothing**: pass `--top-k` above 1 together with `--top-p` / `--temperature` (the default is greedy).\n- **Scripts**: the CLI reads stdin; redirect it (`< /dev/null`) when running from a job with an open pipe.\n\n## Reference\n\n[`docs/deployment/litert.md`](../../docs/deployment/litert.md)\n","frontmatter":{"name":"minicpm5-deploy-litert","description":"Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime — the litert-lm CLI or its OpenAI-compatible server on a desktop, the Kotlin API or the AI Edge Gallery app on Android, the same .litertlm bundle on CPU or GPU. Use when the user says \"LiteRT\", \"LiteRT-LM\", \"litertlm\", \".litertlm\", \"Android\", \"Edge Gallery\", \"on-device app\", or wants one bundle for phones and desktops."},"isInternal":false,"tokens":1522,"sizeBytes":5219},{"name":"SKILL.md","path":"skills/minicpm5-deploy-llama-cpp/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-llama-cpp/SKILL.md","title":"minicpm5-deploy-llama-cpp","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-llama-cpp\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for \"llama.cpp\", \"llama-cli\", \"llama-server\", \"GGUF\", or has no Python available.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with llama.cpp\n\nCPU / edge / consumer-GPU deployment via the released GGUF artifacts. The artifacts work directly with vanilla `llama.cpp` and every downstream runtime (Ollama / LM Studio / `llama-cpp-python`).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `NGL` | `99` (all layers on GPU) / `0` (CPU only) | `99` if NVIDIA GPU, else `0` |\n| `CTX` | `8192` (default) up to `131072` (128 K) | `8192` |\n\n## Steps\n\n### 1. Install llama.cpp\n\n```bash\n# macOS\nbrew install llama.cpp\n\n# Linux / cross-platform: pre-built binary\ncurl -fsSL https://github.com/ggerganov/llama.cpp/releases/latest/download/llama-cli-linux.tar.gz | tar -xz\n# OR build from source:\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git && cd llama.cpp\nmkdir build && cd build\ncmake .. -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release    # CPU-only: omit GGML_CUDA=ON\ncmake --build . --config Release -j $(nproc) --target llama-cli llama-server\n```\n\n### 2. Download the GGUF\n\n```bash\nmkdir -p ~/minicpm5 && cd ~/minicpm5\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n```\n\n### 3a. Interactive chat (CLI)\n\n```bash\nllama-cli -m MiniCPM5-2B-${QUANT}.gguf \\\n    -n 2048 --temp 1.0 --top-p 0.95 --min-p 0.0 -ngl ${NGL} -c ${CTX}\n```\n\n### 3b. OpenAI-compatible HTTP server\n\n```bash\nllama-server -m MiniCPM5-2B-${QUANT}.gguf \\\n    --port 8080 -ngl ${NGL} -c ${CTX} --jinja\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:8080/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"min_p\": 0.0, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nIn llama.cpp, the default `min_p=0.05` can lead to repetitive output: it filters out tokens whose probability is below 5% of the highest-probability token, potentially discarding the exact tokens needed to break out of a repetition loop. To prevent this, we set `min_p=0.0`.\n\n## Sampling defaults\n\n| Mode | `--temp` | `--top-p` | `--min-p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | 1.0 | 0.95 | 0.0 |\n| MiniCPM5-1B Think | 0.9 | 0.95 | 0.0 |\n| MiniCPM5-1B No-think | 0.7 | 0.95 | 0.0 |\n\n## Choosing a quant\n\n| Quant | Disk | Quality |\n| --- | --- | --- |\n| F16 | 5.04 GB | reference |\n| Q8_0 | 2.68 GB | ~indistinguishable from F16 |\n| **Q4_K_M** | **1.56 GB** | small drop, ideal for laptops |\n\n## Common pitfalls\n\n- **Slow on CPU + large context**: drop `-c 131072` to `-c 8192` if you don't need 128 K.\n\n## Building your own GGUF (advanced)\n\nIf you've trained your own MiniCPM5-1B or MiniCPM5-2B variant, build a GGUF with:\n\n```bash\npython convert_hf_to_gguf.py /path/to/your-fp16-hf --outfile out/F16.gguf --outtype f16\nllama-quantize out/F16.gguf out/Q4_K_M.gguf Q4_K_M\n```\n\nTrained a **LoRA adapter** (not a full model) and want to apply it at runtime with `--lora` instead of baking it in? Convert it to a GGUF adapter — see **`minicpm5-finetune-gguf-lora`**.\n\n## When NOT to use\n\n- NVIDIA GPU + want OpenAI-compatible serving → `minicpm5-deploy-vllm`\n- Apple Silicon native → `minicpm5-deploy-mlx` is faster\n- Just want one-line desktop run → `minicpm5-deploy-ollama`\n- Want a desktop GUI → `minicpm5-deploy-lmstudio`\n\n## Reference\n\n[`docs/deployment/llama_cpp.md`](../../docs/deployment/llama_cpp.md)\n","frontmatter":{"name":"minicpm5-deploy-llama-cpp","description":"Run MiniCPM5-1B or MiniCPM5-2B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for \"llama.cpp\", \"llama-cli\", \"llama-server\", \"GGUF\", or has no Python available."},"isInternal":false,"tokens":1348,"sizeBytes":3912},{"name":"SKILL.md","path":"skills/minicpm5-deploy-lmstudio/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-lmstudio/SKILL.md","title":"minicpm5-deploy-lmstudio","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-lmstudio\ndescription: Run MiniCPM5-1B or MiniCPM5-2B in LM Studio (desktop GUI) using either the GGUF runtime (cross-platform) or the MLX runtime (Apple Silicon, faster). Includes OpenAI-compatible local server. Use when the user mentions \"LM Studio\", desktop GUI inference, \"lms\" CLI, or wants a no-code chat UI for MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with LM Studio\n\nDesktop GUI + OpenAI-compatible local server. On Apple Silicon ships **two runtimes**:\n\n| Runtime | Format | When to use |\n| --- | --- | --- |\n| **GGUF** (llama.cpp engine) | F16 / Q8_0 / Q4_K_M | cross-platform, same artifact as Ollama; Q4_K_M build |\n| **MLX** (Apple Silicon only) | bf16 / 4-bit | ~60 % faster, automatic think/answer split via `reasoning_content`; Q4 build |\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| Runtime | `gguf` or `mlx` | `mlx` on Apple Silicon, `gguf` elsewhere |\n| `QUANT` | `Q4_K_M` (GGUF) or `4bit` (MLX) | `Q4_K_M` / `4bit` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install LM Studio + complete onboarding\n\n```bash\nbrew install --cask lm-studio\nopen -a \"LM Studio\"     # accept EULA + pick model source\n```\n\n> ⚠️ The first launch MUST be GUI — `lms` (CLI) refuses with `Cannot find LM Studio installation` until LM Studio has run interactively at least once.\n\n### 2A. GGUF runtime path\n\n```bash\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-${QUANT}.gguf \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF/\n\nLMS=\"/Applications/LM Studio.app/Contents/Resources/app/.webpack/lms\"\n\"$LMS\" server start                              # binds 127.0.0.1:1234\n\"$LMS\" load minicpm5-2b --gpu max --context-length 8192 -y\n\"$LMS\" ps                                        # verify the model is loaded\n```\n\n### 2B. MLX runtime path (Apple Silicon, recommended on Mac)\n\nThe MLX runtime needs an MLX-format checkpoint. The published MiniCPM5-2B MLX repo is [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) (4-bit affine); there is no separate `-bf16` / `-4bit` variant. Either drop that one in as-is, or convert locally from `openbmb/MiniCPM5-2B` (see `minicpm5-deploy-mlx`). Then:\n\n```bash\n# Option A — use the official pre-quantized repo\nhuggingface-cli download openbmb/MiniCPM5-2B-MLX \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX\n\n# Option B — drop a locally converted directory in\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}\ncp -r ./minicpm5-mlx-${QUANT}/* ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}/\n\n\"$LMS\" server start\n\"$LMS\" load minicpm5-2b-mlx${QUANT:+-${QUANT}} --gpu max -y\n```\n\n### 3. Validate\n\n```bash\ncurl http://127.0.0.1:1234/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nFor the MLX runtime, a think prompt produces output split into `message.reasoning_content` (the `<think>` block) and `message.content` (the final answer) automatically — that's an MLX-runtime feature, not a model setting.\n\n## Think vs nothink (MiniCPM5-1B)\n\nLM Studio 0.4.13's chat-completion endpoint does **not** propagate `chat_template_kwargs.enable_thinking` to the GGUF runtime. Instead:\n\n- **Default = think mode** for both runtimes.\n- **For nothink** with the GGUF runtime, prepend the closing think block manually:\n  ```json\n  \"messages\": [\n    {\"role\":\"user\",\"content\":\"1+1=?\"},\n    {\"role\":\"assistant\",\"content\":\"<think>\\n\\n</think>\\n\\n\"}\n  ]\n  ```\n  and the model continues from there.\n- **MLX runtime**: think/answer are auto-split, you don't need to do anything.\n\n## Common pitfalls\n\n- **MLX runtime not available**: only on Apple Silicon. On Intel Mac / Windows / Linux LM Studio, only the GGUF runtime works.\n\n## When NOT to use\n\n- Just want CLI / scripted runs → `minicpm5-deploy-ollama` is leaner\n- Production server → `minicpm5-deploy-vllm`\n- No GUI desired → `minicpm5-deploy-llama-cpp` (`llama-server`)\n\n## Reference\n\n[`docs/deployment/lmstudio.md`](../../docs/deployment/lmstudio.md)\n","frontmatter":{"name":"minicpm5-deploy-lmstudio","description":"Run MiniCPM5-1B or MiniCPM5-2B in LM Studio (desktop GUI) using either the GGUF runtime (cross-platform) or the MLX runtime (Apple Silicon, faster). Includes OpenAI-compatible local server. Use when the user mentions \"LM Studio\", desktop GUI inference, \"lms\" CLI, or wants a no-code chat UI for MiniCPM5."},"isInternal":false,"tokens":1342,"sizeBytes":4265},{"name":"SKILL.md","path":"skills/minicpm5-deploy-mlx/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-mlx/SKILL.md","title":"minicpm5-deploy-mlx","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-mlx\ndescription: Run MiniCPM5-1B or MiniCPM5-2B natively on Apple Silicon with Apple's MLX framework. Use when the user has an Apple Silicon Mac and asks for \"MLX\", \"mlx_lm\", \"mlx_lm.convert\", \"mlx_lm.generate\", or wants the fastest path on Apple Silicon.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with MLX (Apple Silicon)\n\nApple's on-device tensor framework. Highest throughput on M-series. Stays inside one Python process — no separate server, no `llama.cpp` build chain.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MLX_REPO` | `openbmb/MiniCPM5-2B-MLX` (pre-converted 4-bit affine) | required; `openbmb/MiniCPM5-1B-MLX` also works |\n| OR `HF_REPO` + `QUANT` | `openbmb/MiniCPM5-2B`, `4bit` or `bf16` | for local conversion |\n| `MAX_TOKENS` | `200` | `200` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"mlx-lm>=0.31\" \"gguf\"\n```\n\n### 2A. Use a pre-converted MLX repo (recommended when available)\n\n```bash\nmlx_lm.generate --model \"${MLX_REPO}\" \\\n    --prompt \"<|im_start|>user\n1+1=?<|im_end|>\n<|im_start|>assistant\n\" \\\n    --max-tokens ${MAX_TOKENS} --temp 1.0 --top-p 0.95\n```\n\n### 2B. Convert from a HF checkpoint locally (advanced)\n\nUse `mlx_lm.convert` only if you have a self-trained HF fp16 checkpoint:\n\n```bash\nHF=/path/to/your-fp16-hf\n\n# Convert: bf16\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-bf16\n\n# Convert: 4-bit (smaller / faster)\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-q4 -q --q-bits 4\n```\n\nThen run as in 2A.\n\n### 3. Validate\n\nThe reply should contain `\"2\"` for `1+1=?`.\n\n## OpenAI-compatible server (mlx-lm)\n\n```bash\nmlx_lm.server --model \"${MLX_REPO}\" --host 127.0.0.1 --port 8000\n\ncurl http://127.0.0.1:8000/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"default\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\n## Common pitfalls\n\n- **Slow first generate**: MLX JIT-compiles kernels on first call (~5-10 s); subsequent calls hit the warm cache.\n- **Model runs past `<|im_end|>`**: only happens on `mlx-lm < 0.31` (older versions ignore multi-id `eos_token_id` lists). Upgrade, or pass `--extra-eos-token \"<|im_end|>\"` as a manual override — `<|im_end|>` is token id 130073 and is already listed in `generation_config.json` on 0.31+.\n\n## When NOT to use\n\n- Not on Apple Silicon → `minicpm5-deploy-llama-cpp` (CPU/CUDA) or `minicpm5-deploy-vllm` (CUDA)\n- Want a desktop GUI → `minicpm5-deploy-lmstudio` (LM Studio bundles an MLX runtime)\n- Want one-line CLI run → `minicpm5-deploy-ollama`\n\n## Reference\n\n[`docs/deployment/mlx.md`](../../docs/deployment/mlx.md)\n","frontmatter":{"name":"minicpm5-deploy-mlx","description":"Run MiniCPM5-1B or MiniCPM5-2B natively on Apple Silicon with Apple's MLX framework. Use when the user has an Apple Silicon Mac and asks for \"MLX\", \"mlx_lm\", \"mlx_lm.convert\", \"mlx_lm.generate\", or wants the fastest path on Apple Silicon."},"isInternal":false,"tokens":891,"sizeBytes":2712},{"name":"SKILL.md","path":"skills/minicpm5-deploy-ollama/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-ollama/SKILL.md","title":"minicpm5-deploy-ollama","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-ollama\ndescription: Run MiniCPM5-1B or MiniCPM5-2B via Ollama on macOS / Linux laptop using the released GGUF. Use when the user wants \"ollama run\", \"ollama pull\", a Modelfile-driven setup, or one-line laptop deployment.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with Ollama\n\nOne-binary, no-Python laptop deployment. Consumes the released GGUF.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install Ollama (once)\n\n```bash\nbrew install ollama                                # macOS\n# or:\ncurl -fsSL https://ollama.com/install.sh | sh      # Linux\n\nOLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve &\n```\n\n### 2. Download the GGUF + write Modelfile\n\n```bash\nmkdir -p ~/${MODEL_NAME} && cd ~/${MODEL_NAME}\n\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n\ncat > Modelfile <<EOF\nFROM ./MiniCPM5-2B-${QUANT}.gguf\n\n# MiniCPM5 basic chat template\nTEMPLATE \"\"\"{{- if .Messages -}}\n{{- range .Messages -}}\n<|im_start|>{{ .Role }}\n{{ .Content }}<|im_end|>\n{{ end -}}\n<|im_start|>assistant\n{{ end -}}\"\"\"\n\nPARAMETER stop \"<|im_end|>\"\nPARAMETER stop \"</s>\"\n\n# Defaults tuned for think mode\nPARAMETER temperature 1.0\nPARAMETER top_p 0.95\nPARAMETER num_ctx 8192\nEOF\n```\n\n### 3. Create + run\n\n```bash\nollama create ${MODEL_NAME} -f Modelfile\nollama run ${MODEL_NAME}\n```\n\n### 4. Validate via OpenAI-compatible API\n\n```bash\ncurl http://localhost:11434/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply. \n\n## MiniCPM5-1B Think vs No-think\n\nA MiniCPM5-1B Modelfile configured with `temperature=0.7, top_p=0.95` defaults to **no-think**. To switch a single conversation to **think** mode, override the sampling params at request time:\n\n```bash\nollama run minicpm5-1b --temperature 0.9 --top-p 0.95\n```\n\nOr bake it into a separate model tag by raising the temperature line (top_p stays 0.95):\n\n```Modelfile\nPARAMETER temperature 0.9\nPARAMETER top_p 0.95\n```\n\nThen `ollama create minicpm5-1b-think -f Modelfile.think`.\n\nOllama 0.24 does **not** directly evaluate the GGUF-embedded Jinja chat template. It maps recognized Jinja templates to built-in Go templates, so MiniCPM5 uses the explicit Go `TEMPLATE` block above. To enter the MiniCPM5-2B think path, use raw mode and prepend `<think>\\n` manually:\n\n```bash\ncurl http://localhost:11434/api/generate -d '{\n    \"model\": \"minicpm5-2b\",\n    \"raw\": true,\n    \"prompt\": \"<|im_start|>user\\n鸡兔同笼…<|im_end|>\\n<|im_start|>assistant\\n<think>\\n\",\n    \"options\": {\"temperature\": 1.0, \"top_p\": 0.95}\n}'\n```\n\n## Common pitfalls\n\n- **`Error: invalid file magic`**: corrupted download. Re-run `huggingface-cli download`.\n\n## When NOT to use\n\n- Highest throughput on Mac → `minicpm5-deploy-mlx` (Q4 build)\n- GUI experience → `minicpm5-deploy-lmstudio`\n- NVIDIA GPU production → `minicpm5-deploy-vllm`\n\n## Reference\n\n[`docs/deployment/ollama.md`](../../docs/deployment/ollama.md)\n","frontmatter":{"name":"minicpm5-deploy-ollama","description":"Run MiniCPM5-1B or MiniCPM5-2B via Ollama on macOS / Linux laptop using the released GGUF. Use when the user wants \"ollama run\", \"ollama pull\", a Modelfile-driven setup, or one-line laptop deployment."},"isInternal":false,"tokens":1117,"sizeBytes":3375},{"name":"SKILL.md","path":"skills/minicpm5-deploy-sglang/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-sglang/SKILL.md","title":"minicpm5-deploy-sglang","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-sglang\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for \"SGLang\", \"RadixAttention\", \"prefix cache\", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with SGLang\n\nOpenAI-compatible server with RadixAttention prefix cache. Best fit for **tool calling**, batched eval pipelines, and high-concurrency serving.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B`| required; |\n| `PORT` | `30000` | `30000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TOOL_PARSER` | `minicpm5` | `minicpm5`; use `auto` if you want template detection |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"sglang[srt]>=0.5.16\"          # latest, requires CUDA 13.x driver\n# pip install \"sglang==0.5.6.post3\"        # fallback for CUDA 12.x driver hosts\n```\n\n**For tool calling, install from `main`** — the `minicpm5` parser ([PR #25600](https://github.com/sgl-project/sglang/pull/25600), merged 2026-05-22) is not in any pip release yet (`v0.5.12.post1` was branched earlier). Plain chat works on the pip release; only `--tool-call-parser minicpm5` needs `main`:\n\n```bash\npip install \"git+https://github.com/sgl-project/sglang.git@main#subdirectory=python\"\n```\n\n### 2. Recommended runtime env vars\n\n```bash\nexport VLLM_WORKER_MULTIPROC_METHOD=spawn\nexport SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1\nexport SGLANG_DISABLE_CUDNN_CHECK=1\n```\n\n### 3. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT}\n```\n\nWait for `The server is fired up and ready to roll!` .\n\n**Speculative decoding (DSpark)**: we also release [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark), a DSpark draft model trained for MiniCPM5-2B. Enable it in SGLang to accelerate decoding while keeping the target model's outputs unchanged:\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT} \\\n    --speculative-algorithm DSPARK \\\n    --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \\\n    --speculative-dspark-block-size 7\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains `\"2\"`.\n\n## Tool calling\n\nSGLang includes a native MiniCPM5 XML tool-call parser. Keep `TOOL_PARSER=minicpm5` (or `auto`) when launching, then send an OpenAI-style tools request:\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n        \"tools\": [{\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get current weather for a city\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\"city\": {\"type\": \"string\"}},\n                    \"required\": [\"city\"]\n                }\n            }\n        }],\n        \"tool_choice\": \"auto\",\n        \"temperature\": 1.0, \"top_p\": 0.95,\n        \"max_tokens\": 256\n    }'\n```\n\n## Offline / batched (Engine API)\n\n```python\nimport sglang as sgl\n\nllm = sgl.Engine(model_path=\"${MODEL_PATH}\", tp_size=1, mem_fraction_static=0.8, context_length=131072)\n\noutputs = llm.generate(\n    [\"用一句话解释什么是 GQA。\"],\n    sampling_params={\"temperature\": 1.0, \"top_p\": 0.95, \"max_new_tokens\": 1024,\n                     \"skip_special_tokens\": False},\n)\nprint(outputs)\n```\n\n## Common pitfalls\n\n- **`GLIBCXX_3.4.31 not found`** (only if you hit it — not universal): conda Python ships an older `libstdc++` than the SGLang wheel was built against. Force-load the system one: `LD_PRELOAD=/lib/x86_64-linux-gnu/libstdc++.so.6 python -m sglang.launch_server ...`\n\n## When NOT to use\n\n- Production serving without batch-eval needs → `minicpm5-deploy-vllm` is simpler\n- One-shot Python → `minicpm5-deploy-transformers`\n- No NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n\n## Reference\n\n[`docs/deployment/sglang.md`](../../docs/deployment/sglang.md)\n","frontmatter":{"name":"minicpm5-deploy-sglang","description":"Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for \"SGLang\", \"RadixAttention\", \"prefix cache\", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM."},"isInternal":false,"tokens":1551,"sizeBytes":5168},{"name":"SKILL.md","path":"skills/minicpm5-deploy-transformers/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-transformers/SKILL.md","title":"minicpm5-deploy-transformers","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-transformers\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with Hugging Face Transformers for one-shot Python generation on GPU (bfloat16) or CPU (float32). Use when the user wants a quick Python script, no server, no extra deps, or asks for \"Transformers\", \"AutoModelForCausalLM\", \"model.generate\" with MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with HF Transformers\n\nOne-shot Python generation. No server. Works on a single GPU (bfloat16) or CPU only (fp32).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or local dir | required; `openbmb/MiniCPM5-1B` also works |\n| `MODE` | `think` or `nothink` (`nothink` is 1B-only) | `think` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install -U \"transformers>=5.6,<6\" \"torch>=2.11\" accelerate     # latest (CUDA 13.x driver hosts)\n# pip install -U \"transformers==4.57.3\" \"torch==2.7.1\" accelerate  # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Run\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_path = \"${MODEL_PATH}\"      # ← replace\ntok = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_path,\n    torch_dtype=torch.bfloat16,    # CPU users: torch.float32 + device_map=\"cpu\"\n    device_map=\"auto\",\n).eval()\n\nmessages = [{\"role\": \"user\", \"content\": \"用一句话解释什么是 GQA。\"}]\ninputs = tok.apply_chat_template(\n    messages,\n    add_generation_prompt=True,\n    enable_thinking=True,\n    return_tensors=\"pt\",\n    return_dict=True,\n).to(model.device)\n\nwith torch.no_grad():\n    out = model.generate(\n        **inputs,\n        max_new_tokens=1024,\n        do_sample=True,\n        temperature=1.0,\n        top_p=0.95,\n    )\n\nprompt_len = inputs[\"input_ids\"].shape[-1]\nprint(tok.decode(out[0][prompt_len:], skip_special_tokens=True))\n```\n\nFor CPU only: change `torch_dtype=torch.float32, device_map=\"cpu\"`. Keep `enable_thinking=True` and `temperature=1.0` for MiniCPM5-2B; use `enable_thinking=False` and `temperature=0.7` only for MiniCPM5-1B No-think mode.\n\n## Sampling defaults\n\n| Mode | `enable_thinking` | `temperature` | `top_p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | `True` | 1.0 | 0.95 |\n| MiniCPM5-1B Think | `True` | 0.9 | 0.95 |\n| MiniCPM5-1B No-think | `False` | 0.7 | 0.95 |\n\n## Validate\n\nA coherent answer to `1+1=?` (e.g. `\"2\"` or `\"答案是 2\"`).\n\n## LoRA inference\n\n```python\nfrom peft import PeftModel\nbase = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=\"auto\")\nmodel = PeftModel.from_pretrained(base, \"/path/to/adapter\").eval()\n```\n\nAdapters from any of the `minicpm5-finetune-*` skills load directly with no surgery.\n\n## When NOT to use\n\n- Need an OpenAI-compatible HTTP server → `minicpm5-deploy-vllm` or `minicpm5-deploy-sglang`\n- Apple Silicon → `minicpm5-deploy-mlx` is faster\n- CPU-only or low-VRAM laptop → `minicpm5-deploy-llama-cpp` with Q4_K_M is faster\n\n## Reference\n\n[`docs/deployment/transformers.md`](../../docs/deployment/transformers.md)\n","frontmatter":{"name":"minicpm5-deploy-transformers","description":"Run MiniCPM5-1B or MiniCPM5-2B with Hugging Face Transformers for one-shot Python generation on GPU (bfloat16) or CPU (float32). Use when the user wants a quick Python script, no server, no extra deps, or asks for \"Transformers\", \"AutoModelForCausalLM\", \"model.generate\" with MiniCPM5."},"isInternal":false,"tokens":946,"sizeBytes":3082},{"name":"SKILL.md","path":"skills/minicpm5-deploy-vllm-ascend/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-vllm-ascend/SKILL.md","title":"minicpm5-deploy-vllm-ascend","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-vllm-ascend\ndescription: Deploy MiniCPM5-2B with vLLM on Huawei Ascend NPU using vLLM-Ascend. Use when the user mentions vLLM-Ascend, Ascend NPU, Huawei Ascend, CANN, torch_npu, davinci devices, or wants an OpenAI-compatible MiniCPM5 server on Ascend hardware.\n---\n\n# Deploy MiniCPM5-2B with vLLM-Ascend\n\nUse this skill for the BF16 / FP16 `openbmb/MiniCPM5-2B` checkpoint on Huawei Ascend NPU. `vllm-ascend` is installed alongside a matching vLLM release and supplies the Ascend backend.\n\n## Required input\n\n| Variable | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required |\n| `PORT` | `8000` | `8000` |\n| `ASCEND_RT_VISIBLE_DEVICES` | `0` | `0` |\n| `CTX_LEN` | `32768` | `32768`; the native maximum is `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TP_SIZE` | `1` | number of visible devices |\n\nAccept either the Hugging Face id or a local directory containing the MiniCPM5-2B model files.\n\n## Prerequisites\n\nUse a compatible Linux Ascend environment with CANN, PyTorch, and TorchNPU installed. Keep the vLLM and vLLM-Ascend versions matched. The basic serving recipe below follows the tested `vllm==0.18.0` / `vllm-ascend==0.18.0` pair.\n\nDevice visibility is controlled by `ASCEND_RT_VISIBLE_DEVICES`; `CUDA_VISIBLE_DEVICES` does not select Ascend devices.\n\n## Install\n\nThe prebuilt image is the simplest option for a compatible Ascend host. This is a minimal single-device example; adjust device nodes and driver mounts for the host hardware and image documentation.\n\n```bash\ndocker run -it --net=host --shm-size=1g \\\n    --device /dev/davinci0 --device /dev/davinci_manager \\\n    --device /dev/devmm_svm --device /dev/hisi_hdc \\\n    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \\\n    -v /usr/local/dcmi:/usr/local/dcmi \\\n    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \\\n    quay.io/ascend/vllm-ascend:v0.18.0\n```\n\nFor an existing CANN environment:\n\n```bash\npip install vllm==0.18.0 vllm-ascend==0.18.0 \\\n    --extra-index-url https://download.pytorch.org/whl/cpu\n```\n\nFor built-in MiniCPM5 tool-call parsing, use the matching `vllm==0.23.0` / `vllm-ascend==0.23.0` stack described in [Tool calling](#tool-calling). The parser responses in the cookbook were produced with the upstream parser file on an older vLLM and a compatibility shim because the 0.23.0 stack was not available on the test hardware.\n\n## Prepare the environment\n\nRun the CANN and NNAL setup scripts before starting vLLM. Change the paths if those components are installed elsewhere.\n\n```bash\nsource /usr/local/Ascend/ascend-toolkit/set_env.sh\nsource /usr/local/Ascend/nnal/atb/set_env.sh\n\nexport ASCEND_RT_VISIBLE_DEVICES=${ASCEND_RT_VISIBLE_DEVICES:-0}\nexport LD_LIBRARY_PATH=$(python3 -c \"import torch,os;print(os.path.dirname(torch.__file__))\")/lib:\\\n$(python3 -c \"import torch_npu,os;print(os.path.dirname(torch_npu.__file__))\")/lib:$LD_LIBRARY_PATH\n```\n\nCheck the driver and Python packages before loading the model:\n\n```bash\nnpu-smi info\npython3 -c \"import torch, torch_npu; print(torch_npu.npu.is_available(), torch_npu.npu.device_count())\"\n```\n\nIf `npu-smi` reports a missing driver library, add the corresponding driver `lib64` directories to `LD_LIBRARY_PATH` according to the host installation.\n\n## Launch the server\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --port ${PORT:-8000}\n```\n\nSet `TP_SIZE` to the number of devices in `ASCEND_RT_VISIBLE_DEVICES` when using tensor parallelism.\n\n## Validate chat completions\n\nMiniCPM5-2B is think-only. Use `temperature=1.0`, `top_p=0.95`, and `enable_thinking=true`:\n\n```bash\ncurl http://localhost:${PORT:-8000}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0,\n        \"top_p\": 0.95,\n        \"max_tokens\": 256,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpect HTTP 200 and an answer containing `2` in `choices[0].message.content`. Leave enough `max_tokens` for the model's reasoning text.\n\n## Tool calling\n\nMiniCPM5-2B emits XML-style tool calls. This section requires matching vLLM and vLLM-Ascend `0.23.0`; that release includes the `minicpm5` parser in the OpenAI server layer, so no parser plugin file is needed:\n\n```bash\npip install vllm==0.23.0\npip install vllm-ascend==0.23.0 \\\n    --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi\n```\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --enable-auto-tool-choice \\\n    --tool-call-parser minicpm5 \\\n    --port ${PORT:-8000}\n```\n\nWhen sending `tools`, use both server flags above and the same 2B sampling settings:\n\n```json\n{\n  \"model\": \"MiniCPM5-2B\",\n  \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n  \"tools\": [{\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_weather\",\n      \"description\": \"Get current weather for a city\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\"city\": {\"type\": \"string\"}},\n        \"required\": [\"city\"]\n      }\n    }\n  }],\n  \"tool_choice\": \"auto\",\n  \"temperature\": 1.0,\n  \"top_p\": 0.95,\n  \"max_tokens\": 512,\n  \"chat_template_kwargs\": {\"enable_thinking\": true}\n}\n```\n\n## Common failures\n\n- `torch_npu` cannot import: source the CANN/NNAL environment and check that the installed PyTorch, TorchNPU, CANN, vLLM, and vLLM-Ascend versions match the supported stack.\n- No devices are visible: check `npu-smi info`, `ASCEND_RT_VISIBLE_DEVICES`, and the Docker device/driver mounts. Do not use `CUDA_VISIBLE_DEVICES` for this backend.\n- HBM allocation or 128K startup failure: lower `--max-model-len`, then lower `--gpu-memory-utilization`.\n- Tensor-parallel initialization failure: make `--tensor-parallel-size` match the number of visible Ascend devices.\n- Tool requests fail with an auto-tool-choice error: launch with both `--enable-auto-tool-choice` and `--tool-call-parser minicpm5`.\n\n## Reference\n\nSee [`docs/deployment/vllm-ascend.md`](../../docs/deployment/vllm-ascend.md) for the human-readable cookbook and the [official vLLM-Ascend installation guide](https://docs.vllm.ai/projects/ascend/en/v0.23.0/installation.html) for hardware-specific setup.\n","frontmatter":{"name":"minicpm5-deploy-vllm-ascend","description":"Deploy MiniCPM5-2B with vLLM on Huawei Ascend NPU using vLLM-Ascend. Use when the user mentions vLLM-Ascend, Ascend NPU, Huawei Ascend, CANN, torch_npu, davinci devices, or wants an OpenAI-compatible MiniCPM5 server on Ascend hardware."},"isInternal":false,"tokens":1933,"sizeBytes":6631},{"name":"SKILL.md","path":"skills/minicpm5-deploy-vllm/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy-vllm/SKILL.md","title":"minicpm5-deploy-vllm","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy-vllm\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for \"vLLM\", \"OpenAI server\", \"REST API for MiniCPM5\", or \"production deployment\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with vLLM\n\nOpenAI-compatible server for the BF16 / FP16 MiniCPM5-1B or MiniCPM5-2B checkpoint.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` also works |\n| `PORT` | `8000` | `8000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072`; lower if VRAM tight |\n| `MEM_FRAC` | `0.85` | `0.85`; lower on shared GPUs |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"vllm>=0.21\"          # latest (CUDA 13.x driver hosts)\n# pip install \"vllm==0.10.1.1\"    # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} vllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN} \\\n    --gpu-memory-utilization ${MEM_FRAC} \\\n    --port ${PORT}\n```\n\nWait for `Application startup complete` in the log.\n\n### 3. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains a coherent answer.\n\n## Sampling defaults\n\n```json\n{\"temperature\": 1.0, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-2B Think\n{\"temperature\": 0.9, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-1B Think\n{\"temperature\": 0.7, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": false}}   // MiniCPM5-1B No-think\n```\n\n## Common pitfalls\n\n- **`(free / total) < MEM_FRAC` hard error**: lower `--gpu-memory-utilization` (e.g. 0.5 on a shared GPU).\n- **OOM at startup with 128 K**: drop `--max-model-len` to 32768 or 8192.\n\n## Tool calling (plugin)\n\nThe vLLM-side MiniCPM5 XML parser ([PR #43175](https://github.com/vllm-project/vllm/pull/43175)) merged to `main` on 2026-05-27 but is **not in any pip release yet** (`v0.22.0` was cut before the merge). Use the bridge plugin shipped at `tool_parsers/minicpm5xml_tool_parser.py` in this repo:\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 --max-model-len ${CTX_LEN} --port ${PORT} \\\n    --enable-auto-tool-choice \\\n    --tool-parser-plugin /path/to/MiniCPM/tool_parsers/minicpm5xml_tool_parser.py \\\n    --tool-call-parser minicpm5\n```\n\nDrop `--tool-parser-plugin` once vLLM ships a release containing the parser natively.\n\n## When NOT to use\n\n- One-shot Python script → `minicpm5-deploy-transformers`\n- Apple Silicon / no NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n- High-concurrency batch eval w/ prefix cache or tool calling → `minicpm5-deploy-sglang`\n\n## Reference\n\n[`docs/deployment/vllm.md`](../../docs/deployment/vllm.md)\n","frontmatter":{"name":"minicpm5-deploy-vllm","description":"Serve MiniCPM5-1B or MiniCPM5-2B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for \"vLLM\", \"OpenAI server\", \"REST API for MiniCPM5\", or \"production deployment\"."},"isInternal":false,"tokens":1055,"sizeBytes":3259},{"name":"SKILL.md","path":"skills/minicpm5-deploy/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-deploy/SKILL.md","title":"minicpm5-deploy","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-deploy\ndescription: Pick the right inference backend for a MiniCPM5-1B or MiniCPM5-2B checkpoint and route to a backend-specific cookbook skill. Use when the user wants to deploy / serve / chat-with / benchmark a MiniCPM5 model and has not yet committed to a specific engine, or when they say \"deploy MiniCPM5\", \"run MiniCPM5\", \"serve MiniCPM5\", \"MiniCPM5 推理\", \"部署 MiniCPM5\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B — backend router\n\nYou're being asked to deploy / serve / chat-with a MiniCPM5-1B or MiniCPM5-2B checkpoint. Your job is to **pick exactly one backend skill below** based on the user's hardware, format, and goal, then **invoke that skill** rather than improvising.\n\n## 1. Required input from the user\n\nBefore picking a backend, you MUST know:\n\n| Variable | Example | Where to ask |\n| --- | --- | --- |\n| `MODEL_PATH` | HF id `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B` **or** a local path | \"Which checkpoint? HF id or local path?\" |\n| Hardware | NVIDIA GPU / Apple Silicon / CPU only | infer from context, otherwise ask |\n| Goal | \"interactive chat\" / \"OpenAI server\" / \"Python script\" / \"benchmark\" | infer from context |\n\n### Available checkpoints on Hugging Face\n\n| Variant | HF repo | Use with |\n| --- | --- | --- |\n| **HF fp16 (recommended)** | [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) or [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) | `transformers` / `vllm` (no `--quantization`) / `vllm-ascend` / `sglang` / any `minicpm5-finetune-*` |\n| GGUF F16 / Q8_0 / Q4_K_M | [`openbmb/MiniCPM5-2B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) or [`openbmb/MiniCPM5-1B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) | `minicpm5-deploy-llama-cpp` / `-ollama` / `-lmstudio` |\n| MLX (Apple Silicon) | [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) or [`openbmb/MiniCPM5-1B-MLX`](https://huggingface.co/openbmb/MiniCPM5-1B-MLX) | `minicpm5-deploy-mlx` |\n| LiteRT-LM `.litertlm` (Android / iOS / desktop / IoT, CPU + GPU) | [`litert-community/MiniCPM5-2B`](https://huggingface.co/litert-community/MiniCPM5-2B) or [`litert-community/MiniCPM5-1B`](https://huggingface.co/litert-community/MiniCPM5-1B) | `minicpm5-deploy-litert` |\n\nIf the user has a local copy, accept any directory path that contains `config.json` and `model.safetensors` (or the equivalent GGUF / MLX layout).\n\n## 2. Decision matrix — pick exactly one\n\n| User says / wants | Hardware | Format | → Skill to invoke |\n| --- | --- | --- | --- |\n| \"Quick Python script\" / \"one-shot generation\" / \"no server\" | any GPU or CPU | HF safetensors | **`minicpm5-deploy-transformers`** |\n| \"OpenAI server\" / \"production serving\" / \"high QPS\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-vllm`** |\n| \"vLLM-Ascend\" / \"Ascend NPU\" / \"CANN\" / \"torch_npu\" | Huawei Ascend NPU | HF safetensors | **`minicpm5-deploy-vllm-ascend`** |\n| \"RadixAttention\" / \"prefix cache\" / \"batched eval\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-sglang`** |\n| \"GGUF\" / \"llama.cpp\" / \"llama-cli\" / \"CPU only\" | any CPU + optional GPU | GGUF | **`minicpm5-deploy-llama-cpp`** |\n| \"Ollama\" / \"ollama run\" / \"Modelfile\" | macOS / Linux laptop | GGUF | **`minicpm5-deploy-ollama`** |\n| \"LM Studio\" / \"desktop GUI\" | macOS / Windows / Linux | GGUF or MLX | **`minicpm5-deploy-lmstudio`** |\n| \"MLX\" / \"Apple Silicon native\" / \"fastest on Mac\" | Apple Silicon | MLX | **`minicpm5-deploy-mlx`** |\n| \"Android\" / \"on-device app\" / \"Edge Gallery\" / \"LiteRT\" / \"LiteRT-LM\" / \"litertlm\" / \"Raspberry Pi\" | Android phone, iPhone, desktop or IoT board (CPU or GPU) | LiteRT-LM `.litertlm` | **`minicpm5-deploy-litert`** |\n\nIf the user **has not specified** any of the above and asks \"how do I run this?\":\n\n- **CUDA box, want fastest server**: pick `minicpm5-deploy-vllm`.\n- **Ascend NPU, want an OpenAI-compatible server**: pick `minicpm5-deploy-vllm-ascend`.\n- **CUDA box, want minimal Python**: pick `minicpm5-deploy-transformers`.\n- **Apple Silicon laptop**: pick `minicpm5-deploy-ollama` (easiest) or `minicpm5-deploy-mlx` (fastest).\n- **Android phone, or an on-device app (Android / iOS)**: pick `minicpm5-deploy-litert`.\n- **CPU only / Windows / low-VRAM**: pick `minicpm5-deploy-llama-cpp` (Q4_K_M).\n\n## 3. Invocation contract\n\nOnce you've picked a backend skill, **invoke that skill with `MODEL_PATH` set**. Do NOT inline the backend's commands here — each backend has its own pitfalls (mandatory flags, env-var pins, install order) that the dedicated skill handles. The user explicitly does NOT want you to \"improvise\" — read the picked sub-skill in full first.\n\n## 4. Sanity check after deploy\n\nWhichever backend you pick, after launch run this universal sanity check:\n\n```bash\n# Replace localhost:PORT with the backend's actual port (default in each sub-skill)\ncurl http://localhost:PORT/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: HTTP 200 with `choices[0].message.content` containing `\"2\"`.\n\n`minicpm5-deploy-litert` also serves this endpoint: after `litert-lm import … minicpm5-2b`, `litert-lm serve` listens on port 9379 ([guide](https://developers.google.com/edge/litert-lm/cli/openai_server)); use `\"model\": \"minicpm5-2b\"`, add `\"reasoning_effort\": \"none\"` for a direct answer, and cap with `max_completion_tokens`.\n\n## 5. Known cross-backend pitfalls\n\nThese are common to multiple backends — surface to the user up front:\n\n- **Think vs no-think**: MiniCPM5-2B uses Think mode with `temperature=1.0, top_p=0.95`. MiniCPM5-1B also supports No-think with `enable_thinking=false` + `temperature=0.7, top_p=0.95`.\n- **128 K context**: `max_position_embeddings=131072`, `rope_theta=5e6`, **no rope-scaling**. Pass `--max-model-len 131072` (vLLM) / `--context-length 131072` (SGLang) / `-c 131072` (llama.cpp) to use the full window. Lower if VRAM is tight.\n- **Untied lm_head**: `tie_word_embeddings=false`. Tools that assume the Llama tied default (e.g. `mlx_lm.convert` < 0.31) will silently drop `lm_head` → output collapses to random tokens. The MLX skill bakes in the fix.\n\n## 6. Don't reinvent: link to the cookbook\n\nEach sub-skill is paired with a one-page cookbook in [`docs/deployment/`](../../docs/deployment/). The skill is the machine-readable shortcut; the cookbook is the human-readable reference. Both are kept in sync.\n","frontmatter":{"name":"minicpm5-deploy","description":"Pick the right inference backend for a MiniCPM5-1B or MiniCPM5-2B checkpoint and route to a backend-specific cookbook skill. Use when the user wants to deploy / serve / chat-with / benchmark a MiniCPM5 model and has not yet committed to a specific engine, or when they say \"deploy MiniCPM5\", \"run MiniCPM5\", \"serve MiniCPM5\", \"MiniCPM5 推理\", \"部署 MiniCPM5\"."},"isInternal":false,"tokens":2119,"sizeBytes":6589},{"name":"SKILL.md","path":"skills/minicpm5-finetune-gguf-lora/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-gguf-lora/SKILL.md","title":"minicpm5-finetune-gguf-lora","category":"anthropic-skill","format":"markdown","content":"---\nname: minicpm5-finetune-gguf-lora\ndescription: Fine-tune MiniCPM5-1B into a LoRA adapter and convert it to a GGUF adapter that loads directly into llama.cpp / llama-server and the MiniCPM Desk Pet app's custom-LoRA upload. Use when the user wants \"GGUF LoRA\", \"convert LoRA to GGUF\", \"convert_lora_to_gguf\", a custom persona/skin for the desktop pet, \"桌宠自定义 LoRA\", \"上传 LoRA 到桌宠\", or asks how to take a trained adapter and run it on a GGUF base.\n---\n\n# Fine-tune MiniCPM5-1B → GGUF LoRA adapter\n\nThe framework skills (`minicpm5-finetune-*`) all emit a **PEFT adapter** (`adapter_model.safetensors` + `adapter_config.json`). But `llama.cpp` / `llama-server` — and the **MiniCPM Desk Pet** app's custom-LoRA upload — load a **GGUF LoRA adapter** (`--lora some-adapter.gguf`). This skill is the bridge: train a PEFT LoRA, then convert it to GGUF and run/upload it.\n\n> This is the path you want when the base model is served as GGUF (Desk Pet, Ollama, LM Studio, plain `llama-server`). If you serve the fp16 HF base with vLLM / transformers, you don't need GGUF — load the PEFT adapter directly.\n\n## Pipeline overview\n\n```\n  train (any minicpm5-finetune-* skill)        this skill\n ┌────────────────────────────────────┐   ┌──────────────────────────────────┐\n BASE (fp16 HF) ─► adapter_model.safetensors ─► convert_lora_to_gguf.py ─► adapter.gguf\n                   adapter_config.json                                          │\n                                                                  llama-server --lora / Desk Pet upload\n```\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `ADAPTER_DIR` | `./runs/minicpm5_unsloth/adapter_final` | required — a PEFT dir with `adapter_config.json` + `adapter_model.safetensors` |\n| `BASE_MODEL` | `openbmb/MiniCPM5-1B` (HF id) **or** a local fp16 HF dir | required — must be the **same base** the adapter was trained on |\n| `OUTTYPE` | `f16` (recommended) / `q8_0` / `bf16` / `f32` | `f16` |\n| `OUT_GGUF` | `./minicpm5-mylora.gguf` | `<ADAPTER_DIR>/adapter_model.f16.gguf` |\n\n> **Don't have an adapter yet?** First run a training skill — start from the router **`minicpm5-finetune`** (or go straight to **`minicpm5-finetune-unsloth`** for single-GPU LoRA). Come back here with `ADAPTER_DIR` pointing at its output.\n\n## Steps\n\n### 1. Get llama.cpp (has the converter)\n\n```bash\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git\ncd llama.cpp\npip install -r requirements.txt          # converter deps: torch, safetensors, gguf, transformers\n```\n\n`convert_lora_to_gguf.py` lives at the repo root. You only need the Python script for conversion; build the C++ binaries (step 4) only if you want to test locally.\n\n### 2. Convert the PEFT adapter → GGUF\n\n```bash\npython convert_lora_to_gguf.py \"$ADAPTER_DIR\" \\\n    --base \"$BASE_MODEL\" \\\n    --outtype f16 \\\n    --outfile \"$OUT_GGUF\"\n```\n\n- `--base` only needs the base model's **config** (`config.json`, `tokenizer.json`) — not the weights. A local fp16 HF dir or an HF id both work.\n- If the base is only on the Hub, use `--base-model-id openbmb/MiniCPM5-1B` instead of `--base` (it pulls just the config).\n- Output is a single `.gguf` of roughly the same size as the input `.safetensors` (a 22 MB r=16 adapter → ~22 MB GGUF).\n\n> 🔑 **The #1 gotcha — `base_model_name_or_path` points at the training machine.**\n> PEFT writes the absolute path of the base used during training into `adapter_config.json`\n> (e.g. `/user/.../MiniCPM5-models-fixed/official`). On any other machine the converter\n> can't find it. **Always pass `--base` (or `--base-model-id`) explicitly** to override it —\n> don't rely on whatever is baked into the config. The base you pass MUST match the one you\n> trained on, or the adapter math is meaningless.\n\n### 3. (Optional) Sanity-check the GGUF metadata\n\n```bash\npython -c \"import gguf,sys; r=gguf.GGUFReader('$OUT_GGUF'); print('tensors:', len(r.tensors)); print('arch:', r.get_field('general.architecture').parts[-1].tobytes().decode() if r.get_field('general.architecture') else '?')\"\n```\n\nExpect `general.type = adapter` and a nonzero tensor count. MiniCPM5-1B is a **Llama-architecture** model, so the converter treats it as `llama` — this is correct, not an error.\n\n### 4. Test with llama-server before shipping\n\nYou need a GGUF **base** model too (the adapter is applied on top of it). Grab the released base:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-1B-GGUF MiniCPM5-1B-Q8_0.gguf --local-dir .\n```\n\nThen:\n\n```bash\n# CLI\nllama-cli -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" \\\n    -p \"你好\" -n 128 --temp 0.7 --top-p 0.95\n\n# OpenAI-compatible server\nllama-server -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" --port 8080 --jinja\n```\n\nIf the persona/behavior you trained shows up, the GGUF is good. If output is identical to the base, the adapter didn't load (check the `--lora` path and that `--base` matched in step 2).\n\n> **Quant compatibility:** a GGUF LoRA built from an fp16 adapter applies fine on top of a\n> quantized base (Q8_0 / Q4_K_M). You do **not** need a separate adapter per base quant.\n\n### 5. Use it in MiniCPM Desk Pet\n\nThe Desk Pet app loads exactly this kind of file. The fp16 GGUF you just built is upload-ready:\n\n1. Open **Settings → MiniCPM → 适配器 (LoRA)**.\n2. Click **上传** (Upload), pick your `.gguf` — the app copies it into its\n   `<userData>/adapters/uploads/` directory and registers it.\n3. Give it a display name + comma-separated aliases (used so you can switch personas by\n   voice/chat, e.g. \"换成 XX\").\n4. Select it in the adapter list to activate. The sidecar reloads `llama-server` with your\n   `--lora`.\n\nConstraints the app enforces (match them or the upload is rejected):\n\n- **Must be a single `.gguf` file** (the GGUF LoRA from step 2 — *not* the `.safetensors`,\n  *not* a merged full model).\n- It is applied on top of whatever MiniCPM5-1B GGUF base the app already runs, so **train\n  against the MiniCPM5-1B base**, not some other model.\n\n## Common pitfalls\n\n- **`FileNotFoundError` / `can't load base model config`** during conversion → the\n  `adapter_config.json` base path doesn't exist locally. Fix with `--base <local-dir>` or\n  `--base-model-id openbmb/MiniCPM5-1B` (step 2 gotcha).\n- **Adapter loads but output is unchanged** → base mismatch (trained on base A, applied on\n  base B), or the persona needs its **system prompt** too — a LoRA biases style but the\n  system prompt still matters. In Desk Pet the persona's system prompt is wired to the\n  adapter; standalone `llama-cli` users must pass it themselves.\n- **`KeyError` on an unknown tensor / target module** → the adapter targeted modules the\n  converter doesn't map. Stick to the standard attention+MLP projections\n  (`q/k/v/o_proj`, `gate/up/down_proj`) when training (all `minicpm5-finetune-*` skills\n  already default to these).\n- **Uploaded `.gguf` but Desk Pet shows nothing** → you uploaded the merged full model or a\n  base GGUF by mistake. A LoRA adapter GGUF is small (tens of MB) and has\n  `general.type = adapter` (step 3).\n\n## When NOT to use\n\n- Serving the fp16 HF base with **vLLM / transformers / SGLang** → load the PEFT adapter\n  directly, skip GGUF. See `minicpm5-deploy-vllm` / `-transformers` / `-sglang`.\n- You want to **bake the LoRA into the weights** (no runtime `--lora`) → merge first\n  (`model.save_pretrained_merged(...)` or `peft ... merge_and_unload()`), then convert the\n  **merged fp16 model** with `convert_hf_to_gguf.py` (see the \"Building your own GGUF\"\n  section of `minicpm5-deploy-llama-cpp`). That produces a standalone base GGUF, not an\n  adapter.\n\n## Reference\n\n- Training frameworks: **`minicpm5-finetune`** (router) and its sub-skills.\n- Running GGUF: **`minicpm5-deploy-llama-cpp`**.\n- Converter: `convert_lora_to_gguf.py` in [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp).\n","frontmatter":{"name":"minicpm5-finetune-gguf-lora","description":"Fine-tune MiniCPM5-1B into a LoRA adapter and convert it to a GGUF adapter that loads directly into llama.cpp / llama-server and the MiniCPM Desk Pet app's custom-LoRA upload. Use when the user wants \"GGUF LoRA\", \"convert LoRA to GGUF\", \"convert_lora_to_gguf\", a custom persona/skin for the desktop pet, \"桌宠自定义 LoRA\", \"上传 LoRA 到桌宠\", or asks how to take a trained adapter and run it on a GGUF base."},"isInternal":false,"tokens":2306,"sizeBytes":8045},{"name":"SKILL.md","path":"skills/minicpm5-finetune-llamafactory/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-llamafactory/SKILL.md","title":"minicpm5-finetune-llamafactory","category":"anthropic-skill","format":"markdown","content":"# minicpm5-finetune-llamafactory\n\nPath: `skills/minicpm5-finetune-llamafactory/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-llamafactory/SKILL.md)","tokens":300,"sizeBytes":221},{"name":"SKILL.md","path":"skills/minicpm5-finetune-ms-swift/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-ms-swift/SKILL.md","title":"minicpm5-finetune-ms-swift","category":"anthropic-skill","format":"markdown","content":"# minicpm5-finetune-ms-swift\n\nPath: `skills/minicpm5-finetune-ms-swift/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-ms-swift/SKILL.md)","tokens":300,"sizeBytes":209},{"name":"SKILL.md","path":"skills/minicpm5-finetune-trl/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-trl/SKILL.md","title":"minicpm5-finetune-trl","category":"anthropic-skill","format":"markdown","content":"# minicpm5-finetune-trl\n\nPath: `skills/minicpm5-finetune-trl/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-trl/SKILL.md)","tokens":300,"sizeBytes":194},{"name":"SKILL.md","path":"skills/minicpm5-finetune-unsloth/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-unsloth/SKILL.md","title":"minicpm5-finetune-unsloth","category":"anthropic-skill","format":"markdown","content":"# minicpm5-finetune-unsloth\n\nPath: `skills/minicpm5-finetune-unsloth/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-unsloth/SKILL.md)","tokens":300,"sizeBytes":206},{"name":"SKILL.md","path":"skills/minicpm5-finetune-xtuner/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-xtuner/SKILL.md","title":"minicpm5-finetune-xtuner","category":"anthropic-skill","format":"markdown","content":"# minicpm5-finetune-xtuner\n\nPath: `skills/minicpm5-finetune-xtuner/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-xtuner/SKILL.md)","tokens":300,"sizeBytes":203},{"name":"SKILL.md","path":"skills/minicpm5-finetune/SKILL.md","rawUrl":"https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune/SKILL.md","title":"minicpm5-finetune","category":"anthropic-skill","format":"markdown","content":"# minicpm5-finetune\n\nPath: `skills/minicpm5-finetune/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune/SKILL.md)","tokens":300,"sizeBytes":182}],"systemPromptSnippet":"<yakaai_skills repo=\"OpenBMB/MiniCPM\">\n<!-- File: skills/minicpm5-deploy-arclight/SKILL.md (Tokens: ~1497 | Category: anthropic-skill) -->\n# Deploy ArcLight from Source (CPU)\n\nArcLight is a lightweight C/C++ LLM inference framework for unified-memory systems. The recommended path is to build from source, then run a GGUF model with `al-gen`, `al-chat`, or `al-ppl`.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL` | `/path/to/MiniCPM5-2B-Q4_0.gguf` | required |\n| `PROMPT` | `\"Hello!\"` | `\"Hello!\"` |\n| `THREADS` | `4` | choose for the target CPU |\n| `NUMA_MODE` | `none` or `tp` | `none` for first run |\n| `NODES` | `1`, `2`, `4` | `1` with `NUMA_MODE=none` |\n| `MAX_GEN` | `256` | `256` |\n\n## Steps\n\n### 1. Build from source\n\n```bash\ngit clone https://github.com/OpenBMB/ArcLight.git\ncd ArcLight\n\ncmake -B build -DARCLIGHT_BACKEND=AUTO\ncmake --build build --config Release -j 32\n```\n\nUse `ARCLIGHT_BACKEND=AUTO` by default. Set it explicitly only when needed:\n\n- `X86`: force the x86 backend\n- `NEON`: force the ARM NEON backend\n- `NONE`: build without architecture-specific backend code\n\n### 2. Prepare a GGUF model\n\nArcLight uses GGUF checkpoints from `llama.cpp`. The `nnml` backend only loads `f32 / f16 / q4_0 / q8_0 / q6_K / q8_K` tensor types — **`Q4_K_M` is not supported**.\n\nSupported model families: MiniCPM5-2B, Qwen3, Llama2.\n\nFor first validation use the released `Q8_0` (`openbmb/MiniCPM5-2B-GGUF`), or quantize an unreleased `Q4_0` yourself from the F16:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-F16.gguf --local-dir .\nllama-quantize ./MiniCPM5-2B-F16.gguf ./MiniCPM5-2B-Q4_0.gguf Q4_0\n```\n\n### 3A. One-shot generation\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\n### 3B. Interactive chat\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_length 4096 \\\n    --max_gen ${MAX_GEN}\n```\n\nTo seed the first turn:\n\n```bash\n./build/al-chat \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n### 3C. Perplexity\n\n```bash\n./build/al-ppl \\\n    --model \"${MODEL}\" \\\n    --prompt \"Good morning, Miss Lee!\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\n## Many-core CPU / NUMA\n\nStart with single-node mode for correctness:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS}\n```\n\nUse cross-NUMA tensor parallelism on many-core machines:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa tp --nodes ${NODES} \\\n    --threads ${THREADS}\n```\n\nRules:\n\n- `--numa none` requires `--nodes 1`.\n- `--numa tp` requires `--nodes N` where `N > 1`.\n- In the current version, `NODES` should be a power of 2.\n- Choose `THREADS` so it can be evenly divided across `NODES`.\n- `--numa pp` is reserved for future pipeline parallelism and is not implemented.\n\n## Optional memory buffers\n\nIf allocation fails or the model is larger, pass manual buffer sizes:\n\n```bash\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"${PROMPT}\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --w_gb 4 --a_gb 8 --kv_gb 2 --work_gb 2\n```\n\nMeaning:\n\n- `--w_gb`: weight buffer size\n- `--a_gb`: activation buffer size\n- `--kv_gb`: KV cache buffer size\n- `--work_gb`: temporary workspace size\n\nIncrease `--kv_gb` for longer `--max_length`. Increase `--w_gb` for larger models.\n\n## Validate\n\nRun:\n\n```bash\nMODEL=/path/to/MiniCPM5-2B-Q4_0.gguf\nTHREADS=4\n\n./build/al-gen \\\n    --model \"${MODEL}\" \\\n    --prompt \"1+1=?\" \\\n    --numa none --nodes 1 \\\n    --threads ${THREADS} \\\n    --max_gen 64\n```\n\nThe reply should contain `2` or a short explanation that evaluates to `2`.\n\n## Common pitfalls\n\n- **Program aborts in single-node mode**: use `--numa none --nodes 1`.\n- **Tensor parallel mode fails**: use `--numa tp --nodes N` with `N > 1`; in this version, `N` should be a power of 2.\n- **Pipeline parallelism fails**: `--numa pp` is not implemented yet.\n- **Model loading fails**: verify the checkpoint is GGUF and from a supported model family.\n- **Out of memory**: increase `--w_gb`, `--a_gb`, `--kv_gb`, or `--work_gb`; longer contexts usually need a larger KV cache.\n- **Poor CPU throughput**: check `--threads`, NUMA layout, and thread-core bindings; use `--print_binding 1 --print_perf 1` for diagnostics.\n\n## When NOT to use\n\n- Need CUDA server inference -> use a GPU-oriented runtime instead.\n- Need Apple Silicon MLX inference -> use an MLX deployment path instead.\n- Need a desktop GUI -> use a GUI runtime that supports GGUF models.\n- Need pipeline parallelism -> wait for ArcLight `--numa pp` support.\n\n## Reference\n\n- [`Arclight`](https://github.com/OpenBMB/ArcLight)\n- [llama.cpp GGUF documentation](https://github.com/ggml-org/llama.cpp)\n\n<!-- File: skills/minicpm5-deploy-litert/SKILL.md (Tokens: ~1522 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-litert\ndescription: Run MiniCPM5-2B or MiniCPM5-1B on-device with Google's LiteRT-LM runtime — the litert-lm CLI or its OpenAI-compatible server on a desktop, the Kotlin API or the AI Edge Gallery app on Android, the same .litertlm bundle on CPU or GPU. Use when the user says \"LiteRT\", \"LiteRT-LM\", \"litertlm\", \".litertlm\", \"Android\", \"Edge Gallery\", \"on-device app\", or wants one bundle for phones and desktops.\n---\n\n# Deploy MiniCPM5-2B / MiniCPM5-1B with LiteRT-LM (Android / iOS / desktop / IoT)\n\n[LiteRT-LM](https://github.com/google-ai-edge/LiteRT-LM), Google's on-device runtime built on [LiteRT](https://github.com/google-ai-edge/litert) (formerly TensorFlow Lite). One `.litertlm` bundle, CPU or GPU, through a CLI, an OpenAI-compatible server, Python, Kotlin and Swift. The bundles are hosted in `litert-community`. Human-readable reference: [`docs/deployment/litert.md`](../../docs/deployment/litert.md).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `LITERT_REPO` | `litert-community/MiniCPM5-2B` or `litert-community/MiniCPM5-1B` | `litert-community/MiniCPM5-2B` |\n| `FILE` | `MiniCPM5-2B_int4.litertlm` (1.55 GB, the phone file) or `MiniCPM5-2B_int8.litertlm` (2.60 GB, reasoning that has to finish); 1B: `minicpm_wi4b32_wi8_afp32.litertlm` (CPU) or `minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm` (GPU) | `MiniCPM5-2B_int4.litertlm` |\n| `BACKEND` | `cpu` or `gpu` | `cpu` |\n| `THINKING` | `true` (reason first) or `false` (direct answer) | `false` |\n| `PROMPT` | `1+1=?` | `1+1=?` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\nuv tool install litert-lm\n```\n\n### 2A. Run a pre-converted bundle (desktop CLI)\n\n```bash\nlitert-lm run --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" \\\n    --backend \"${BACKEND}\" --thinking \"${THINKING:-false}\" --prompt \"${PROMPT}\"\n```\n\nThe first run downloads the file into `~/.litert-lm/cache/huggingface/<repo>/`; later runs skip the download. With thinking on, the reasoning prints between `[thought]` and `[/thought]` and the answer follows; `--thinking-budget 2048` caps the reasoning (4096 for math); `THINKING=true` makes the model reason first. Sampling as OpenBMB recommends: `--top-k 40 --top-p 0.95 --temperature 1.0` (the CLI's default top-k is 1 = greedy, so temperature alone changes nothing).\n\n### 2B. Android\n\n- **No code**: the [AI Edge Gallery](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery) app, Model manager → **+** → **Import from HF** → paste the file's Hugging Face link.\n- **Your own app**: `implementation(\"com.google.ai.edge.litertlm:litertlm-android:0.17.0\")` from Google Maven, the `libOpenCL.so` `<uses-native-library>` entries in the manifest for the GPU, then:\n\n```kotlin\nval engine = Engine(EngineConfig(modelPath = path, backend = Backend.GPU(), cacheDir = context.cacheDir.absolutePath))\nengine.initialize()\nengine.createConversation(ConversationConfig(maxOutputToken = 1024)).use { conversation ->\n    val reply = conversation.sendMessage(\"1+1=?\")\n    println(reply); println(reply.channels[\"thought\"])\n}\n```\n\n`ConversationConfig(thinkingConfig = ThinkingConfig(enableThinking = false))` turns thinking off. Full snippet, manifest and the Galaxy S26 check: the cookbook.\n\n### 2C. OpenAI-compatible server (desktop)\n\n```bash\nlitert-lm import --from-huggingface-repo=\"${LITERT_REPO}\" \"${FILE}\" minicpm5-2b\nlitert-lm serve --host 127.0.0.1 --port 9379\ncurl http://127.0.0.1:9379/v1/chat/completions -H \"Content-Type: application/json\" \\\n    -d '{\"model\": \"minicpm5-2b\", \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}], \"reasoning_effort\": \"none\"}'\n```\n\n`reasoning_effort` `none` = direct answer, any other value = thinking on; the cap field is `max_completion_tokens`; `top_k` is accepted alongside `temperature` / `top_p` / `seed`; `\"stream\": true` streams ([server guide](https://developers.google.com/edge/litert-lm/cli/openai_server)).\n\n### 3. Validate\n\nThe reply contains `2` for `1+1=?`: with `--thinking false` it is the whole reply; with thinking on it follows `[/thought]`. Through the server (2C), the same check is HTTP 200 with `choices[0].message.content` containing `2`, and `GET /v1/models` lists `minicpm5-2b`.\n\n## Common pitfalls\n\n- **Thinking on the int4 file** runs long chains: pass `--thinking false` for direct answers, or use the int8 file when the reasoning has to complete.\n- **int8 on iOS needs the increased-memory-limit entitlement** (2.33 GB weight section; the default entitlements do not map it). int4's 1.28 GB main section is under the limit (checked on iPhone 17 Pro, GPU and CPU).\n- **First run per backend is slow**: an XNNPACK weight cache is written beside the file on the CPU, kernels are compiled on the GPU (`--cache no` to skip the disk cache).\n- **Android GPU without the manifest entries**: `initialize()` succeeds, the first message fails with `Can not find OpenCL library on this device`.\n- **Temperature without top-k does nothing**: pass `--top-k` above 1 together with `--top-p` / `--temperature` (the default is greedy).\n- **Scripts**: the CLI reads stdin; redirect it (`< /dev/null`) when running from a job with an open pipe.\n\n## Reference\n\n[`docs/deployment/litert.md`](../../docs/deployment/litert.md)\n\n<!-- File: skills/minicpm5-deploy-llama-cpp/SKILL.md (Tokens: ~1348 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-llama-cpp\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with llama.cpp using the released GGUF artifacts (F16 / Q8_0 / Q4_K_M). Use when the user wants CPU-only / consumer-GPU / cross-platform native deployment, asks for \"llama.cpp\", \"llama-cli\", \"llama-server\", \"GGUF\", or has no Python available.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with llama.cpp\n\nCPU / edge / consumer-GPU deployment via the released GGUF artifacts. The artifacts work directly with vanilla `llama.cpp` and every downstream runtime (Ollama / LM Studio / `llama-cpp-python`).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `NGL` | `99` (all layers on GPU) / `0` (CPU only) | `99` if NVIDIA GPU, else `0` |\n| `CTX` | `8192` (default) up to `131072` (128 K) | `8192` |\n\n## Steps\n\n### 1. Install llama.cpp\n\n```bash\n# macOS\nbrew install llama.cpp\n\n# Linux / cross-platform: pre-built binary\ncurl -fsSL https://github.com/ggerganov/llama.cpp/releases/latest/download/llama-cli-linux.tar.gz | tar -xz\n# OR build from source:\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git && cd llama.cpp\nmkdir build && cd build\ncmake .. -DGGML_CUDA=ON -DCMAKE_BUILD_TYPE=Release    # CPU-only: omit GGML_CUDA=ON\ncmake --build . --config Release -j $(nproc) --target llama-cli llama-server\n```\n\n### 2. Download the GGUF\n\n```bash\nmkdir -p ~/minicpm5 && cd ~/minicpm5\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n```\n\n### 3a. Interactive chat (CLI)\n\n```bash\nllama-cli -m MiniCPM5-2B-${QUANT}.gguf \\\n    -n 2048 --temp 1.0 --top-p 0.95 --min-p 0.0 -ngl ${NGL} -c ${CTX}\n```\n\n### 3b. OpenAI-compatible HTTP server\n\n```bash\nllama-server -m MiniCPM5-2B-${QUANT}.gguf \\\n    --port 8080 -ngl ${NGL} -c ${CTX} --jinja\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:8080/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"min_p\": 0.0, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nIn llama.cpp, the default `min_p=0.05` can lead to repetitive output: it filters out tokens whose probability is below 5% of the highest-probability token, potentially discarding the exact tokens needed to break out of a repetition loop. To prevent this, we set `min_p=0.0`.\n\n## Sampling defaults\n\n| Mode | `--temp` | `--top-p` | `--min-p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | 1.0 | 0.95 | 0.0 |\n| MiniCPM5-1B Think | 0.9 | 0.95 | 0.0 |\n| MiniCPM5-1B No-think | 0.7 | 0.95 | 0.0 |\n\n## Choosing a quant\n\n| Quant | Disk | Quality |\n| --- | --- | --- |\n| F16 | 5.04 GB | reference |\n| Q8_0 | 2.68 GB | ~indistinguishable from F16 |\n| **Q4_K_M** | **1.56 GB** | small drop, ideal for laptops |\n\n## Common pitfalls\n\n- **Slow on CPU + large context**: drop `-c 131072` to `-c 8192` if you don't need 128 K.\n\n## Building your own GGUF (advanced)\n\nIf you've trained your own MiniCPM5-1B or MiniCPM5-2B variant, build a GGUF with:\n\n```bash\npython convert_hf_to_gguf.py /path/to/your-fp16-hf --outfile out/F16.gguf --outtype f16\nllama-quantize out/F16.gguf out/Q4_K_M.gguf Q4_K_M\n```\n\nTrained a **LoRA adapter** (not a full model) and want to apply it at runtime with `--lora` instead of baking it in? Convert it to a GGUF adapter — see **`minicpm5-finetune-gguf-lora`**.\n\n## When NOT to use\n\n- NVIDIA GPU + want OpenAI-compatible serving → `minicpm5-deploy-vllm`\n- Apple Silicon native → `minicpm5-deploy-mlx` is faster\n- Just want one-line desktop run → `minicpm5-deploy-ollama`\n- Want a desktop GUI → `minicpm5-deploy-lmstudio`\n\n## Reference\n\n[`docs/deployment/llama_cpp.md`](../../docs/deployment/llama_cpp.md)\n\n<!-- File: skills/minicpm5-deploy-lmstudio/SKILL.md (Tokens: ~1342 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-lmstudio\ndescription: Run MiniCPM5-1B or MiniCPM5-2B in LM Studio (desktop GUI) using either the GGUF runtime (cross-platform) or the MLX runtime (Apple Silicon, faster). Includes OpenAI-compatible local server. Use when the user mentions \"LM Studio\", desktop GUI inference, \"lms\" CLI, or wants a no-code chat UI for MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with LM Studio\n\nDesktop GUI + OpenAI-compatible local server. On Apple Silicon ships **two runtimes**:\n\n| Runtime | Format | When to use |\n| --- | --- | --- |\n| **GGUF** (llama.cpp engine) | F16 / Q8_0 / Q4_K_M | cross-platform, same artifact as Ollama; Q4_K_M build |\n| **MLX** (Apple Silicon only) | bf16 / 4-bit | ~60 % faster, automatic think/answer split via `reasoning_content`; Q4 build |\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| Runtime | `gguf` or `mlx` | `mlx` on Apple Silicon, `gguf` elsewhere |\n| `QUANT` | `Q4_K_M` (GGUF) or `4bit` (MLX) | `Q4_K_M` / `4bit` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install LM Studio + complete onboarding\n\n```bash\nbrew install --cask lm-studio\nopen -a \"LM Studio\"     # accept EULA + pick model source\n```\n\n> ⚠️ The first launch MUST be GUI — `lms` (CLI) refuses with `Cannot find LM Studio installation` until LM Studio has run interactively at least once.\n\n### 2A. GGUF runtime path\n\n```bash\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF\nhuggingface-cli download openbmb/MiniCPM5-2B-GGUF MiniCPM5-2B-${QUANT}.gguf \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-GGUF/\n\nLMS=\"/Applications/LM Studio.app/Contents/Resources/app/.webpack/lms\"\n\"$LMS\" server start                              # binds 127.0.0.1:1234\n\"$LMS\" load minicpm5-2b --gpu max --context-length 8192 -y\n\"$LMS\" ps                                        # verify the model is loaded\n```\n\n### 2B. MLX runtime path (Apple Silicon, recommended on Mac)\n\nThe MLX runtime needs an MLX-format checkpoint. The published MiniCPM5-2B MLX repo is [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) (4-bit affine); there is no separate `-bf16` / `-4bit` variant. Either drop that one in as-is, or convert locally from `openbmb/MiniCPM5-2B` (see `minicpm5-deploy-mlx`). Then:\n\n```bash\n# Option A — use the official pre-quantized repo\nhuggingface-cli download openbmb/MiniCPM5-2B-MLX \\\n    --local-dir ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX\n\n# Option B — drop a locally converted directory in\nmkdir -p ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}\ncp -r ./minicpm5-mlx-${QUANT}/* ~/.lmstudio/models/openbmb/MiniCPM5-2B-MLX-${QUANT}/\n\n\"$LMS\" server start\n\"$LMS\" load minicpm5-2b-mlx${QUANT:+-${QUANT}} --gpu max -y\n```\n\n### 3. Validate\n\n```bash\ncurl http://127.0.0.1:1234/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply.\n\nFor the MLX runtime, a think prompt produces output split into `message.reasoning_content` (the `<think>` block) and `message.content` (the final answer) automatically — that's an MLX-runtime feature, not a model setting.\n\n## Think vs nothink (MiniCPM5-1B)\n\nLM Studio 0.4.13's chat-completion endpoint does **not** propagate `chat_template_kwargs.enable_thinking` to the GGUF runtime. Instead:\n\n- **Default = think mode** for both runtimes.\n- **For nothink** with the GGUF runtime, prepend the closing think block manually:\n  ```json\n  \"messages\": [\n    {\"role\":\"user\",\"content\":\"1+1=?\"},\n    {\"role\":\"assistant\",\"content\":\"<think>\\n\\n</think>\\n\\n\"}\n  ]\n  ```\n  and the model continues from there.\n- **MLX runtime**: think/answer are auto-split, you don't need to do anything.\n\n## Common pitfalls\n\n- **MLX runtime not available**: only on Apple Silicon. On Intel Mac / Windows / Linux LM Studio, only the GGUF runtime works.\n\n## When NOT to use\n\n- Just want CLI / scripted runs → `minicpm5-deploy-ollama` is leaner\n- Production server → `minicpm5-deploy-vllm`\n- No GUI desired → `minicpm5-deploy-llama-cpp` (`llama-server`)\n\n## Reference\n\n[`docs/deployment/lmstudio.md`](../../docs/deployment/lmstudio.md)\n\n<!-- File: skills/minicpm5-deploy-mlx/SKILL.md (Tokens: ~891 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-mlx\ndescription: Run MiniCPM5-1B or MiniCPM5-2B natively on Apple Silicon with Apple's MLX framework. Use when the user has an Apple Silicon Mac and asks for \"MLX\", \"mlx_lm\", \"mlx_lm.convert\", \"mlx_lm.generate\", or wants the fastest path on Apple Silicon.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with MLX (Apple Silicon)\n\nApple's on-device tensor framework. Highest throughput on M-series. Stays inside one Python process — no separate server, no `llama.cpp` build chain.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MLX_REPO` | `openbmb/MiniCPM5-2B-MLX` (pre-converted 4-bit affine) | required; `openbmb/MiniCPM5-1B-MLX` also works |\n| OR `HF_REPO` + `QUANT` | `openbmb/MiniCPM5-2B`, `4bit` or `bf16` | for local conversion |\n| `MAX_TOKENS` | `200` | `200` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"mlx-lm>=0.31\" \"gguf\"\n```\n\n### 2A. Use a pre-converted MLX repo (recommended when available)\n\n```bash\nmlx_lm.generate --model \"${MLX_REPO}\" \\\n    --prompt \"<|im_start|>user\n1+1=?<|im_end|>\n<|im_start|>assistant\n\" \\\n    --max-tokens ${MAX_TOKENS} --temp 1.0 --top-p 0.95\n```\n\n### 2B. Convert from a HF checkpoint locally (advanced)\n\nUse `mlx_lm.convert` only if you have a self-trained HF fp16 checkpoint:\n\n```bash\nHF=/path/to/your-fp16-hf\n\n# Convert: bf16\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-bf16\n\n# Convert: 4-bit (smaller / faster)\nmlx_lm.convert --hf-path \"$HF\" --mlx-path ./minicpm5-mlx-q4 -q --q-bits 4\n```\n\nThen run as in 2A.\n\n### 3. Validate\n\nThe reply should contain `\"2\"` for `1+1=?`.\n\n## OpenAI-compatible server (mlx-lm)\n\n```bash\nmlx_lm.server --model \"${MLX_REPO}\" --host 127.0.0.1 --port 8000\n\ncurl http://127.0.0.1:8000/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"default\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\n## Common pitfalls\n\n- **Slow first generate**: MLX JIT-compiles kernels on first call (~5-10 s); subsequent calls hit the warm cache.\n- **Model runs past `<|im_end|>`**: only happens on `mlx-lm < 0.31` (older versions ignore multi-id `eos_token_id` lists). Upgrade, or pass `--extra-eos-token \"<|im_end|>\"` as a manual override — `<|im_end|>` is token id 130073 and is already listed in `generation_config.json` on 0.31+.\n\n## When NOT to use\n\n- Not on Apple Silicon → `minicpm5-deploy-llama-cpp` (CPU/CUDA) or `minicpm5-deploy-vllm` (CUDA)\n- Want a desktop GUI → `minicpm5-deploy-lmstudio` (LM Studio bundles an MLX runtime)\n- Want one-line CLI run → `minicpm5-deploy-ollama`\n\n## Reference\n\n[`docs/deployment/mlx.md`](../../docs/deployment/mlx.md)\n\n<!-- File: skills/minicpm5-deploy-ollama/SKILL.md (Tokens: ~1117 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-ollama\ndescription: Run MiniCPM5-1B or MiniCPM5-2B via Ollama on macOS / Linux laptop using the released GGUF. Use when the user wants \"ollama run\", \"ollama pull\", a Modelfile-driven setup, or one-line laptop deployment.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with Ollama\n\nOne-binary, no-Python laptop deployment. Consumes the released GGUF.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `GGUF_REPO` | `openbmb/MiniCPM5-2B-GGUF` | required; `openbmb/MiniCPM5-1B-GGUF` also works |\n| `QUANT` | `Q4_K_M` (1.56 GB, recommended) / `Q8_0` (2.68 GB) / `F16` (5.04 GB) | `Q4_K_M` |\n| `MODEL_NAME` | `minicpm5-2b` | `minicpm5-2b` |\n\n## Steps\n\n### 1. Install Ollama (once)\n\n```bash\nbrew install ollama                                # macOS\n# or:\ncurl -fsSL https://ollama.com/install.sh | sh      # Linux\n\nOLLAMA_FLASH_ATTENTION=1 OLLAMA_KV_CACHE_TYPE=q8_0 ollama serve &\n```\n\n### 2. Download the GGUF + write Modelfile\n\n```bash\nmkdir -p ~/${MODEL_NAME} && cd ~/${MODEL_NAME}\n\nhuggingface-cli download ${GGUF_REPO} MiniCPM5-2B-${QUANT}.gguf --local-dir .\n\ncat > Modelfile <<EOF\nFROM ./MiniCPM5-2B-${QUANT}.gguf\n\n# MiniCPM5 basic chat template\nTEMPLATE \"\"\"{{- if .Messages -}}\n{{- range .Messages -}}\n<|im_start|>{{ .Role }}\n{{ .Content }}<|im_end|>\n{{ end -}}\n<|im_start|>assistant\n{{ end -}}\"\"\"\n\nPARAMETER stop \"<|im_end|>\"\nPARAMETER stop \"</s>\"\n\n# Defaults tuned for think mode\nPARAMETER temperature 1.0\nPARAMETER top_p 0.95\nPARAMETER num_ctx 8192\nEOF\n```\n\n### 3. Create + run\n\n```bash\nollama create ${MODEL_NAME} -f Modelfile\nollama run ${MODEL_NAME}\n```\n\n### 4. Validate via OpenAI-compatible API\n\n```bash\ncurl http://localhost:11434/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"minicpm5-2b\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64\n    }'\n```\n\nExpected: `\"2\"` in the reply. \n\n## MiniCPM5-1B Think vs No-think\n\nA MiniCPM5-1B Modelfile configured with `temperature=0.7, top_p=0.95` defaults to **no-think**. To switch a single conversation to **think** mode, override the sampling params at request time:\n\n```bash\nollama run minicpm5-1b --temperature 0.9 --top-p 0.95\n```\n\nOr bake it into a separate model tag by raising the temperature line (top_p stays 0.95):\n\n```Modelfile\nPARAMETER temperature 0.9\nPARAMETER top_p 0.95\n```\n\nThen `ollama create minicpm5-1b-think -f Modelfile.think`.\n\nOllama 0.24 does **not** directly evaluate the GGUF-embedded Jinja chat template. It maps recognized Jinja templates to built-in Go templates, so MiniCPM5 uses the explicit Go `TEMPLATE` block above. To enter the MiniCPM5-2B think path, use raw mode and prepend `<think>\\n` manually:\n\n```bash\ncurl http://localhost:11434/api/generate -d '{\n    \"model\": \"minicpm5-2b\",\n    \"raw\": true,\n    \"prompt\": \"<|im_start|>user\\n鸡兔同笼…<|im_end|>\\n<|im_start|>assistant\\n<think>\\n\",\n    \"options\": {\"temperature\": 1.0, \"top_p\": 0.95}\n}'\n```\n\n## Common pitfalls\n\n- **`Error: invalid file magic`**: corrupted download. Re-run `huggingface-cli download`.\n\n## When NOT to use\n\n- Highest throughput on Mac → `minicpm5-deploy-mlx` (Q4 build)\n- GUI experience → `minicpm5-deploy-lmstudio`\n- NVIDIA GPU production → `minicpm5-deploy-vllm`\n\n## Reference\n\n[`docs/deployment/ollama.md`](../../docs/deployment/ollama.md)\n\n<!-- File: skills/minicpm5-deploy-sglang/SKILL.md (Tokens: ~1551 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-sglang\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via SGLang as an OpenAI-compatible HTTP server with RadixAttention prefix cache and built-in MiniCPM5 tool-call parsing. Use when the user asks for \"SGLang\", \"RadixAttention\", \"prefix cache\", batch evaluation, tool calling, or wants a high-concurrency NVIDIA-GPU server alternative to vLLM.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with SGLang\n\nOpenAI-compatible server with RadixAttention prefix cache. Best fit for **tool calling**, batched eval pipelines, and high-concurrency serving.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B`| required; |\n| `PORT` | `30000` | `30000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TOOL_PARSER` | `minicpm5` | `minicpm5`; use `auto` if you want template detection |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"sglang[srt]>=0.5.16\"          # latest, requires CUDA 13.x driver\n# pip install \"sglang==0.5.6.post3\"        # fallback for CUDA 12.x driver hosts\n```\n\n**For tool calling, install from `main`** — the `minicpm5` parser ([PR #25600](https://github.com/sgl-project/sglang/pull/25600), merged 2026-05-22) is not in any pip release yet (`v0.5.12.post1` was branched earlier). Plain chat works on the pip release; only `--tool-call-parser minicpm5` needs `main`:\n\n```bash\npip install \"git+https://github.com/sgl-project/sglang.git@main#subdirectory=python\"\n```\n\n### 2. Recommended runtime env vars\n\n```bash\nexport VLLM_WORKER_MULTIPROC_METHOD=spawn\nexport SGLANG_ALLOW_OVERWRITE_LONGER_CONTEXT_LEN=1\nexport SGLANG_DISABLE_CUDNN_CHECK=1\n```\n\n### 3. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT}\n```\n\nWait for `The server is fired up and ready to roll!` .\n\n**Speculative decoding (DSpark)**: we also release [MiniCPM5-2B-DSpark](https://huggingface.co/openbmb/MiniCPM5-2B-DSpark), a DSpark draft model trained for MiniCPM5-2B. Enable it in SGLang to accelerate decoding while keeping the target model's outputs unchanged:\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} python -m sglang.launch_server \\\n    --model-path \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --context-length ${CTX_LEN} \\\n    --mem-fraction-static ${MEM_FRAC} \\\n    --tool-call-parser ${TOOL_PARSER} \\\n    --host 0.0.0.0 \\\n    --port ${PORT} \\\n    --speculative-algorithm DSPARK \\\n    --speculative-draft-model-path openbmb/MiniCPM5-2B-DSpark \\\n    --speculative-dspark-block-size 7\n```\n\n### 4. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains `\"2\"`.\n\n## Tool calling\n\nSGLang includes a native MiniCPM5 XML tool-call parser. Keep `TOOL_PARSER=minicpm5` (or `auto`) when launching, then send an OpenAI-style tools request:\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n        \"tools\": [{\n            \"type\": \"function\",\n            \"function\": {\n                \"name\": \"get_weather\",\n                \"description\": \"Get current weather for a city\",\n                \"parameters\": {\n                    \"type\": \"object\",\n                    \"properties\": {\"city\": {\"type\": \"string\"}},\n                    \"required\": [\"city\"]\n                }\n            }\n        }],\n        \"tool_choice\": \"auto\",\n        \"temperature\": 1.0, \"top_p\": 0.95,\n        \"max_tokens\": 256\n    }'\n```\n\n## Offline / batched (Engine API)\n\n```python\nimport sglang as sgl\n\nllm = sgl.Engine(model_path=\"${MODEL_PATH}\", tp_size=1, mem_fraction_static=0.8, context_length=131072)\n\noutputs = llm.generate(\n    [\"用一句话解释什么是 GQA。\"],\n    sampling_params={\"temperature\": 1.0, \"top_p\": 0.95, \"max_new_tokens\": 1024,\n                     \"skip_special_tokens\": False},\n)\nprint(outputs)\n```\n\n## Common pitfalls\n\n- **`GLIBCXX_3.4.31 not found`** (only if you hit it — not universal): conda Python ships an older `libstdc++` than the SGLang wheel was built against. Force-load the system one: `LD_PRELOAD=/lib/x86_64-linux-gnu/libstdc++.so.6 python -m sglang.launch_server ...`\n\n## When NOT to use\n\n- Production serving without batch-eval needs → `minicpm5-deploy-vllm` is simpler\n- One-shot Python → `minicpm5-deploy-transformers`\n- No NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n\n## Reference\n\n[`docs/deployment/sglang.md`](../../docs/deployment/sglang.md)\n\n<!-- File: skills/minicpm5-deploy-transformers/SKILL.md (Tokens: ~946 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-transformers\ndescription: Run MiniCPM5-1B or MiniCPM5-2B with Hugging Face Transformers for one-shot Python generation on GPU (bfloat16) or CPU (float32). Use when the user wants a quick Python script, no server, no extra deps, or asks for \"Transformers\", \"AutoModelForCausalLM\", \"model.generate\" with MiniCPM5.\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with HF Transformers\n\nOne-shot Python generation. No server. Works on a single GPU (bfloat16) or CPU only (fp32).\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` or local dir | required; `openbmb/MiniCPM5-1B` also works |\n| `MODE` | `think` or `nothink` (`nothink` is 1B-only) | `think` |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install -U \"transformers>=5.6,<6\" \"torch>=2.11\" accelerate     # latest (CUDA 13.x driver hosts)\n# pip install -U \"transformers==4.57.3\" \"torch==2.7.1\" accelerate  # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Run\n\n```python\nimport torch\nfrom transformers import AutoModelForCausalLM, AutoTokenizer\n\nmodel_path = \"${MODEL_PATH}\"      # ← replace\ntok = AutoTokenizer.from_pretrained(model_path)\nmodel = AutoModelForCausalLM.from_pretrained(\n    model_path,\n    torch_dtype=torch.bfloat16,    # CPU users: torch.float32 + device_map=\"cpu\"\n    device_map=\"auto\",\n).eval()\n\nmessages = [{\"role\": \"user\", \"content\": \"用一句话解释什么是 GQA。\"}]\ninputs = tok.apply_chat_template(\n    messages,\n    add_generation_prompt=True,\n    enable_thinking=True,\n    return_tensors=\"pt\",\n    return_dict=True,\n).to(model.device)\n\nwith torch.no_grad():\n    out = model.generate(\n        **inputs,\n        max_new_tokens=1024,\n        do_sample=True,\n        temperature=1.0,\n        top_p=0.95,\n    )\n\nprompt_len = inputs[\"input_ids\"].shape[-1]\nprint(tok.decode(out[0][prompt_len:], skip_special_tokens=True))\n```\n\nFor CPU only: change `torch_dtype=torch.float32, device_map=\"cpu\"`. Keep `enable_thinking=True` and `temperature=1.0` for MiniCPM5-2B; use `enable_thinking=False` and `temperature=0.7` only for MiniCPM5-1B No-think mode.\n\n## Sampling defaults\n\n| Mode | `enable_thinking` | `temperature` | `top_p` |\n| --- | --- | --- | --- |\n| MiniCPM5-2B Think | `True` | 1.0 | 0.95 |\n| MiniCPM5-1B Think | `True` | 0.9 | 0.95 |\n| MiniCPM5-1B No-think | `False` | 0.7 | 0.95 |\n\n## Validate\n\nA coherent answer to `1+1=?` (e.g. `\"2\"` or `\"答案是 2\"`).\n\n## LoRA inference\n\n```python\nfrom peft import PeftModel\nbase = AutoModelForCausalLM.from_pretrained(model_path, torch_dtype=torch.bfloat16, device_map=\"auto\")\nmodel = PeftModel.from_pretrained(base, \"/path/to/adapter\").eval()\n```\n\nAdapters from any of the `minicpm5-finetune-*` skills load directly with no surgery.\n\n## When NOT to use\n\n- Need an OpenAI-compatible HTTP server → `minicpm5-deploy-vllm` or `minicpm5-deploy-sglang`\n- Apple Silicon → `minicpm5-deploy-mlx` is faster\n- CPU-only or low-VRAM laptop → `minicpm5-deploy-llama-cpp` with Q4_K_M is faster\n\n## Reference\n\n[`docs/deployment/transformers.md`](../../docs/deployment/transformers.md)\n\n<!-- File: skills/minicpm5-deploy-vllm-ascend/SKILL.md (Tokens: ~1933 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-vllm-ascend\ndescription: Deploy MiniCPM5-2B with vLLM on Huawei Ascend NPU using vLLM-Ascend. Use when the user mentions vLLM-Ascend, Ascend NPU, Huawei Ascend, CANN, torch_npu, davinci devices, or wants an OpenAI-compatible MiniCPM5 server on Ascend hardware.\n---\n\n# Deploy MiniCPM5-2B with vLLM-Ascend\n\nUse this skill for the BF16 / FP16 `openbmb/MiniCPM5-2B` checkpoint on Huawei Ascend NPU. `vllm-ascend` is installed alongside a matching vLLM release and supplies the Ascend backend.\n\n## Required input\n\n| Variable | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required |\n| `PORT` | `8000` | `8000` |\n| `ASCEND_RT_VISIBLE_DEVICES` | `0` | `0` |\n| `CTX_LEN` | `32768` | `32768`; the native maximum is `131072` |\n| `MEM_FRAC` | `0.85` | `0.85` |\n| `TP_SIZE` | `1` | number of visible devices |\n\nAccept either the Hugging Face id or a local directory containing the MiniCPM5-2B model files.\n\n## Prerequisites\n\nUse a compatible Linux Ascend environment with CANN, PyTorch, and TorchNPU installed. Keep the vLLM and vLLM-Ascend versions matched. The basic serving recipe below follows the tested `vllm==0.18.0` / `vllm-ascend==0.18.0` pair.\n\nDevice visibility is controlled by `ASCEND_RT_VISIBLE_DEVICES`; `CUDA_VISIBLE_DEVICES` does not select Ascend devices.\n\n## Install\n\nThe prebuilt image is the simplest option for a compatible Ascend host. This is a minimal single-device example; adjust device nodes and driver mounts for the host hardware and image documentation.\n\n```bash\ndocker run -it --net=host --shm-size=1g \\\n    --device /dev/davinci0 --device /dev/davinci_manager \\\n    --device /dev/devmm_svm --device /dev/hisi_hdc \\\n    -v /usr/local/Ascend/driver:/usr/local/Ascend/driver \\\n    -v /usr/local/dcmi:/usr/local/dcmi \\\n    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \\\n    quay.io/ascend/vllm-ascend:v0.18.0\n```\n\nFor an existing CANN environment:\n\n```bash\npip install vllm==0.18.0 vllm-ascend==0.18.0 \\\n    --extra-index-url https://download.pytorch.org/whl/cpu\n```\n\nFor built-in MiniCPM5 tool-call parsing, use the matching `vllm==0.23.0` / `vllm-ascend==0.23.0` stack described in [Tool calling](#tool-calling). The parser responses in the cookbook were produced with the upstream parser file on an older vLLM and a compatibility shim because the 0.23.0 stack was not available on the test hardware.\n\n## Prepare the environment\n\nRun the CANN and NNAL setup scripts before starting vLLM. Change the paths if those components are installed elsewhere.\n\n```bash\nsource /usr/local/Ascend/ascend-toolkit/set_env.sh\nsource /usr/local/Ascend/nnal/atb/set_env.sh\n\nexport ASCEND_RT_VISIBLE_DEVICES=${ASCEND_RT_VISIBLE_DEVICES:-0}\nexport LD_LIBRARY_PATH=$(python3 -c \"import torch,os;print(os.path.dirname(torch.__file__))\")/lib:\\\n$(python3 -c \"import torch_npu,os;print(os.path.dirname(torch_npu.__file__))\")/lib:$LD_LIBRARY_PATH\n```\n\nCheck the driver and Python packages before loading the model:\n\n```bash\nnpu-smi info\npython3 -c \"import torch, torch_npu; print(torch_npu.npu.is_available(), torch_npu.npu.device_count())\"\n```\n\nIf `npu-smi` reports a missing driver library, add the corresponding driver `lib64` directories to `LD_LIBRARY_PATH` according to the host installation.\n\n## Launch the server\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --port ${PORT:-8000}\n```\n\nSet `TP_SIZE` to the number of devices in `ASCEND_RT_VISIBLE_DEVICES` when using tensor parallelism.\n\n## Validate chat completions\n\nMiniCPM5-2B is think-only. Use `temperature=1.0`, `top_p=0.95`, and `enable_thinking=true`:\n\n```bash\ncurl http://localhost:${PORT:-8000}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0,\n        \"top_p\": 0.95,\n        \"max_tokens\": 256,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpect HTTP 200 and an answer containing `2` in `choices[0].message.content`. Leave enough `max_tokens` for the model's reasoning text.\n\n## Tool calling\n\nMiniCPM5-2B emits XML-style tool calls. This section requires matching vLLM and vLLM-Ascend `0.23.0`; that release includes the `minicpm5` parser in the OpenAI server layer, so no parser plugin file is needed:\n\n```bash\npip install vllm==0.23.0\npip install vllm-ascend==0.23.0 \\\n    --extra-index-url https://mirrors.huaweicloud.com/ascend/repos/pypi\n```\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN:-32768} \\\n    --gpu-memory-utilization ${MEM_FRAC:-0.85} \\\n    --tensor-parallel-size ${TP_SIZE:-1} \\\n    --trust-remote-code \\\n    --enable-auto-tool-choice \\\n    --tool-call-parser minicpm5 \\\n    --port ${PORT:-8000}\n```\n\nWhen sending `tools`, use both server flags above and the same 2B sampling settings:\n\n```json\n{\n  \"model\": \"MiniCPM5-2B\",\n  \"messages\": [{\"role\": \"user\", \"content\": \"What is the weather in Beijing?\"}],\n  \"tools\": [{\n    \"type\": \"function\",\n    \"function\": {\n      \"name\": \"get_weather\",\n      \"description\": \"Get current weather for a city\",\n      \"parameters\": {\n        \"type\": \"object\",\n        \"properties\": {\"city\": {\"type\": \"string\"}},\n        \"required\": [\"city\"]\n      }\n    }\n  }],\n  \"tool_choice\": \"auto\",\n  \"temperature\": 1.0,\n  \"top_p\": 0.95,\n  \"max_tokens\": 512,\n  \"chat_template_kwargs\": {\"enable_thinking\": true}\n}\n```\n\n## Common failures\n\n- `torch_npu` cannot import: source the CANN/NNAL environment and check that the installed PyTorch, TorchNPU, CANN, vLLM, and vLLM-Ascend versions match the supported stack.\n- No devices are visible: check `npu-smi info`, `ASCEND_RT_VISIBLE_DEVICES`, and the Docker device/driver mounts. Do not use `CUDA_VISIBLE_DEVICES` for this backend.\n- HBM allocation or 128K startup failure: lower `--max-model-len`, then lower `--gpu-memory-utilization`.\n- Tensor-parallel initialization failure: make `--tensor-parallel-size` match the number of visible Ascend devices.\n- Tool requests fail with an auto-tool-choice error: launch with both `--enable-auto-tool-choice` and `--tool-call-parser minicpm5`.\n\n## Reference\n\nSee [`docs/deployment/vllm-ascend.md`](../../docs/deployment/vllm-ascend.md) for the human-readable cookbook and the [official vLLM-Ascend installation guide](https://docs.vllm.ai/projects/ascend/en/v0.23.0/installation.html) for hardware-specific setup.\n\n<!-- File: skills/minicpm5-deploy-vllm/SKILL.md (Tokens: ~1055 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy-vllm\ndescription: Serve MiniCPM5-1B or MiniCPM5-2B via vLLM as an OpenAI-compatible HTTP server. Use when the user wants high-throughput production serving on NVIDIA GPU, asks for \"vLLM\", \"OpenAI server\", \"REST API for MiniCPM5\", or \"production deployment\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B with vLLM\n\nOpenAI-compatible server for the BF16 / FP16 MiniCPM5-1B or MiniCPM5-2B checkpoint.\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `MODEL_PATH` | `openbmb/MiniCPM5-2B` | required; `openbmb/MiniCPM5-1B` also works |\n| `PORT` | `8000` | `8000` |\n| `GPU_ID` | `0` | `0` |\n| `CTX_LEN` | `131072` (128 K) | `131072`; lower if VRAM tight |\n| `MEM_FRAC` | `0.85` | `0.85`; lower on shared GPUs |\n\n## Steps\n\n### 1. Install (once)\n\n```bash\npip install \"vllm>=0.21\"          # latest (CUDA 13.x driver hosts)\n# pip install \"vllm==0.10.1.1\"    # fallback for CUDA 12.x driver hosts\n```\n\n### 2. Launch\n\n```bash\nCUDA_VISIBLE_DEVICES=${GPU_ID} vllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 \\\n    --max-model-len ${CTX_LEN} \\\n    --gpu-memory-utilization ${MEM_FRAC} \\\n    --port ${PORT}\n```\n\nWait for `Application startup complete` in the log.\n\n### 3. Validate\n\n```bash\ncurl http://localhost:${PORT}/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\": \"user\", \"content\": \"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: `choices[0].message.content` contains a coherent answer.\n\n## Sampling defaults\n\n```json\n{\"temperature\": 1.0, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-2B Think\n{\"temperature\": 0.9, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": true}}    // MiniCPM5-1B Think\n{\"temperature\": 0.7, \"top_p\": 0.95, \"chat_template_kwargs\": {\"enable_thinking\": false}}   // MiniCPM5-1B No-think\n```\n\n## Common pitfalls\n\n- **`(free / total) < MEM_FRAC` hard error**: lower `--gpu-memory-utilization` (e.g. 0.5 on a shared GPU).\n- **OOM at startup with 128 K**: drop `--max-model-len` to 32768 or 8192.\n\n## Tool calling (plugin)\n\nThe vLLM-side MiniCPM5 XML parser ([PR #43175](https://github.com/vllm-project/vllm/pull/43175)) merged to `main` on 2026-05-27 but is **not in any pip release yet** (`v0.22.0` was cut before the merge). Use the bridge plugin shipped at `tool_parsers/minicpm5xml_tool_parser.py` in this repo:\n\n```bash\nvllm serve \"${MODEL_PATH}\" \\\n    --served-model-name MiniCPM5-2B \\\n    --dtype bfloat16 --max-model-len ${CTX_LEN} --port ${PORT} \\\n    --enable-auto-tool-choice \\\n    --tool-parser-plugin /path/to/MiniCPM/tool_parsers/minicpm5xml_tool_parser.py \\\n    --tool-call-parser minicpm5\n```\n\nDrop `--tool-parser-plugin` once vLLM ships a release containing the parser natively.\n\n## When NOT to use\n\n- One-shot Python script → `minicpm5-deploy-transformers`\n- Apple Silicon / no NVIDIA GPU → `minicpm5-deploy-llama-cpp` / `minicpm5-deploy-mlx`\n- High-concurrency batch eval w/ prefix cache or tool calling → `minicpm5-deploy-sglang`\n\n## Reference\n\n[`docs/deployment/vllm.md`](../../docs/deployment/vllm.md)\n\n<!-- File: skills/minicpm5-deploy/SKILL.md (Tokens: ~2119 | Category: anthropic-skill) -->\n---\nname: minicpm5-deploy\ndescription: Pick the right inference backend for a MiniCPM5-1B or MiniCPM5-2B checkpoint and route to a backend-specific cookbook skill. Use when the user wants to deploy / serve / chat-with / benchmark a MiniCPM5 model and has not yet committed to a specific engine, or when they say \"deploy MiniCPM5\", \"run MiniCPM5\", \"serve MiniCPM5\", \"MiniCPM5 推理\", \"部署 MiniCPM5\".\n---\n\n# Deploy MiniCPM5-1B and MiniCPM5-2B — backend router\n\nYou're being asked to deploy / serve / chat-with a MiniCPM5-1B or MiniCPM5-2B checkpoint. Your job is to **pick exactly one backend skill below** based on the user's hardware, format, and goal, then **invoke that skill** rather than improvising.\n\n## 1. Required input from the user\n\nBefore picking a backend, you MUST know:\n\n| Variable | Example | Where to ask |\n| --- | --- | --- |\n| `MODEL_PATH` | HF id `openbmb/MiniCPM5-2B` or `openbmb/MiniCPM5-1B` **or** a local path | \"Which checkpoint? HF id or local path?\" |\n| Hardware | NVIDIA GPU / Apple Silicon / CPU only | infer from context, otherwise ask |\n| Goal | \"interactive chat\" / \"OpenAI server\" / \"Python script\" / \"benchmark\" | infer from context |\n\n### Available checkpoints on Hugging Face\n\n| Variant | HF repo | Use with |\n| --- | --- | --- |\n| **HF fp16 (recommended)** | [`openbmb/MiniCPM5-2B`](https://huggingface.co/openbmb/MiniCPM5-2B) or [`openbmb/MiniCPM5-1B`](https://huggingface.co/openbmb/MiniCPM5-1B) | `transformers` / `vllm` (no `--quantization`) / `vllm-ascend` / `sglang` / any `minicpm5-finetune-*` |\n| GGUF F16 / Q8_0 / Q4_K_M | [`openbmb/MiniCPM5-2B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-2B-GGUF) or [`openbmb/MiniCPM5-1B-GGUF`](https://huggingface.co/openbmb/MiniCPM5-1B-GGUF) | `minicpm5-deploy-llama-cpp` / `-ollama` / `-lmstudio` |\n| MLX (Apple Silicon) | [`openbmb/MiniCPM5-2B-MLX`](https://huggingface.co/openbmb/MiniCPM5-2B-MLX) or [`openbmb/MiniCPM5-1B-MLX`](https://huggingface.co/openbmb/MiniCPM5-1B-MLX) | `minicpm5-deploy-mlx` |\n| LiteRT-LM `.litertlm` (Android / iOS / desktop / IoT, CPU + GPU) | [`litert-community/MiniCPM5-2B`](https://huggingface.co/litert-community/MiniCPM5-2B) or [`litert-community/MiniCPM5-1B`](https://huggingface.co/litert-community/MiniCPM5-1B) | `minicpm5-deploy-litert` |\n\nIf the user has a local copy, accept any directory path that contains `config.json` and `model.safetensors` (or the equivalent GGUF / MLX layout).\n\n## 2. Decision matrix — pick exactly one\n\n| User says / wants | Hardware | Format | → Skill to invoke |\n| --- | --- | --- | --- |\n| \"Quick Python script\" / \"one-shot generation\" / \"no server\" | any GPU or CPU | HF safetensors | **`minicpm5-deploy-transformers`** |\n| \"OpenAI server\" / \"production serving\" / \"high QPS\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-vllm`** |\n| \"vLLM-Ascend\" / \"Ascend NPU\" / \"CANN\" / \"torch_npu\" | Huawei Ascend NPU | HF safetensors | **`minicpm5-deploy-vllm-ascend`** |\n| \"RadixAttention\" / \"prefix cache\" / \"batched eval\" | NVIDIA GPU | HF safetensors | **`minicpm5-deploy-sglang`** |\n| \"GGUF\" / \"llama.cpp\" / \"llama-cli\" / \"CPU only\" | any CPU + optional GPU | GGUF | **`minicpm5-deploy-llama-cpp`** |\n| \"Ollama\" / \"ollama run\" / \"Modelfile\" | macOS / Linux laptop | GGUF | **`minicpm5-deploy-ollama`** |\n| \"LM Studio\" / \"desktop GUI\" | macOS / Windows / Linux | GGUF or MLX | **`minicpm5-deploy-lmstudio`** |\n| \"MLX\" / \"Apple Silicon native\" / \"fastest on Mac\" | Apple Silicon | MLX | **`minicpm5-deploy-mlx`** |\n| \"Android\" / \"on-device app\" / \"Edge Gallery\" / \"LiteRT\" / \"LiteRT-LM\" / \"litertlm\" / \"Raspberry Pi\" | Android phone, iPhone, desktop or IoT board (CPU or GPU) | LiteRT-LM `.litertlm` | **`minicpm5-deploy-litert`** |\n\nIf the user **has not specified** any of the above and asks \"how do I run this?\":\n\n- **CUDA box, want fastest server**: pick `minicpm5-deploy-vllm`.\n- **Ascend NPU, want an OpenAI-compatible server**: pick `minicpm5-deploy-vllm-ascend`.\n- **CUDA box, want minimal Python**: pick `minicpm5-deploy-transformers`.\n- **Apple Silicon laptop**: pick `minicpm5-deploy-ollama` (easiest) or `minicpm5-deploy-mlx` (fastest).\n- **Android phone, or an on-device app (Android / iOS)**: pick `minicpm5-deploy-litert`.\n- **CPU only / Windows / low-VRAM**: pick `minicpm5-deploy-llama-cpp` (Q4_K_M).\n\n## 3. Invocation contract\n\nOnce you've picked a backend skill, **invoke that skill with `MODEL_PATH` set**. Do NOT inline the backend's commands here — each backend has its own pitfalls (mandatory flags, env-var pins, install order) that the dedicated skill handles. The user explicitly does NOT want you to \"improvise\" — read the picked sub-skill in full first.\n\n## 4. Sanity check after deploy\n\nWhichever backend you pick, after launch run this universal sanity check:\n\n```bash\n# Replace localhost:PORT with the backend's actual port (default in each sub-skill)\ncurl http://localhost:PORT/v1/chat/completions \\\n    -H \"Content-Type: application/json\" \\\n    -d '{\n        \"model\": \"MiniCPM5-2B\",\n        \"messages\": [{\"role\":\"user\",\"content\":\"1+1=?\"}],\n        \"temperature\": 1.0, \"top_p\": 0.95, \"max_tokens\": 64,\n        \"chat_template_kwargs\": {\"enable_thinking\": true}\n    }'\n```\n\nExpected: HTTP 200 with `choices[0].message.content` containing `\"2\"`.\n\n`minicpm5-deploy-litert` also serves this endpoint: after `litert-lm import … minicpm5-2b`, `litert-lm serve` listens on port 9379 ([guide](https://developers.google.com/edge/litert-lm/cli/openai_server)); use `\"model\": \"minicpm5-2b\"`, add `\"reasoning_effort\": \"none\"` for a direct answer, and cap with `max_completion_tokens`.\n\n## 5. Known cross-backend pitfalls\n\nThese are common to multiple backends — surface to the user up front:\n\n- **Think vs no-think**: MiniCPM5-2B uses Think mode with `temperature=1.0, top_p=0.95`. MiniCPM5-1B also supports No-think with `enable_thinking=false` + `temperature=0.7, top_p=0.95`.\n- **128 K context**: `max_position_embeddings=131072`, `rope_theta=5e6`, **no rope-scaling**. Pass `--max-model-len 131072` (vLLM) / `--context-length 131072` (SGLang) / `-c 131072` (llama.cpp) to use the full window. Lower if VRAM is tight.\n- **Untied lm_head**: `tie_word_embeddings=false`. Tools that assume the Llama tied default (e.g. `mlx_lm.convert` < 0.31) will silently drop `lm_head` → output collapses to random tokens. The MLX skill bakes in the fix.\n\n## 6. Don't reinvent: link to the cookbook\n\nEach sub-skill is paired with a one-page cookbook in [`docs/deployment/`](../../docs/deployment/). The skill is the machine-readable shortcut; the cookbook is the human-readable reference. Both are kept in sync.\n\n<!-- File: skills/minicpm5-finetune-gguf-lora/SKILL.md (Tokens: ~2306 | Category: anthropic-skill) -->\n---\nname: minicpm5-finetune-gguf-lora\ndescription: Fine-tune MiniCPM5-1B into a LoRA adapter and convert it to a GGUF adapter that loads directly into llama.cpp / llama-server and the MiniCPM Desk Pet app's custom-LoRA upload. Use when the user wants \"GGUF LoRA\", \"convert LoRA to GGUF\", \"convert_lora_to_gguf\", a custom persona/skin for the desktop pet, \"桌宠自定义 LoRA\", \"上传 LoRA 到桌宠\", or asks how to take a trained adapter and run it on a GGUF base.\n---\n\n# Fine-tune MiniCPM5-1B → GGUF LoRA adapter\n\nThe framework skills (`minicpm5-finetune-*`) all emit a **PEFT adapter** (`adapter_model.safetensors` + `adapter_config.json`). But `llama.cpp` / `llama-server` — and the **MiniCPM Desk Pet** app's custom-LoRA upload — load a **GGUF LoRA adapter** (`--lora some-adapter.gguf`). This skill is the bridge: train a PEFT LoRA, then convert it to GGUF and run/upload it.\n\n> This is the path you want when the base model is served as GGUF (Desk Pet, Ollama, LM Studio, plain `llama-server`). If you serve the fp16 HF base with vLLM / transformers, you don't need GGUF — load the PEFT adapter directly.\n\n## Pipeline overview\n\n```\n  train (any minicpm5-finetune-* skill)        this skill\n ┌────────────────────────────────────┐   ┌──────────────────────────────────┐\n BASE (fp16 HF) ─► adapter_model.safetensors ─► convert_lora_to_gguf.py ─► adapter.gguf\n                   adapter_config.json                                          │\n                                                                  llama-server --lora / Desk Pet upload\n```\n\n## Required input\n\n| Var | Example | Default |\n| --- | --- | --- |\n| `ADAPTER_DIR` | `./runs/minicpm5_unsloth/adapter_final` | required — a PEFT dir with `adapter_config.json` + `adapter_model.safetensors` |\n| `BASE_MODEL` | `openbmb/MiniCPM5-1B` (HF id) **or** a local fp16 HF dir | required — must be the **same base** the adapter was trained on |\n| `OUTTYPE` | `f16` (recommended) / `q8_0` / `bf16` / `f32` | `f16` |\n| `OUT_GGUF` | `./minicpm5-mylora.gguf` | `<ADAPTER_DIR>/adapter_model.f16.gguf` |\n\n> **Don't have an adapter yet?** First run a training skill — start from the router **`minicpm5-finetune`** (or go straight to **`minicpm5-finetune-unsloth`** for single-GPU LoRA). Come back here with `ADAPTER_DIR` pointing at its output.\n\n## Steps\n\n### 1. Get llama.cpp (has the converter)\n\n```bash\ngit clone --depth=1 https://github.com/ggerganov/llama.cpp.git\ncd llama.cpp\npip install -r requirements.txt          # converter deps: torch, safetensors, gguf, transformers\n```\n\n`convert_lora_to_gguf.py` lives at the repo root. You only need the Python script for conversion; build the C++ binaries (step 4) only if you want to test locally.\n\n### 2. Convert the PEFT adapter → GGUF\n\n```bash\npython convert_lora_to_gguf.py \"$ADAPTER_DIR\" \\\n    --base \"$BASE_MODEL\" \\\n    --outtype f16 \\\n    --outfile \"$OUT_GGUF\"\n```\n\n- `--base` only needs the base model's **config** (`config.json`, `tokenizer.json`) — not the weights. A local fp16 HF dir or an HF id both work.\n- If the base is only on the Hub, use `--base-model-id openbmb/MiniCPM5-1B` instead of `--base` (it pulls just the config).\n- Output is a single `.gguf` of roughly the same size as the input `.safetensors` (a 22 MB r=16 adapter → ~22 MB GGUF).\n\n> 🔑 **The #1 gotcha — `base_model_name_or_path` points at the training machine.**\n> PEFT writes the absolute path of the base used during training into `adapter_config.json`\n> (e.g. `/user/.../MiniCPM5-models-fixed/official`). On any other machine the converter\n> can't find it. **Always pass `--base` (or `--base-model-id`) explicitly** to override it —\n> don't rely on whatever is baked into the config. The base you pass MUST match the one you\n> trained on, or the adapter math is meaningless.\n\n### 3. (Optional) Sanity-check the GGUF metadata\n\n```bash\npython -c \"import gguf,sys; r=gguf.GGUFReader('$OUT_GGUF'); print('tensors:', len(r.tensors)); print('arch:', r.get_field('general.architecture').parts[-1].tobytes().decode() if r.get_field('general.architecture') else '?')\"\n```\n\nExpect `general.type = adapter` and a nonzero tensor count. MiniCPM5-1B is a **Llama-architecture** model, so the converter treats it as `llama` — this is correct, not an error.\n\n### 4. Test with llama-server before shipping\n\nYou need a GGUF **base** model too (the adapter is applied on top of it). Grab the released base:\n\n```bash\nhuggingface-cli download openbmb/MiniCPM5-1B-GGUF MiniCPM5-1B-Q8_0.gguf --local-dir .\n```\n\nThen:\n\n```bash\n# CLI\nllama-cli -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" \\\n    -p \"你好\" -n 128 --temp 0.7 --top-p 0.95\n\n# OpenAI-compatible server\nllama-server -m MiniCPM5-1B-Q8_0.gguf --lora \"$OUT_GGUF\" --port 8080 --jinja\n```\n\nIf the persona/behavior you trained shows up, the GGUF is good. If output is identical to the base, the adapter didn't load (check the `--lora` path and that `--base` matched in step 2).\n\n> **Quant compatibility:** a GGUF LoRA built from an fp16 adapter applies fine on top of a\n> quantized base (Q8_0 / Q4_K_M). You do **not** need a separate adapter per base quant.\n\n### 5. Use it in MiniCPM Desk Pet\n\nThe Desk Pet app loads exactly this kind of file. The fp16 GGUF you just built is upload-ready:\n\n1. Open **Settings → MiniCPM → 适配器 (LoRA)**.\n2. Click **上传** (Upload), pick your `.gguf` — the app copies it into its\n   `<userData>/adapters/uploads/` directory and registers it.\n3. Give it a display name + comma-separated aliases (used so you can switch personas by\n   voice/chat, e.g. \"换成 XX\").\n4. Select it in the adapter list to activate. The sidecar reloads `llama-server` with your\n   `--lora`.\n\nConstraints the app enforces (match them or the upload is rejected):\n\n- **Must be a single `.gguf` file** (the GGUF LoRA from step 2 — *not* the `.safetensors`,\n  *not* a merged full model).\n- It is applied on top of whatever MiniCPM5-1B GGUF base the app already runs, so **train\n  against the MiniCPM5-1B base**, not some other model.\n\n## Common pitfalls\n\n- **`FileNotFoundError` / `can't load base model config`** during conversion → the\n  `adapter_config.json` base path doesn't exist locally. Fix with `--base <local-dir>` or\n  `--base-model-id openbmb/MiniCPM5-1B` (step 2 gotcha).\n- **Adapter loads but output is unchanged** → base mismatch (trained on base A, applied on\n  base B), or the persona needs its **system prompt** too — a LoRA biases style but the\n  system prompt still matters. In Desk Pet the persona's system prompt is wired to the\n  adapter; standalone `llama-cli` users must pass it themselves.\n- **`KeyError` on an unknown tensor / target module** → the adapter targeted modules the\n  converter doesn't map. Stick to the standard attention+MLP projections\n  (`q/k/v/o_proj`, `gate/up/down_proj`) when training (all `minicpm5-finetune-*` skills\n  already default to these).\n- **Uploaded `.gguf` but Desk Pet shows nothing** → you uploaded the merged full model or a\n  base GGUF by mistake. A LoRA adapter GGUF is small (tens of MB) and has\n  `general.type = adapter` (step 3).\n\n## When NOT to use\n\n- Serving the fp16 HF base with **vLLM / transformers / SGLang** → load the PEFT adapter\n  directly, skip GGUF. See `minicpm5-deploy-vllm` / `-transformers` / `-sglang`.\n- You want to **bake the LoRA into the weights** (no runtime `--lora`) → merge first\n  (`model.save_pretrained_merged(...)` or `peft ... merge_and_unload()`), then convert the\n  **merged fp16 model** with `convert_hf_to_gguf.py` (see the \"Building your own GGUF\"\n  section of `minicpm5-deploy-llama-cpp`). That produces a standalone base GGUF, not an\n  adapter.\n\n## Reference\n\n- Training frameworks: **`minicpm5-finetune`** (router) and its sub-skills.\n- Running GGUF: **`minicpm5-deploy-llama-cpp`**.\n- Converter: `convert_lora_to_gguf.py` in [ggerganov/llama.cpp](https://github.com/ggerganov/llama.cpp).\n\n<!-- File: skills/minicpm5-finetune-llamafactory/SKILL.md (Tokens: ~300 | Category: anthropic-skill) -->\n# minicpm5-finetune-llamafactory\n\nPath: `skills/minicpm5-finetune-llamafactory/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-llamafactory/SKILL.md)\n\n<!-- File: skills/minicpm5-finetune-ms-swift/SKILL.md (Tokens: ~300 | Category: anthropic-skill) -->\n# minicpm5-finetune-ms-swift\n\nPath: `skills/minicpm5-finetune-ms-swift/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-ms-swift/SKILL.md)\n\n<!-- File: skills/minicpm5-finetune-trl/SKILL.md (Tokens: ~300 | Category: anthropic-skill) -->\n# minicpm5-finetune-trl\n\nPath: `skills/minicpm5-finetune-trl/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-trl/SKILL.md)\n\n<!-- File: skills/minicpm5-finetune-unsloth/SKILL.md (Tokens: ~300 | Category: anthropic-skill) -->\n# minicpm5-finetune-unsloth\n\nPath: `skills/minicpm5-finetune-unsloth/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-unsloth/SKILL.md)\n\n<!-- File: skills/minicpm5-finetune-xtuner/SKILL.md (Tokens: ~300 | Category: anthropic-skill) -->\n# minicpm5-finetune-xtuner\n\nPath: `skills/minicpm5-finetune-xtuner/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune-xtuner/SKILL.md)\n\n<!-- File: skills/minicpm5-finetune/SKILL.md (Tokens: ~300 | Category: anthropic-skill) -->\n# minicpm5-finetune\n\nPath: `skills/minicpm5-finetune/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/OpenBMB/MiniCPM/HEAD/skills/minicpm5-finetune/SKILL.md)\n</yakaai_skills>"}