{"owner":"alibaba","repo":"MNN","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"skills":{"AGENTS.md":"# MNN Project Instructions\n\nMNN is a lightweight deep learning **inference engine** (not a training framework), targeting mobile and server platforms. Supports CNN / Transformer / LLM / Diffusion models. Code must prioritize **performance and binary size**.\n\n## Restricted Access\n\n> The following directories contain internal proprietary code. **Do NOT read, modify, or reference** any files within:\n> - `schema/private/`\n> - `source/internal/`\n\n## Architecture Overview\n\nMNN uses a **graph optimization + heterogeneous backend scheduling** architecture.\n\nTwo inference APIs are available:\n- **Session API** (low-level): `Interpreter → createSession → runSession`, operates on Tensor directly\n- **Module API** (high-level, recommended): `Module::load → onForward(VARP)`, Express-based dynamic graph. Used by LLM / Diffusion and most modern workloads\n\n**Key abstractions** (see corresponding headers under `source/core/`):\n- **Interpreter** / **Session**: model loading and inference session management\n- **Backend** / **Execution**: hardware backend abstraction and per-op implementation (CPU/Metal/CUDA/OpenCL/Vulkan/...)\n- **Tensor**: data container; internally uses NC4HW4 format (channels packed by 4 for SIMD)\n- **Op / Schema**: FlatBuffers-defined operator descriptors (`schema/default/*.fbs`)\n\n**Op registration pattern**: Schema definition → shape inference (`source/shape/`) → Geometry decomposition (optional) → Backend Execution implementation\n\n## LLM Subsystem\n\nMNN supports end-to-end LLM export and inference:\n\n- **Python export** (`transformers/llm/export/`): HuggingFace model → MNN format. Core modules: `llmexport.py` entry point, `utils/model_mapper.py` (model field mapping), `utils/model.py` (unified LlmModel class), `utils/transformers.py` (Attention/Decoder/RoPE export)\n- **C++ inference** (`transformers/llm/engine/`): `llm.cpp` (text inference), `omni.cpp` (multimodal: vision/audio), includes KVCache management and sampling strategies\n\n## Repository Structure\n\n| Directory | Description |\n|-----------|-------------|\n| `include/MNN/` | Public C++ headers |\n| `source/core/` | Inference core (Interpreter, Session, Pipeline, Backend) |\n| `source/backend/` | Hardware backend implementations (cpu, arm82, metal, cuda, opencl, vulkan, ...) |\n| `source/shape/` | Shape inference |\n| `source/geometry/` | Geometry computation (op decomposition) |\n| `express/` | Express API (high-level dynamic graph, VARP) |\n| `schema/default/` | FlatBuffers schema (op definitions) |\n| `tools/converter/` | Model converter (ONNX/TF/Caffe → MNN) |\n| `transformers/llm/` | LLM export (Python) + inference engine (C++) |\n| `transformers/diffusion/` | Diffusion model support |\n| `pymnn/` | Python bindings |\n| `test/` | Test cases |\n| `skills/` | AI Agent Skills |\n\n## Coding Style\n\n- **C++**: Google Style variant, see `.clang-format`. 4-space indent, 120-char line width, attached braces. Class names `PascalCase`, functions `camelCase`, member variables `mCamelCase`. RTTI and exceptions disabled (`-fno-rtti -fno-exceptions`). Default standard: C++11.\n- **Python**: Standard Python conventions\n- **Formatting**: `clang-format -i -style=file <file>`\n\n> ⚠️ **禁止 C++ 全局对象（动态初始化）**:不允许在命名空间作用域定义需要动态初始化的非 POD 对象（如 `static const std::string` / `static const json` / 全局 STL 容器，**头文件中同样禁止**,每个包含它的 TU 都会生成一份）。这类对象产生 `__GLOBAL__sub_I_*` 启动初始化函数，会被手淘等打包静态检查拦截（代码安全风险 + 启动性能下降）。替代方案:`constexpr`/POD 常量（直接进 .rodata)、函数内 `static`（懒初始化）、或按需构造返回。\n\n## Build & Test\n\n```bash\n# Build C++ (with LLM)\nmkdir build && cd build\ncmake .. -DMNN_BUILD_LLM=ON -DMNN_LOW_MEMORY=ON && make -j$(nproc)\n\n# Common CMake options: MNN_BUILD_TEST, MNN_BUILD_CONVERTER, MNN_METAL, MNN_OPENCL,\n# MNN_VULKAN, MNN_CUDA, MNN_ARM82, MNN_BUILD_QUANTOOLS, MNN_SUPPORT_TRANSFORMER_FUSE\n# Full list: see option() declarations at the top of CMakeLists.txt\n\n# Unit tests\ncd build && ./run_test.out\n\n# LLM export\ncd transformers/llm/export\npython llmexport.py --path /path/to/model --export mnn --hqq --dst_path ./MODEL\n\n# LLM test\ncd build\n./llm_demo /path/to/MODEL/config.json prompt.txt\n\n# LLM benchmark\n./llm_bench -m /path/to/MODEL/config.json\n```\n\nTest suite includes: unit tests (`run_test.out`), model tests, conversion tests (ONNX/TF/TFLite/Torch), quantization tests, LLM tests, PyMNN tests. See `test.sh`, `test_stages.json`, and `test/` directory for details.\n\n## Development Workflow\n\n- **Branch**: Use `feature/<short-description>`, for example `feature/llm-streaming`.\n- **Commit**: Use a concise, one-line English message: `[Module:Type] Summary`, for example `[LLM:Feature] Add streaming support`.\n  - Modules: `LLM`, `CPU`, `Metal`, `CUDA`, `OpenCL`, `Vulkan`, `Core`, `Infra`, `Doc`, etc.\n  - Types: `Feature`, `Bugfix`, `Perf`, `Refact`, `Style`, `Doc`, `Test`, `Chore`, `Release`.\n\n## Skills\n\nFor the following tasks, **read the Skill entry file first** and execute step by step. Each step must pass its tests before proceeding.\n\n**After non-trivial skill-driven tasks, run Retrospective only when there are reusable lessons.**\n\n上表可能未覆盖全部 skill。遇到打包/发布/摩天轮/MTL/crash/集成等关键词时，**先 `Glob skills/**/SKILL.md` 确认是否有对应 skill**，不要仅凭表格判断。\n\nPublic skills are listed below. Environment-dependent skills may exist under `skills/*/SKILL.md`.\n\n| Skill | Entry File | Trigger |\n|-------|-----------|---------|\n| Support new LLM | `skills/support-new-llm/SKILL.md` | Add / adapt a new LLM model |\n| Add new op | `skills/add-new-op/SKILL.md` | Add a new operator |\n| ARM CPU optimization | `skills/arm-cpu-optimize/SKILL.md` | Optimize op performance on ARM CPU |\n| RISC-V CPU optimization | `skills/riscv-cpu-optimize/SKILL.md` | Optimize standard RVV or vendor matrix-extension CPU paths |\n| OpenCL optimization | `skills/opencl-optimize/SKILL.md` | Optimize op performance on OpenCL |\n| Vulkan optimization | `skills/vulkan-optimize/SKILL.md` | Optimize op performance on Vulkan |\n| Metal optimization | `skills/metal-optimize/SKILL.md` | Optimize op performance on Metal |\n| Bugfix / debugging | `skills/general-debug/SKILL.md` | Diagnose correctness bugs / regressions in MNN — organized by bug category. |\n| Run tests / CI | `skills/test-ci/SKILL.md` | Run the regression / CI suite (host or on-device), benchmark LLM on a real iPhone/iPad, or add / select / retune a test stage |\n| Retrospective | `skills/retrospective/SKILL.md` | After non-trivial tasks with reusable lessons |\n"},"files":{"AGENTS.md":"# MNN Project Instructions\n\nMNN is a lightweight deep learning **inference engine** (not a training framework), targeting mobile and server platforms. Supports CNN / Transformer / LLM / Diffusion models. Code must prioritize **performance and binary size**.\n\n## Restricted Access\n\n> The following directories contain internal proprietary code. **Do NOT read, modify, or reference** any files within:\n> - `schema/private/`\n> - `source/internal/`\n\n## Architecture Overview\n\nMNN uses a **graph optimization + heterogeneous backend scheduling** architecture.\n\nTwo inference APIs are available:\n- **Session API** (low-level): `Interpreter → createSession → runSession`, operates on Tensor directly\n- **Module API** (high-level, recommended): `Module::load → onForward(VARP)`, Express-based dynamic graph. Used by LLM / Diffusion and most modern workloads\n\n**Key abstractions** (see corresponding headers under `source/core/`):\n- **Interpreter** / **Session**: model loading and inference session management\n- **Backend** / **Execution**: hardware backend abstraction and per-op implementation (CPU/Metal/CUDA/OpenCL/Vulkan/...)\n- **Tensor**: data container; internally uses NC4HW4 format (channels packed by 4 for SIMD)\n- **Op / Schema**: FlatBuffers-defined operator descriptors (`schema/default/*.fbs`)\n\n**Op registration pattern**: Schema definition → shape inference (`source/shape/`) → Geometry decomposition (optional) → Backend Execution implementation\n\n## LLM Subsystem\n\nMNN supports end-to-end LLM export and inference:\n\n- **Python export** (`transformers/llm/export/`): HuggingFace model → MNN format. Core modules: `llmexport.py` entry point, `utils/model_mapper.py` (model field mapping), `utils/model.py` (unified LlmModel class), `utils/transformers.py` (Attention/Decoder/RoPE export)\n- **C++ inference** (`transformers/llm/engine/`): `llm.cpp` (text inference), `omni.cpp` (multimodal: vision/audio), includes KVCache management and sampling strategies\n\n## Repository Structure\n\n| Directory | Description |\n|-----------|-------------|\n| `include/MNN/` | Public C++ headers |\n| `source/core/` | Inference core (Interpreter, Session, Pipeline, Backend) |\n| `source/backend/` | Hardware backend implementations (cpu, arm82, metal, cuda, opencl, vulkan, ...) |\n| `source/shape/` | Shape inference |\n| `source/geometry/` | Geometry computation (op decomposition) |\n| `express/` | Express API (high-level dynamic graph, VARP) |\n| `schema/default/` | FlatBuffers schema (op definitions) |\n| `tools/converter/` | Model converter (ONNX/TF/Caffe → MNN) |\n| `transformers/llm/` | LLM export (Python) + inference engine (C++) |\n| `transformers/diffusion/` | Diffusion model support |\n| `pymnn/` | Python bindings |\n| `test/` | Test cases |\n| `skills/` | AI Agent Skills |\n\n## Coding Style\n\n- **C++**: Google Style variant, see `.clang-format`. 4-space indent, 120-char line width, attached braces. Class names `PascalCase`, functions `camelCase`, member variables `mCamelCase`. RTTI and exceptions disabled (`-fno-rtti -fno-exceptions`). Default standard: C++11.\n- **Python**: Standard Python conventions\n- **Formatting**: `clang-format -i -style=file <file>`\n\n> ⚠️ **禁止 C++ 全局对象（动态初始化）**:不允许在命名空间作用域定义需要动态初始化的非 POD 对象（如 `static const std::string` / `static const json` / 全局 STL 容器，**头文件中同样禁止**,每个包含它的 TU 都会生成一份）。这类对象产生 `__GLOBAL__sub_I_*` 启动初始化函数，会被手淘等打包静态检查拦截（代码安全风险 + 启动性能下降）。替代方案:`constexpr`/POD 常量（直接进 .rodata)、函数内 `static`（懒初始化）、或按需构造返回。\n\n## Build & Test\n\n```bash\n# Build C++ (with LLM)\nmkdir build && cd build\ncmake .. -DMNN_BUILD_LLM=ON -DMNN_LOW_MEMORY=ON && make -j$(nproc)\n\n# Common CMake options: MNN_BUILD_TEST, MNN_BUILD_CONVERTER, MNN_METAL, MNN_OPENCL,\n# MNN_VULKAN, MNN_CUDA, MNN_ARM82, MNN_BUILD_QUANTOOLS, MNN_SUPPORT_TRANSFORMER_FUSE\n# Full list: see option() declarations at the top of CMakeLists.txt\n\n# Unit tests\ncd build && ./run_test.out\n\n# LLM export\ncd transformers/llm/export\npython llmexport.py --path /path/to/model --export mnn --hqq --dst_path ./MODEL\n\n# LLM test\ncd build\n./llm_demo /path/to/MODEL/config.json prompt.txt\n\n# LLM benchmark\n./llm_bench -m /path/to/MODEL/config.json\n```\n\nTest suite includes: unit tests (`run_test.out`), model tests, conversion tests (ONNX/TF/TFLite/Torch), quantization tests, LLM tests, PyMNN tests. See `test.sh`, `test_stages.json`, and `test/` directory for details.\n\n## Development Workflow\n\n- **Branch**: Use `feature/<short-description>`, for example `feature/llm-streaming`.\n- **Commit**: Use a concise, one-line English message: `[Module:Type] Summary`, for example `[LLM:Feature] Add streaming support`.\n  - Modules: `LLM`, `CPU`, `Metal`, `CUDA`, `OpenCL`, `Vulkan`, `Core`, `Infra`, `Doc`, etc.\n  - Types: `Feature`, `Bugfix`, `Perf`, `Refact`, `Style`, `Doc`, `Test`, `Chore`, `Release`.\n\n## Skills\n\nFor the following tasks, **read the Skill entry file first** and execute step by step. Each step must pass its tests before proceeding.\n\n**After non-trivial skill-driven tasks, run Retrospective only when there are reusable lessons.**\n\n上表可能未覆盖全部 skill。遇到打包/发布/摩天轮/MTL/crash/集成等关键词时，**先 `Glob skills/**/SKILL.md` 确认是否有对应 skill**，不要仅凭表格判断。\n\nPublic skills are listed below. Environment-dependent skills may exist under `skills/*/SKILL.md`.\n\n| Skill | Entry File | Trigger |\n|-------|-----------|---------|\n| Support new LLM | `skills/support-new-llm/SKILL.md` | Add / adapt a new LLM model |\n| Add new op | `skills/add-new-op/SKILL.md` | Add a new operator |\n| ARM CPU optimization | `skills/arm-cpu-optimize/SKILL.md` | Optimize op performance on ARM CPU |\n| RISC-V CPU optimization | `skills/riscv-cpu-optimize/SKILL.md` | Optimize standard RVV or vendor matrix-extension CPU paths |\n| OpenCL optimization | `skills/opencl-optimize/SKILL.md` | Optimize op performance on OpenCL |\n| Vulkan optimization | `skills/vulkan-optimize/SKILL.md` | Optimize op performance on Vulkan |\n| Metal optimization | `skills/metal-optimize/SKILL.md` | Optimize op performance on Metal |\n| Bugfix / debugging | `skills/general-debug/SKILL.md` | Diagnose correctness bugs / regressions in MNN — organized by bug category. |\n| Run tests / CI | `skills/test-ci/SKILL.md` | Run the regression / CI suite (host or on-device), benchmark LLM on a real iPhone/iPad, or add / select / retune a test stage |\n| Retrospective | `skills/retrospective/SKILL.md` | After non-trivial tasks with reusable lessons |\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"# MNN Project Instructions\n\nMNN is a lightweight deep learning **inference engine** (not a training framework), targeting mobile and server platforms. Supports CNN / Transformer / LLM / Diffusion models. Code must prioritize **performance and binary size**.\n\n## Restricted Access\n\n> The following directories contain internal proprietary code. **Do NOT read, modify, or reference** any files within:\n> - `schema/private/`\n> - `source/internal/`\n\n## Architecture Overview\n\nMNN uses a **graph optimization + heterogeneous backend scheduling** architecture.\n\nTwo inference APIs are available:\n- **Session API** (low-level): `Interpreter → createSession → runSession`, operates on Tensor directly\n- **Module API** (high-level, recommended): `Module::load → onForward(VARP)`, Express-based dynamic graph. Used by LLM / Diffusion and most modern workloads\n\n**Key abstractions** (see corresponding headers under `source/core/`):\n- **Interpreter** / **Session**: model loading and inference session management\n- **Backend** / **Execution**: hardware backend abstraction and per-op implementation (CPU/Metal/CUDA/OpenCL/Vulkan/...)\n- **Tensor**: data container; internally uses NC4HW4 format (channels packed by 4 for SIMD)\n- **Op / Schema**: FlatBuffers-defined operator descriptors (`schema/default/*.fbs`)\n\n**Op registration pattern**: Schema definition → shape inference (`source/shape/`) → Geometry decomposition (optional) → Backend Execution implementation\n\n## LLM Subsystem\n\nMNN supports end-to-end LLM export and inference:\n\n- **Python export** (`transformers/llm/export/`): HuggingFace model → MNN format. Core modules: `llmexport.py` entry point, `utils/model_mapper.py` (model field mapping), `utils/model.py` (unified LlmModel class), `utils/transformers.py` (Attention/Decoder/RoPE export)\n- **C++ inference** (`transformers/llm/engine/`): `llm.cpp` (text inference), `omni.cpp` (multimodal: vision/audio), includes KVCache management and sampling strategies\n\n## Repository Structure\n\n| Directory | Description |\n|-----------|-------------|\n| `include/MNN/` | Public C++ headers |\n| `source/core/` | Inference core (Interpreter, Session, Pipeline, Backend) |\n| `source/backend/` | Hardware backend implementations (cpu, arm82, metal, cuda, opencl, vulkan, ...) |\n| `source/shape/` | Shape inference |\n| `source/geometry/` | Geometry computation (op decomposition) |\n| `express/` | Express API (high-level dynamic graph, VARP) |\n| `schema/default/` | FlatBuffers schema (op definitions) |\n| `tools/converter/` | Model converter (ONNX/TF/Caffe → MNN) |\n| `transformers/llm/` | LLM export (Python) + inference engine (C++) |\n| `transformers/diffusion/` | Diffusion model support |\n| `pymnn/` | Python bindings |\n| `test/` | Test cases |\n| `skills/` | AI Agent Skills |\n\n## Coding Style\n\n- **C++**: Google Style variant, see `.clang-format`. 4-space indent, 120-char line width, attached braces. Class names `PascalCase`, functions `camelCase`, member variables `mCamelCase`. RTTI and exceptions disabled (`-fno-rtti -fno-exceptions`). Default standard: C++11.\n- **Python**: Standard Python conventions\n- **Formatting**: `clang-format -i -style=file <file>`\n\n> ⚠️ **禁止 C++ 全局对象（动态初始化）**:不允许在命名空间作用域定义需要动态初始化的非 POD 对象（如 `static const std::string` / `static const json` / 全局 STL 容器，**头文件中同样禁止**,每个包含它的 TU 都会生成一份）。这类对象产生 `__GLOBAL__sub_I_*` 启动初始化函数，会被手淘等打包静态检查拦截（代码安全风险 + 启动性能下降）。替代方案:`constexpr`/POD 常量（直接进 .rodata)、函数内 `static`（懒初始化）、或按需构造返回。\n\n## Build & Test\n\n```bash\n# Build C++ (with LLM)\nmkdir build && cd build\ncmake .. -DMNN_BUILD_LLM=ON -DMNN_LOW_MEMORY=ON && make -j$(nproc)\n\n# Common CMake options: MNN_BUILD_TEST, MNN_BUILD_CONVERTER, MNN_METAL, MNN_OPENCL,\n# MNN_VULKAN, MNN_CUDA, MNN_ARM82, MNN_BUILD_QUANTOOLS, MNN_SUPPORT_TRANSFORMER_FUSE\n# Full list: see option() declarations at the top of CMakeLists.txt\n\n# Unit tests\ncd build && ./run_test.out\n\n# LLM export\ncd transformers/llm/export\npython llmexport.py --path /path/to/model --export mnn --hqq --dst_path ./MODEL\n\n# LLM test\ncd build\n./llm_demo /path/to/MODEL/config.json prompt.txt\n\n# LLM benchmark\n./llm_bench -m /path/to/MODEL/config.json\n```\n\nTest suite includes: unit tests (`run_test.out`), model tests, conversion tests (ONNX/TF/TFLite/Torch), quantization tests, LLM tests, PyMNN tests. See `test.sh`, `test_stages.json`, and `test/` directory for details.\n\n## Development Workflow\n\n- **Branch**: Use `feature/<short-description>`, for example `feature/llm-streaming`.\n- **Commit**: Use a concise, one-line English message: `[Module:Type] Summary`, for example `[LLM:Feature] Add streaming support`.\n  - Modules: `LLM`, `CPU`, `Metal`, `CUDA`, `OpenCL`, `Vulkan`, `Core`, `Infra`, `Doc`, etc.\n  - Types: `Feature`, `Bugfix`, `Perf`, `Refact`, `Style`, `Doc`, `Test`, `Chore`, `Release`.\n\n## Skills\n\nFor the following tasks, **read the Skill entry file first** and execute step by step. Each step must pass its tests before proceeding.\n\n**After non-trivial skill-driven tasks, run Retrospective only when there are reusable lessons.**\n\n上表可能未覆盖全部 skill。遇到打包/发布/摩天轮/MTL/crash/集成等关键词时，**先 `Glob skills/**/SKILL.md` 确认是否有对应 skill**，不要仅凭表格判断。\n\nPublic skills are listed below. Environment-dependent skills may exist under `skills/*/SKILL.md`.\n\n| Skill | Entry File | Trigger |\n|-------|-----------|---------|\n| Support new LLM | `skills/support-new-llm/SKILL.md` | Add / adapt a new LLM model |\n| Add new op | `skills/add-new-op/SKILL.md` | Add a new operator |\n| ARM CPU optimization | `skills/arm-cpu-optimize/SKILL.md` | Optimize op performance on ARM CPU |\n| RISC-V CPU optimization | `skills/riscv-cpu-optimize/SKILL.md` | Optimize standard RVV or vendor matrix-extension CPU paths |\n| OpenCL optimization | `skills/opencl-optimize/SKILL.md` | Optimize op performance on OpenCL |\n| Vulkan optimization | `skills/vulkan-optimize/SKILL.md` | Optimize op performance on Vulkan |\n| Metal optimization | `skills/metal-optimize/SKILL.md` | Optimize op performance on Metal |\n| Bugfix / debugging | `skills/general-debug/SKILL.md` | Diagnose correctness bugs / regressions in MNN — organized by bug category. |\n| Run tests / CI | `skills/test-ci/SKILL.md` | Run the regression / CI suite (host or on-device), benchmark LLM on a real iPhone/iPad, or add / select / retune a test stage |\n| Retrospective | `skills/retrospective/SKILL.md` | After non-trivial tasks with reusable lessons |\n","category":"root","tokens":1589}]}