{"owner":"inclusionAI","repo":"AReaL","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["CLAUDE.md","AGENTS.md"],"skills":{"CLAUDE.md":"# CLAUDE.md - AReaL\n\n## WHAT: Project Overview\n\nAReaL is a distributed RL training framework for LLM alignment via reinforcement\nlearning.\n\n**Tech Stack**: Python 3.12+ | PyTorch | FSDP2/Megatron | SGLang/vLLM\n\n**Core Directories**:\n\n- `areal/` - Core package\n  - `api/` - Config dataclasses, workflow/engine contracts\n  - `engine/` - FSDP2, Megatron, SGLang/vLLM adapters\n    - `fsdp_utils/` - FSDP2-specific utilities (checkpoint, grad, optimizer, parallel)\n    - `megatron_utils/` - Megatron/FP8 utilities (checkpoint, pipeline, quantization)\n    - `core/` - Engine-shared utilities (distributed, lock, model, offload)\n  - `infra/` - Infrastructure (launcher, scheduler, RPC)\n    - `utils/` - Infrastructure utilities (launcher, proc, http, concurrent, slurm, ray)\n  - `workflow/` - RolloutWorkflow implementations\n  - `reward/` - Reward functions\n  - `dataset/` - Dataset loaders\n  - `utils/` - Cross-cutting utilities (logging, data, checkpoints, network, RL\n    functional)\n- `examples/` - Training scripts and configs\n- `docs/` - Jupyter Book source\n\n## WHY: Purpose\n\n- Enable efficient RL training for LLM alignment at scale\n- Async rollout + distributed training for high throughput\n- Modular design: workflows, engines, rewards, and datasets are independently extensible\n\n## HOW: Core Commands\n\n```bash\n# Check environment\npython --version              # Requires 3.12+\nuv --version                  # Install: https://docs.astral.sh/uv/\n\n# Sync dependencies\nuv sync --extra cuda          # CUDA + SGLang inference (default)\n# For vLLM: cp pyproject.vllm.toml pyproject.toml && cp uv.vllm.lock uv.lock && uv sync --extra cuda\nuv sync --group dev           # Include dev/test packages\nuv run python3 areal/tools/validate_installation.py  # Validate installation\n\n# Pre-commit hooks\npre-commit install --install-hooks  # Set up hooks (run once)\npre-commit run --all-files    # Format and lint\n\n# Run tests\n# First check GPU availability (many tests require GPU)\npython -c \"import torch; print('GPU available:', torch.cuda.is_available())\"\nuv run pytest tests/test_<topic>.py\n\n# Generate CLI docs\nuv run python docs/generate_cli_docs.py\n\n# Build docs (canonical, release-aligned)\n./docs/build_all.sh\n# Do NOT use `jupyter-book build docs/en|docs/zh` directly for final preview/release,\n# because it skips AReaL-specific static setup and output packaging.\n```\n\n## Boundaries\n\n### Constraints\n\n- Designed for distributed GPU clusters; assume containerized execution\n- Integration tests require multi-node hardware; explain skips when unavailable\n- Secrets and endpoints are managed outside the repo\n\n### Always Do\n\n- Read relevant files before modifying code\n- Run `pre-commit run --all-files` before committing\n- Follow existing code patterns in the same module\n- Add tests for new functionality\n\n### Ask First\n\n- Modifying config structures in `areal/api/cli_args.py`\n- Adding new dependencies\n- Changing launcher or scheduler logic\n- Deleting or renaming public APIs\n- Running GPU/distributed tests (check GPU first:\n  `python -c \"import torch; print('GPU available:', torch.cuda.is_available())\"`)\n\n### Never Do\n\n- Hardcode secrets, paths, or endpoints\n- Skip pre-commit hooks\n- Guess cluster configs or rebuild CUDA/driver stacks\n- Use wildcard imports (`from x import *`)\n\n## Progressive Disclosure: Detailed Guides\n\n| Task                   | Reference                                                     |\n| ---------------------- | ------------------------------------------------------------- |\n| Add Workflow           | `docs/customization/agent.md`, `areal/workflow/multi_turn.py` |\n| Add Dataset            | `docs/customization/`, `areal/dataset/gsm8k.py`               |\n| Add Reward             | `areal/api/reward_api.py`, `areal/reward/geometry3k.py`       |\n| Add Archon Model       | `areal/experimental/models/archon/qwen2/`, `qwen3/`           |\n| Algorithm Details      | `docs/algorithms/*.md`                                        |\n| Quickstart             | `docs/tutorial/quickstart.md`                                 |\n| Architecture Deep Dive | `docs/tutorial/gsm8k_grpo.md`                                 |\n| CLI Reference          | `docs/cli_reference.md`                                       |\n\n## Git Workflow\n\n- **Commits**: Conventional Commits (e.g., `feat:`, `fix:`, `docs:`, `gov:`), ~72 chars\n  subject, imperative voice, reasoning in body\n- **Squash**: Squash WIP commits before opening PR\n- **PR requirements**: Run pre-commit, document test coverage, note hardware limitations\n\n## Extended Configuration\n\nSee `.claude/agents/`, `.claude/skills/`, `.claude/commands/`, and `.claude/rules/` for\nspecialized instructions.\n\n### Agents\n\n| Agent                       | Purpose                                   | Activation Trigger                                                  |\n| --------------------------- | ----------------------------------------- | ------------------------------------------------------------------- |\n| `planner`                   | Implementation planning                   | Before multi-file changes, new features, or architectural decisions |\n| `simple-code-reviewer`      | Quick code quality checks                 | After code changes, before committing                               |\n| `code-verifier`             | Formatting/linting/tests                  | After code changes, before committing                               |\n| `fsdp-engine-expert`        | FSDPEngine implementation                 | FSDPEngine code changes or questions                                |\n| `archon-engine-expert`      | ArchonEngine implementation               | ArchonEngine code changes or questions                              |\n| `megatron-engine-expert`    | MegatronEngine implementation             | MegatronEngine code changes or questions                            |\n| `algorithm-expert`          | RL algorithms                             | GRPO/PPO/DAPO questions                                             |\n| `launcher-scheduler-expert` | Cluster launching and resource scheduling | Launcher/scheduler code changes or configuration questions          |\n\n**Stage-by-Stage Agent Guidance**:\n\n1. **Planning Stage** (Before coding): Use `planner` for architecture design and\n   implementation planning\n1. **Code Formatting & Linting** (After coding): Use `code-verifier` to automatically\n   run formatting, linting, and tests, catching syntax errors and style issues quickly\n1. **Code Quality Check** (After formatting): Use `simple-code-reviewer` for quick code\n   quality checks, focusing on logic issues and code smells\n\n### Skills (Guided Development Workflows)\n\nSkills provide step-by-step guides for common development tasks:\n\n- `/add-dataset` - Dataset loader creation guide\n- `/add-workflow` - Workflow implementation guide\n- `/add-reward` - Reward function guide\n- `/add-archon-model` - Archon engine model architecture guide\n- `/debug-distributed` - Distributed debugging guide\n- `/add-unit-tests` - Test development guide (NEW)\n\n### Commands (User-invoked Actions)\n\nCommands perform specific actions when invoked:\n\n- `/create-pr` - Rebase, squash commits, and create/update PR with intelligent messages\n- `/gen-commit-msg` - Generate commit messages from staged changes\n- `/review-pr` - Intelligent PR code review with dynamic agent allocation\n- `/translate-doc-zh` - Translate English documentation to Chinese\n\n### Rules (Code Quality Standards)\n\nProject-wide standards enforced across all code changes:\n\n- `api-config.md` - Configuration dataclass design patterns\n- `code-style.md` - Coding conventions beyond pre-commit hooks\n- `distributed.md` - Distributed training patterns and constraints\n- `testing.md` - Testing strategy and coverage requirements\n","AGENTS.md":"<!-- Go-to brief for AI coding agents working on AReaL. -->\n\n# AGENTS.md -- AReaL Agent Operations Guide\n\n## Quick reference\n\n**Tech stack**: Python 3.12+ | PyTorch | FSDP2 / Megatron / Archon | SGLang / vLLM\n\n```bash\n# Environment\nuv sync --extra cuda            # CUDA + SGLang inference (default); for vLLM: cp pyproject.vllm.toml pyproject.toml && cp uv.vllm.lock uv.lock && uv sync --extra cuda\nuv sync --extra sandbox         # Daytona cloud sandbox backend (optional)\nsource .venv/bin/activate        # activate venv BEFORE pre-commit or git commit if venv exists\npre-commit install --install-hooks  # hooks: Ruff, clang-format, mdformat, nbstripout, conventional-commits\npre-commit run --all-files       # lint + format everything\n\n# Tests\nuv run pytest tests/test_<topic>.py\n\n# CLI docs\nuv run python docs/generate_cli_docs.py\n\n# Docs build (canonical, release-aligned)\n./docs/build_all.sh\n# Do NOT use `jupyter-book build docs/en|docs/zh` directly for final preview/release,\n# because it skips AReaL-specific static setup and output packaging.\n```\n\n**Hard rules** -- never violate:\n\n- No wildcard imports (`from x import *`).\n- No hardcoded secrets, paths, or endpoints.\n- No skipping pre-commit hooks.\n- No guessing cluster configs or rebuilding CUDA/driver stacks.\n- Integration tests require multi-node hardware -- explain skips explicitly.\n\n**Always do**:\n\n- Read relevant files before modifying code.\n- Run `pre-commit run --all-files` before committing.\n- Follow existing code patterns in the same module.\n- Add tests for new functionality.\n- Ask for decisions and clarifications with short, structured options instead of broad\n  open-ended questions. Use the platform's native question/clarification tool if\n  available.\n\n**Ask first** before:\n\n- Modifying config structures in `areal/api/cli_args.py`.\n- Adding new dependencies.\n- Changing launcher or scheduler logic.\n- Deleting or renaming public APIs.\n\nWhen unsure, leave a `TODO(agent)` comment and note the constraint in your response.\n\n______________________________________________________________________\n\n## Repository map\n\n```\nareal/                     Core Python package\n|-- api/                   Config dataclasses, contracts, IO structs\n|-- dataset/               Stateful dataset loaders (GSM8K, Geometry3K, CLEVR, ...)\n|-- engine/                Training backends (FSDP2, Megatron) + inference adapters\n|-- experimental/          Prototype engines/workflows (Archon MoE engine)\n|-- infra/                 Launchers (Local/Ray/Slurm), schedulers, utilities\n|-- models/                Model adapters (Megatron-Core, Transformers, custom heads)\n|-- reward/                Built-in reward functions + math parsers\n|-- tests/                 Unit/integration test suites\n|-- trainer/               High-level orchestrators (PPOTrainer, SFTTrainer)\n|-- utils/                 Cross-cutting helpers (logging, data, checkpoints, RL ops)\n+-- workflow/              RolloutWorkflow implementations (RLVR, multi-turn, vision)\n\ndocs/                      Jupyter Book docs (https://areal-project.github.io/AReaL/)\nexamples/                  Training scripts and launcher recipes\n```\n\n______________________________________________________________________\n\n## Code style & patterns\n\n- **Composition over inheritance** -- keep hierarchies \\<= 2 levels; prefer delegation.\n\n| Type             | Pattern         | Example                                   |\n| ---------------- | --------------- | ----------------------------------------- |\n| Config dataclass | `XxxConfig`     | `GRPOConfig`, `FSDPEngineConfig`          |\n| Engine class     | `XxxEngine`     | `FSDPEngine`, `ArchonEngine`              |\n| Workflow class   | `XxxWorkflow`   | `RLVRWorkflow`, `MultiTurnWorkflow`       |\n| Reward function  | `xxx_reward_fn` | `gsm8k_reward_fn`, `geometry3k_reward_fn` |\n\n**Logging**: `areal.utils.logging.getLogger(name)` with **PascalCase** names -- never\n`print` or `logging.__name__`. Per-rank format: `[{Component} Rank {N}]`. Register new\nloggers with color in `areal/utils/logging.py`.\n\n**Performance**:\n\n- No GPU-CPU sync in hot paths (`.item()`, `.tolist()`, `print(tensor)`).\n- Batch ops over Python loops on tensor elements.\n- Explicit `dtype`/`device`; `torch.Size` assertions for shape validation.\n\n**Typing & imports**: explicit type hints; reuse `areal/api/cli_args.py` dataclasses; no\nwildcard imports; heavy optional deps inside functions.\n\n**Async**: rollout workflows must stay non-blocking (`await` + `aiofiles`); no sync I/O\nin `arun_episode`.\n\n______________________________________________________________________\n\n## Domain experts & skills\n\nFire the appropriate **expert subagent** or **load a skill** based on what you're\nworking on. Experts are read-only consultants with deep domain knowledge; skills are\nstep-by-step implementation guides.\n\n| Working on...                | Fire subagent      | Load skill          |\n| ---------------------------- | ------------------ | ------------------- |\n| FSDP engine code             | `fsdp-expert`      | --                  |\n| Archon engine / new model    | `archon-expert`    | `add-archon-model`  |\n| Megatron engine code         | `megatron-expert`  | --                  |\n| RL algorithms / PPO / GRPO   | `algorithm-expert` | --                  |\n| Launcher / scheduler / infra | `launcher-expert`  | `debug-distributed` |\n| New reward function          | --                 | `add-reward`        |\n| New dataset loader           | --                 | `add-dataset`       |\n| New rollout workflow         | --                 | `add-workflow`      |\n| Unit tests                   | --                 | `add-unit-tests`    |\n| Distributed debugging        | --                 | `debug-distributed` |\n\n**How to invoke experts and skills** (platform-specific):\n\n| Platform | Fire expert subagent                                                               | Load skill                                         |\n| -------- | ---------------------------------------------------------------------------------- | -------------------------------------------------- |\n| OpenCode | `task(subagent_type=\"<name>\", load_skills=[], run_in_background=true, prompt=\"…\")` | `skill(name=\"<name>\")` or `load_skills=[\"<name>\"]` |\n| Codex    | Invoke registered subagent by canonical name (see `.codex/config.toml`)            | Reference `.agents/skills/<name>/SKILL.md`         |\n\n**Harness layout**:\n\n| Component         | OpenCode                                | Codex                                                  |\n| ----------------- | --------------------------------------- | ------------------------------------------------------ |\n| Root instructions | `AGENTS.md`                             | `AGENTS.md`                                            |\n| Agent configs     | `.opencode/agents/*.md` (frontmatter)   | `.codex/config.toml` + `.codex/agents/*.toml` + `*.md` |\n| Skills            | `.opencode/skills/` + `.agents/skills/` | `.agents/skills/<name>/SKILL.md`                       |\n\nDirectly executable workflows (both platforms): `add-workflow`, `review-pr`,\n`create-pr`, `translate-doc-zh`.\n\n______________________________________________________________________\n\n## Core concepts\n\n**Trainer** orchestrator (`areal/trainer/`, `PPOTrainer`, `SFTTrainer`): manages the\ntraining loop, dataset loading, and workflow execution. Entry point:\n`examples/math/gsm8k_rl.py`.\n\n**Rollout workflows** (`areal/workflow/`, `RolloutWorkflow.arun_episode`): define how\nepisodes are generated. Use `add-workflow` skill for step-by-step guide.\n\n**Engines**: *Inference engines* handle async generation via `engine.agenerate()` and\nmanage weight updates. *Training engines* consume rollout tensors, compute PPO/GRPO\nupdates, and broadcast weight versions (FSDP2, Megatron, or Archon).\n\n**Weight versioning**: async workflows require version alignment via `WeightUpdateMeta`\n(`areal/api/engine_api.py`). Critical for correctness across distributed training.\n\n**Observability**: emit metrics via `stats_tracker.get()`, persist artifacts under\n`dump_dir`, checkpoint via `areal/utils/saver.py` / `recover.py`.\n\n**Launcher / scheduler**: training requires cluster setup (local / Ray / Slurm) via\nconfigs in `areal/infra/launcher/`. See `launcher-expert` for deployment guidance.\n\n______________________________________________________________________\n\n## API & config rules\n\n*Applies to: `areal/api/**`*\n\n- **Field ordering**: required -> common optional -> rare optional -> internal (`_`\n  prefix).\n- **Validation**: `__post_init__` with `ValueError` and clear message.\n- **Backward compat**: add fields with defaults; deprecate before removing; avoid type\n  changes.\n- **CLI**: use `Literal` for enum choices; all public configs need docstrings with\n  constraints.\n\n______________________________________________________________________\n\n## Distributed code rules\n\n*Applies to: `areal/engine/**`, `areal/experimental/**`*\n\n- Never create global process groups at module level; always pass `process_group`\n  explicitly.\n- `dist.get_rank(group)` not `dist.get_rank()` when group matters.\n- DeviceMesh dimensions must match `ArchonParallelDims`: `dp_shard`, `tp`, `cp`, `ep`,\n  `etp`.\n- All-reduce: all ranks must call. Broadcast: explicit `src`. Barrier: debugging only.\n\n| Issue         | Cause                            | Fix                       |\n| ------------- | -------------------------------- | ------------------------- |\n| Hang          | Mismatched collective calls      | All ranks call same op    |\n| Wrong results | Incorrect `ReduceOp`             | Check SUM vs MEAN         |\n| OOM           | Unsharded tensor on wrong device | Verify DTensor placements |\n\nDebug env vars: `TORCH_DISTRIBUTED_DEBUG=DETAIL`, `NCCL_DEBUG=INFO`,\n`CUDA_LAUNCH_BLOCKING=1`. See the `debug-distributed` skill for the full workflow.\n\n______________________________________________________________________\n\n## Testing rules\n\n*Applies to: `**/tests/**`, `test_*.py`*\n\n| Marker                                  | When                             |\n| --------------------------------------- | -------------------------------- |\n| `@pytest.mark.slow`                     | > 10s (excluded from default CI) |\n| `@pytest.mark.slow` + `@pytest.mark.ci` | Slow but must run in CI          |\n| `@pytest.mark.asyncio`                  | Async tests                      |\n\n- Naming: `test_<what>_<condition>_<expected>()` with Arrange/Act/Assert.\n- GPU: skip gracefully (`@pytest.mark.skipif(not CUDA_AVAILABLE, reason=\"...\")`).\n- Distributed mocking: `torch.distributed.fake_pg`; don't mock FSDP/DTensor internals.\n- Assertions: `torch.testing.assert_close()` with explicit `rtol`/`atol`; prefer\n  `tmp_path`, `monkeypatch`.\n\n| Suite       | Command                       | GPU       |\n| ----------- | ----------------------------- | --------- |\n| Unit        | `pytest tests/test_*.py`      | No        |\n| GRPO        | `pytest tests/grpo/`          | Yes       |\n| FSDP        | `pytest tests/test_fsdp_*.py` | Yes       |\n| Distributed | `pytest tests/torchrun/`      | Multi-GPU |\n\n______________________________________________________________________\n\n## Collaboration & review\n\n- **Branches**: kebab-case (`feature/multi-turn-metrics`, `bugfix/fsdp-weight-sync`).\n- **Commits**: Conventional Commits (e.g., `feat:`, `fix:`, `docs:`, `gov:`), ~72 char\n  subject, imperative voice. Squash WIP before PR.\n- **Pre-merge**: full pre-commit stack; doc-only edits need at least `mdformat --check`.\n- **PRs**: tie to issue, highlight risk areas, list test commands executed, note skipped\n  suites with reasons.\n\n| Skill                | Purpose                                                |\n| -------------------- | ------------------------------------------------------ |\n| `create-pr`          | Rebase, squash, and create or update a PR              |\n| `commit-conventions` | Commit message conventions to load before `git commit` |\n| `review-pr`          | Dynamic PR review with targeted expert consultation    |\n| `translate-doc-zh`   | Translate English docs to Chinese                      |\n\n______________________________________________________________________\n\n## Reference material\n\n- **Docs portal**: <https://areal-project.github.io/AReaL/>\n- **Quickstart**: `docs/tutorial/quickstart.md`\n- **Architecture**: `docs/tutorial/gsm8k_grpo.md`\n- **Customization**: `docs/customization/*.md`\n- **Algorithms**: `docs/algorithms/*.md`\n- **Best practices**: `docs/best_practices/*.md`\n- **CLI reference**: `docs/cli_reference.md`\n- **Agent workflow**: `docs/customization/agent.md`\n"},"files":{"CLAUDE.md":"# CLAUDE.md - AReaL\n\n## WHAT: Project Overview\n\nAReaL is a distributed RL training framework for LLM alignment via reinforcement\nlearning.\n\n**Tech Stack**: Python 3.12+ | PyTorch | FSDP2/Megatron | SGLang/vLLM\n\n**Core Directories**:\n\n- `areal/` - Core package\n  - `api/` - Config dataclasses, workflow/engine contracts\n  - `engine/` - FSDP2, Megatron, SGLang/vLLM adapters\n    - `fsdp_utils/` - FSDP2-specific utilities (checkpoint, grad, optimizer, parallel)\n    - `megatron_utils/` - Megatron/FP8 utilities (checkpoint, pipeline, quantization)\n    - `core/` - Engine-shared utilities (distributed, lock, model, offload)\n  - `infra/` - Infrastructure (launcher, scheduler, RPC)\n    - `utils/` - Infrastructure utilities (launcher, proc, http, concurrent, slurm, ray)\n  - `workflow/` - RolloutWorkflow implementations\n  - `reward/` - Reward functions\n  - `dataset/` - Dataset loaders\n  - `utils/` - Cross-cutting utilities (logging, data, checkpoints, network, RL\n    functional)\n- `examples/` - Training scripts and configs\n- `docs/` - Jupyter Book source\n\n## WHY: Purpose\n\n- Enable efficient RL training for LLM alignment at scale\n- Async rollout + distributed training for high throughput\n- Modular design: workflows, engines, rewards, and datasets are independently extensible\n\n## HOW: Core Commands\n\n```bash\n# Check environment\npython --version              # Requires 3.12+\nuv --version                  # Install: https://docs.astral.sh/uv/\n\n# Sync dependencies\nuv sync --extra cuda          # CUDA + SGLang inference (default)\n# For vLLM: cp pyproject.vllm.toml pyproject.toml && cp uv.vllm.lock uv.lock && uv sync --extra cuda\nuv sync --group dev           # Include dev/test packages\nuv run python3 areal/tools/validate_installation.py  # Validate installation\n\n# Pre-commit hooks\npre-commit install --install-hooks  # Set up hooks (run once)\npre-commit run --all-files    # Format and lint\n\n# Run tests\n# First check GPU availability (many tests require GPU)\npython -c \"import torch; print('GPU available:', torch.cuda.is_available())\"\nuv run pytest tests/test_<topic>.py\n\n# Generate CLI docs\nuv run python docs/generate_cli_docs.py\n\n# Build docs (canonical, release-aligned)\n./docs/build_all.sh\n# Do NOT use `jupyter-book build docs/en|docs/zh` directly for final preview/release,\n# because it skips AReaL-specific static setup and output packaging.\n```\n\n## Boundaries\n\n### Constraints\n\n- Designed for distributed GPU clusters; assume containerized execution\n- Integration tests require multi-node hardware; explain skips when unavailable\n- Secrets and endpoints are managed outside the repo\n\n### Always Do\n\n- Read relevant files before modifying code\n- Run `pre-commit run --all-files` before committing\n- Follow existing code patterns in the same module\n- Add tests for new functionality\n\n### Ask First\n\n- Modifying config structures in `areal/api/cli_args.py`\n- Adding new dependencies\n- Changing launcher or scheduler logic\n- Deleting or renaming public APIs\n- Running GPU/distributed tests (check GPU first:\n  `python -c \"import torch; print('GPU available:', torch.cuda.is_available())\"`)\n\n### Never Do\n\n- Hardcode secrets, paths, or endpoints\n- Skip pre-commit hooks\n- Guess cluster configs or rebuild CUDA/driver stacks\n- Use wildcard imports (`from x import *`)\n\n## Progressive Disclosure: Detailed Guides\n\n| Task                   | Reference                                                     |\n| ---------------------- | ------------------------------------------------------------- |\n| Add Workflow           | `docs/customization/agent.md`, `areal/workflow/multi_turn.py` |\n| Add Dataset            | `docs/customization/`, `areal/dataset/gsm8k.py`               |\n| Add Reward             | `areal/api/reward_api.py`, `areal/reward/geometry3k.py`       |\n| Add Archon Model       | `areal/experimental/models/archon/qwen2/`, `qwen3/`           |\n| Algorithm Details      | `docs/algorithms/*.md`                                        |\n| Quickstart             | `docs/tutorial/quickstart.md`                                 |\n| Architecture Deep Dive | `docs/tutorial/gsm8k_grpo.md`                                 |\n| CLI Reference          | `docs/cli_reference.md`                                       |\n\n## Git Workflow\n\n- **Commits**: Conventional Commits (e.g., `feat:`, `fix:`, `docs:`, `gov:`), ~72 chars\n  subject, imperative voice, reasoning in body\n- **Squash**: Squash WIP commits before opening PR\n- **PR requirements**: Run pre-commit, document test coverage, note hardware limitations\n\n## Extended Configuration\n\nSee `.claude/agents/`, `.claude/skills/`, `.claude/commands/`, and `.claude/rules/` for\nspecialized instructions.\n\n### Agents\n\n| Agent                       | Purpose                                   | Activation Trigger                                                  |\n| --------------------------- | ----------------------------------------- | ------------------------------------------------------------------- |\n| `planner`                   | Implementation planning                   | Before multi-file changes, new features, or architectural decisions |\n| `simple-code-reviewer`      | Quick code quality checks                 | After code changes, before committing                               |\n| `code-verifier`             | Formatting/linting/tests                  | After code changes, before committing                               |\n| `fsdp-engine-expert`        | FSDPEngine implementation                 | FSDPEngine code changes or questions                                |\n| `archon-engine-expert`      | ArchonEngine implementation               | ArchonEngine code changes or questions                              |\n| `megatron-engine-expert`    | MegatronEngine implementation             | MegatronEngine code changes or questions                            |\n| `algorithm-expert`          | RL algorithms                             | GRPO/PPO/DAPO questions                                             |\n| `launcher-scheduler-expert` | Cluster launching and resource scheduling | Launcher/scheduler code changes or configuration questions          |\n\n**Stage-by-Stage Agent Guidance**:\n\n1. **Planning Stage** (Before coding): Use `planner` for architecture design and\n   implementation planning\n1. **Code Formatting & Linting** (After coding): Use `code-verifier` to automatically\n   run formatting, linting, and tests, catching syntax errors and style issues quickly\n1. **Code Quality Check** (After formatting): Use `simple-code-reviewer` for quick code\n   quality checks, focusing on logic issues and code smells\n\n### Skills (Guided Development Workflows)\n\nSkills provide step-by-step guides for common development tasks:\n\n- `/add-dataset` - Dataset loader creation guide\n- `/add-workflow` - Workflow implementation guide\n- `/add-reward` - Reward function guide\n- `/add-archon-model` - Archon engine model architecture guide\n- `/debug-distributed` - Distributed debugging guide\n- `/add-unit-tests` - Test development guide (NEW)\n\n### Commands (User-invoked Actions)\n\nCommands perform specific actions when invoked:\n\n- `/create-pr` - Rebase, squash commits, and create/update PR with intelligent messages\n- `/gen-commit-msg` - Generate commit messages from staged changes\n- `/review-pr` - Intelligent PR code review with dynamic agent allocation\n- `/translate-doc-zh` - Translate English documentation to Chinese\n\n### Rules (Code Quality Standards)\n\nProject-wide standards enforced across all code changes:\n\n- `api-config.md` - Configuration dataclass design patterns\n- `code-style.md` - Coding conventions beyond pre-commit hooks\n- `distributed.md` - Distributed training patterns and constraints\n- `testing.md` - Testing strategy and coverage requirements\n","AGENTS.md":"<!-- Go-to brief for AI coding agents working on AReaL. -->\n\n# AGENTS.md -- AReaL Agent Operations Guide\n\n## Quick reference\n\n**Tech stack**: Python 3.12+ | PyTorch | FSDP2 / Megatron / Archon | SGLang / vLLM\n\n```bash\n# Environment\nuv sync --extra cuda            # CUDA + SGLang inference (default); for vLLM: cp pyproject.vllm.toml pyproject.toml && cp uv.vllm.lock uv.lock && uv sync --extra cuda\nuv sync --extra sandbox         # Daytona cloud sandbox backend (optional)\nsource .venv/bin/activate        # activate venv BEFORE pre-commit or git commit if venv exists\npre-commit install --install-hooks  # hooks: Ruff, clang-format, mdformat, nbstripout, conventional-commits\npre-commit run --all-files       # lint + format everything\n\n# Tests\nuv run pytest tests/test_<topic>.py\n\n# CLI docs\nuv run python docs/generate_cli_docs.py\n\n# Docs build (canonical, release-aligned)\n./docs/build_all.sh\n# Do NOT use `jupyter-book build docs/en|docs/zh` directly for final preview/release,\n# because it skips AReaL-specific static setup and output packaging.\n```\n\n**Hard rules** -- never violate:\n\n- No wildcard imports (`from x import *`).\n- No hardcoded secrets, paths, or endpoints.\n- No skipping pre-commit hooks.\n- No guessing cluster configs or rebuilding CUDA/driver stacks.\n- Integration tests require multi-node hardware -- explain skips explicitly.\n\n**Always do**:\n\n- Read relevant files before modifying code.\n- Run `pre-commit run --all-files` before committing.\n- Follow existing code patterns in the same module.\n- Add tests for new functionality.\n- Ask for decisions and clarifications with short, structured options instead of broad\n  open-ended questions. Use the platform's native question/clarification tool if\n  available.\n\n**Ask first** before:\n\n- Modifying config structures in `areal/api/cli_args.py`.\n- Adding new dependencies.\n- Changing launcher or scheduler logic.\n- Deleting or renaming public APIs.\n\nWhen unsure, leave a `TODO(agent)` comment and note the constraint in your response.\n\n______________________________________________________________________\n\n## Repository map\n\n```\nareal/                     Core Python package\n|-- api/                   Config dataclasses, contracts, IO structs\n|-- dataset/               Stateful dataset loaders (GSM8K, Geometry3K, CLEVR, ...)\n|-- engine/                Training backends (FSDP2, Megatron) + inference adapters\n|-- experimental/          Prototype engines/workflows (Archon MoE engine)\n|-- infra/                 Launchers (Local/Ray/Slurm), schedulers, utilities\n|-- models/                Model adapters (Megatron-Core, Transformers, custom heads)\n|-- reward/                Built-in reward functions + math parsers\n|-- tests/                 Unit/integration test suites\n|-- trainer/               High-level orchestrators (PPOTrainer, SFTTrainer)\n|-- utils/                 Cross-cutting helpers (logging, data, checkpoints, RL ops)\n+-- workflow/              RolloutWorkflow implementations (RLVR, multi-turn, vision)\n\ndocs/                      Jupyter Book docs (https://areal-project.github.io/AReaL/)\nexamples/                  Training scripts and launcher recipes\n```\n\n______________________________________________________________________\n\n## Code style & patterns\n\n- **Composition over inheritance** -- keep hierarchies \\<= 2 levels; prefer delegation.\n\n| Type             | Pattern         | Example                                   |\n| ---------------- | --------------- | ----------------------------------------- |\n| Config dataclass | `XxxConfig`     | `GRPOConfig`, `FSDPEngineConfig`          |\n| Engine class     | `XxxEngine`     | `FSDPEngine`, `ArchonEngine`              |\n| Workflow class   | `XxxWorkflow`   | `RLVRWorkflow`, `MultiTurnWorkflow`       |\n| Reward function  | `xxx_reward_fn` | `gsm8k_reward_fn`, `geometry3k_reward_fn` |\n\n**Logging**: `areal.utils.logging.getLogger(name)` with **PascalCase** names -- never\n`print` or `logging.__name__`. Per-rank format: `[{Component} Rank {N}]`. Register new\nloggers with color in `areal/utils/logging.py`.\n\n**Performance**:\n\n- No GPU-CPU sync in hot paths (`.item()`, `.tolist()`, `print(tensor)`).\n- Batch ops over Python loops on tensor elements.\n- Explicit `dtype`/`device`; `torch.Size` assertions for shape validation.\n\n**Typing & imports**: explicit type hints; reuse `areal/api/cli_args.py` dataclasses; no\nwildcard imports; heavy optional deps inside functions.\n\n**Async**: rollout workflows must stay non-blocking (`await` + `aiofiles`); no sync I/O\nin `arun_episode`.\n\n______________________________________________________________________\n\n## Domain experts & skills\n\nFire the appropriate **expert subagent** or **load a skill** based on what you're\nworking on. Experts are read-only consultants with deep domain knowledge; skills are\nstep-by-step implementation guides.\n\n| Working on...                | Fire subagent      | Load skill          |\n| ---------------------------- | ------------------ | ------------------- |\n| FSDP engine code             | `fsdp-expert`      | --                  |\n| Archon engine / new model    | `archon-expert`    | `add-archon-model`  |\n| Megatron engine code         | `megatron-expert`  | --                  |\n| RL algorithms / PPO / GRPO   | `algorithm-expert` | --                  |\n| Launcher / scheduler / infra | `launcher-expert`  | `debug-distributed` |\n| New reward function          | --                 | `add-reward`        |\n| New dataset loader           | --                 | `add-dataset`       |\n| New rollout workflow         | --                 | `add-workflow`      |\n| Unit tests                   | --                 | `add-unit-tests`    |\n| Distributed debugging        | --                 | `debug-distributed` |\n\n**How to invoke experts and skills** (platform-specific):\n\n| Platform | Fire expert subagent                                                               | Load skill                                         |\n| -------- | ---------------------------------------------------------------------------------- | -------------------------------------------------- |\n| OpenCode | `task(subagent_type=\"<name>\", load_skills=[], run_in_background=true, prompt=\"…\")` | `skill(name=\"<name>\")` or `load_skills=[\"<name>\"]` |\n| Codex    | Invoke registered subagent by canonical name (see `.codex/config.toml`)            | Reference `.agents/skills/<name>/SKILL.md`         |\n\n**Harness layout**:\n\n| Component         | OpenCode                                | Codex                                                  |\n| ----------------- | --------------------------------------- | ------------------------------------------------------ |\n| Root instructions | `AGENTS.md`                             | `AGENTS.md`                                            |\n| Agent configs     | `.opencode/agents/*.md` (frontmatter)   | `.codex/config.toml` + `.codex/agents/*.toml` + `*.md` |\n| Skills            | `.opencode/skills/` + `.agents/skills/` | `.agents/skills/<name>/SKILL.md`                       |\n\nDirectly executable workflows (both platforms): `add-workflow`, `review-pr`,\n`create-pr`, `translate-doc-zh`.\n\n______________________________________________________________________\n\n## Core concepts\n\n**Trainer** orchestrator (`areal/trainer/`, `PPOTrainer`, `SFTTrainer`): manages the\ntraining loop, dataset loading, and workflow execution. Entry point:\n`examples/math/gsm8k_rl.py`.\n\n**Rollout workflows** (`areal/workflow/`, `RolloutWorkflow.arun_episode`): define how\nepisodes are generated. Use `add-workflow` skill for step-by-step guide.\n\n**Engines**: *Inference engines* handle async generation via `engine.agenerate()` and\nmanage weight updates. *Training engines* consume rollout tensors, compute PPO/GRPO\nupdates, and broadcast weight versions (FSDP2, Megatron, or Archon).\n\n**Weight versioning**: async workflows require version alignment via `WeightUpdateMeta`\n(`areal/api/engine_api.py`). Critical for correctness across distributed training.\n\n**Observability**: emit metrics via `stats_tracker.get()`, persist artifacts under\n`dump_dir`, checkpoint via `areal/utils/saver.py` / `recover.py`.\n\n**Launcher / scheduler**: training requires cluster setup (local / Ray / Slurm) via\nconfigs in `areal/infra/launcher/`. See `launcher-expert` for deployment guidance.\n\n______________________________________________________________________\n\n## API & config rules\n\n*Applies to: `areal/api/**`*\n\n- **Field ordering**: required -> common optional -> rare optional -> internal (`_`\n  prefix).\n- **Validation**: `__post_init__` with `ValueError` and clear message.\n- **Backward compat**: add fields with defaults; deprecate before removing; avoid type\n  changes.\n- **CLI**: use `Literal` for enum choices; all public configs need docstrings with\n  constraints.\n\n______________________________________________________________________\n\n## Distributed code rules\n\n*Applies to: `areal/engine/**`, `areal/experimental/**`*\n\n- Never create global process groups at module level; always pass `process_group`\n  explicitly.\n- `dist.get_rank(group)` not `dist.get_rank()` when group matters.\n- DeviceMesh dimensions must match `ArchonParallelDims`: `dp_shard`, `tp`, `cp`, `ep`,\n  `etp`.\n- All-reduce: all ranks must call. Broadcast: explicit `src`. Barrier: debugging only.\n\n| Issue         | Cause                            | Fix                       |\n| ------------- | -------------------------------- | ------------------------- |\n| Hang          | Mismatched collective calls      | All ranks call same op    |\n| Wrong results | Incorrect `ReduceOp`             | Check SUM vs MEAN         |\n| OOM           | Unsharded tensor on wrong device | Verify DTensor placements |\n\nDebug env vars: `TORCH_DISTRIBUTED_DEBUG=DETAIL`, `NCCL_DEBUG=INFO`,\n`CUDA_LAUNCH_BLOCKING=1`. See the `debug-distributed` skill for the full workflow.\n\n______________________________________________________________________\n\n## Testing rules\n\n*Applies to: `**/tests/**`, `test_*.py`*\n\n| Marker                                  | When                             |\n| --------------------------------------- | -------------------------------- |\n| `@pytest.mark.slow`                     | > 10s (excluded from default CI) |\n| `@pytest.mark.slow` + `@pytest.mark.ci` | Slow but must run in CI          |\n| `@pytest.mark.asyncio`                  | Async tests                      |\n\n- Naming: `test_<what>_<condition>_<expected>()` with Arrange/Act/Assert.\n- GPU: skip gracefully (`@pytest.mark.skipif(not CUDA_AVAILABLE, reason=\"...\")`).\n- Distributed mocking: `torch.distributed.fake_pg`; don't mock FSDP/DTensor internals.\n- Assertions: `torch.testing.assert_close()` with explicit `rtol`/`atol`; prefer\n  `tmp_path`, `monkeypatch`.\n\n| Suite       | Command                       | GPU       |\n| ----------- | ----------------------------- | --------- |\n| Unit        | `pytest tests/test_*.py`      | No        |\n| GRPO        | `pytest tests/grpo/`          | Yes       |\n| FSDP        | `pytest tests/test_fsdp_*.py` | Yes       |\n| Distributed | `pytest tests/torchrun/`      | Multi-GPU |\n\n______________________________________________________________________\n\n## Collaboration & review\n\n- **Branches**: kebab-case (`feature/multi-turn-metrics`, `bugfix/fsdp-weight-sync`).\n- **Commits**: Conventional Commits (e.g., `feat:`, `fix:`, `docs:`, `gov:`), ~72 char\n  subject, imperative voice. Squash WIP before PR.\n- **Pre-merge**: full pre-commit stack; doc-only edits need at least `mdformat --check`.\n- **PRs**: tie to issue, highlight risk areas, list test commands executed, note skipped\n  suites with reasons.\n\n| Skill                | Purpose                                                |\n| -------------------- | ------------------------------------------------------ |\n| `create-pr`          | Rebase, squash, and create or update a PR              |\n| `commit-conventions` | Commit message conventions to load before `git commit` |\n| `review-pr`          | Dynamic PR review with targeted expert consultation    |\n| `translate-doc-zh`   | Translate English docs to Chinese                      |\n\n______________________________________________________________________\n\n## Reference material\n\n- **Docs portal**: <https://areal-project.github.io/AReaL/>\n- **Quickstart**: `docs/tutorial/quickstart.md`\n- **Architecture**: `docs/tutorial/gsm8k_grpo.md`\n- **Customization**: `docs/customization/*.md`\n- **Algorithms**: `docs/algorithms/*.md`\n- **Best practices**: `docs/best_practices/*.md`\n- **CLI reference**: `docs/cli_reference.md`\n- **Agent workflow**: `docs/customization/agent.md`\n"},"items":[{"name":"CLAUDE.md","path":"CLAUDE.md","title":"CLAUDE.md","content":"# CLAUDE.md - AReaL\n\n## WHAT: Project Overview\n\nAReaL is a distributed RL training framework for LLM alignment via reinforcement\nlearning.\n\n**Tech Stack**: Python 3.12+ | PyTorch | FSDP2/Megatron | SGLang/vLLM\n\n**Core Directories**:\n\n- `areal/` - Core package\n  - `api/` - Config dataclasses, workflow/engine contracts\n  - `engine/` - FSDP2, Megatron, SGLang/vLLM adapters\n    - `fsdp_utils/` - FSDP2-specific utilities (checkpoint, grad, optimizer, parallel)\n    - `megatron_utils/` - Megatron/FP8 utilities (checkpoint, pipeline, quantization)\n    - `core/` - Engine-shared utilities (distributed, lock, model, offload)\n  - `infra/` - Infrastructure (launcher, scheduler, RPC)\n    - `utils/` - Infrastructure utilities (launcher, proc, http, concurrent, slurm, ray)\n  - `workflow/` - RolloutWorkflow implementations\n  - `reward/` - Reward functions\n  - `dataset/` - Dataset loaders\n  - `utils/` - Cross-cutting utilities (logging, data, checkpoints, network, RL\n    functional)\n- `examples/` - Training scripts and configs\n- `docs/` - Jupyter Book source\n\n## WHY: Purpose\n\n- Enable efficient RL training for LLM alignment at scale\n- Async rollout + distributed training for high throughput\n- Modular design: workflows, engines, rewards, and datasets are independently extensible\n\n## HOW: Core Commands\n\n```bash\n# Check environment\npython --version              # Requires 3.12+\nuv --version                  # Install: https://docs.astral.sh/uv/\n\n# Sync dependencies\nuv sync --extra cuda          # CUDA + SGLang inference (default)\n# For vLLM: cp pyproject.vllm.toml pyproject.toml && cp uv.vllm.lock uv.lock && uv sync --extra cuda\nuv sync --group dev           # Include dev/test packages\nuv run python3 areal/tools/validate_installation.py  # Validate installation\n\n# Pre-commit hooks\npre-commit install --install-hooks  # Set up hooks (run once)\npre-commit run --all-files    # Format and lint\n\n# Run tests\n# First check GPU availability (many tests require GPU)\npython -c \"import torch; print('GPU available:', torch.cuda.is_available())\"\nuv run pytest tests/test_<topic>.py\n\n# Generate CLI docs\nuv run python docs/generate_cli_docs.py\n\n# Build docs (canonical, release-aligned)\n./docs/build_all.sh\n# Do NOT use `jupyter-book build docs/en|docs/zh` directly for final preview/release,\n# because it skips AReaL-specific static setup and output packaging.\n```\n\n## Boundaries\n\n### Constraints\n\n- Designed for distributed GPU clusters; assume containerized execution\n- Integration tests require multi-node hardware; explain skips when unavailable\n- Secrets and endpoints are managed outside the repo\n\n### Always Do\n\n- Read relevant files before modifying code\n- Run `pre-commit run --all-files` before committing\n- Follow existing code patterns in the same module\n- Add tests for new functionality\n\n### Ask First\n\n- Modifying config structures in `areal/api/cli_args.py`\n- Adding new dependencies\n- Changing launcher or scheduler logic\n- Deleting or renaming public APIs\n- Running GPU/distributed tests (check GPU first:\n  `python -c \"import torch; print('GPU available:', torch.cuda.is_available())\"`)\n\n### Never Do\n\n- Hardcode secrets, paths, or endpoints\n- Skip pre-commit hooks\n- Guess cluster configs or rebuild CUDA/driver stacks\n- Use wildcard imports (`from x import *`)\n\n## Progressive Disclosure: Detailed Guides\n\n| Task                   | Reference                                                     |\n| ---------------------- | ------------------------------------------------------------- |\n| Add Workflow           | `docs/customization/agent.md`, `areal/workflow/multi_turn.py` |\n| Add Dataset            | `docs/customization/`, `areal/dataset/gsm8k.py`               |\n| Add Reward             | `areal/api/reward_api.py`, `areal/reward/geometry3k.py`       |\n| Add Archon Model       | `areal/experimental/models/archon/qwen2/`, `qwen3/`           |\n| Algorithm Details      | `docs/algorithms/*.md`                                        |\n| Quickstart             | `docs/tutorial/quickstart.md`                                 |\n| Architecture Deep Dive | `docs/tutorial/gsm8k_grpo.md`                                 |\n| CLI Reference          | `docs/cli_reference.md`                                       |\n\n## Git Workflow\n\n- **Commits**: Conventional Commits (e.g., `feat:`, `fix:`, `docs:`, `gov:`), ~72 chars\n  subject, imperative voice, reasoning in body\n- **Squash**: Squash WIP commits before opening PR\n- **PR requirements**: Run pre-commit, document test coverage, note hardware limitations\n\n## Extended Configuration\n\nSee `.claude/agents/`, `.claude/skills/`, `.claude/commands/`, and `.claude/rules/` for\nspecialized instructions.\n\n### Agents\n\n| Agent                       | Purpose                                   | Activation Trigger                                                  |\n| --------------------------- | ----------------------------------------- | ------------------------------------------------------------------- |\n| `planner`                   | Implementation planning                   | Before multi-file changes, new features, or architectural decisions |\n| `simple-code-reviewer`      | Quick code quality checks                 | After code changes, before committing                               |\n| `code-verifier`             | Formatting/linting/tests                  | After code changes, before committing                               |\n| `fsdp-engine-expert`        | FSDPEngine implementation                 | FSDPEngine code changes or questions                                |\n| `archon-engine-expert`      | ArchonEngine implementation               | ArchonEngine code changes or questions                              |\n| `megatron-engine-expert`    | MegatronEngine implementation             | MegatronEngine code changes or questions                            |\n| `algorithm-expert`          | RL algorithms                             | GRPO/PPO/DAPO questions                                             |\n| `launcher-scheduler-expert` | Cluster launching and resource scheduling | Launcher/scheduler code changes or configuration questions          |\n\n**Stage-by-Stage Agent Guidance**:\n\n1. **Planning Stage** (Before coding): Use `planner` for architecture design and\n   implementation planning\n1. **Code Formatting & Linting** (After coding): Use `code-verifier` to automatically\n   run formatting, linting, and tests, catching syntax errors and style issues quickly\n1. **Code Quality Check** (After formatting): Use `simple-code-reviewer` for quick code\n   quality checks, focusing on logic issues and code smells\n\n### Skills (Guided Development Workflows)\n\nSkills provide step-by-step guides for common development tasks:\n\n- `/add-dataset` - Dataset loader creation guide\n- `/add-workflow` - Workflow implementation guide\n- `/add-reward` - Reward function guide\n- `/add-archon-model` - Archon engine model architecture guide\n- `/debug-distributed` - Distributed debugging guide\n- `/add-unit-tests` - Test development guide (NEW)\n\n### Commands (User-invoked Actions)\n\nCommands perform specific actions when invoked:\n\n- `/create-pr` - Rebase, squash commits, and create/update PR with intelligent messages\n- `/gen-commit-msg` - Generate commit messages from staged changes\n- `/review-pr` - Intelligent PR code review with dynamic agent allocation\n- `/translate-doc-zh` - Translate English documentation to Chinese\n\n### Rules (Code Quality Standards)\n\nProject-wide standards enforced across all code changes:\n\n- `api-config.md` - Configuration dataclass design patterns\n- `code-style.md` - Coding conventions beyond pre-commit hooks\n- `distributed.md` - Distributed training patterns and constraints\n- `testing.md` - Testing strategy and coverage requirements\n","category":"root","tokens":1939},{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"<!-- Go-to brief for AI coding agents working on AReaL. -->\n\n# AGENTS.md -- AReaL Agent Operations Guide\n\n## Quick reference\n\n**Tech stack**: Python 3.12+ | PyTorch | FSDP2 / Megatron / Archon | SGLang / vLLM\n\n```bash\n# Environment\nuv sync --extra cuda            # CUDA + SGLang inference (default); for vLLM: cp pyproject.vllm.toml pyproject.toml && cp uv.vllm.lock uv.lock && uv sync --extra cuda\nuv sync --extra sandbox         # Daytona cloud sandbox backend (optional)\nsource .venv/bin/activate        # activate venv BEFORE pre-commit or git commit if venv exists\npre-commit install --install-hooks  # hooks: Ruff, clang-format, mdformat, nbstripout, conventional-commits\npre-commit run --all-files       # lint + format everything\n\n# Tests\nuv run pytest tests/test_<topic>.py\n\n# CLI docs\nuv run python docs/generate_cli_docs.py\n\n# Docs build (canonical, release-aligned)\n./docs/build_all.sh\n# Do NOT use `jupyter-book build docs/en|docs/zh` directly for final preview/release,\n# because it skips AReaL-specific static setup and output packaging.\n```\n\n**Hard rules** -- never violate:\n\n- No wildcard imports (`from x import *`).\n- No hardcoded secrets, paths, or endpoints.\n- No skipping pre-commit hooks.\n- No guessing cluster configs or rebuilding CUDA/driver stacks.\n- Integration tests require multi-node hardware -- explain skips explicitly.\n\n**Always do**:\n\n- Read relevant files before modifying code.\n- Run `pre-commit run --all-files` before committing.\n- Follow existing code patterns in the same module.\n- Add tests for new functionality.\n- Ask for decisions and clarifications with short, structured options instead of broad\n  open-ended questions. Use the platform's native question/clarification tool if\n  available.\n\n**Ask first** before:\n\n- Modifying config structures in `areal/api/cli_args.py`.\n- Adding new dependencies.\n- Changing launcher or scheduler logic.\n- Deleting or renaming public APIs.\n\nWhen unsure, leave a `TODO(agent)` comment and note the constraint in your response.\n\n______________________________________________________________________\n\n## Repository map\n\n```\nareal/                     Core Python package\n|-- api/                   Config dataclasses, contracts, IO structs\n|-- dataset/               Stateful dataset loaders (GSM8K, Geometry3K, CLEVR, ...)\n|-- engine/                Training backends (FSDP2, Megatron) + inference adapters\n|-- experimental/          Prototype engines/workflows (Archon MoE engine)\n|-- infra/                 Launchers (Local/Ray/Slurm), schedulers, utilities\n|-- models/                Model adapters (Megatron-Core, Transformers, custom heads)\n|-- reward/                Built-in reward functions + math parsers\n|-- tests/                 Unit/integration test suites\n|-- trainer/               High-level orchestrators (PPOTrainer, SFTTrainer)\n|-- utils/                 Cross-cutting helpers (logging, data, checkpoints, RL ops)\n+-- workflow/              RolloutWorkflow implementations (RLVR, multi-turn, vision)\n\ndocs/                      Jupyter Book docs (https://areal-project.github.io/AReaL/)\nexamples/                  Training scripts and launcher recipes\n```\n\n______________________________________________________________________\n\n## Code style & patterns\n\n- **Composition over inheritance** -- keep hierarchies \\<= 2 levels; prefer delegation.\n\n| Type             | Pattern         | Example                                   |\n| ---------------- | --------------- | ----------------------------------------- |\n| Config dataclass | `XxxConfig`     | `GRPOConfig`, `FSDPEngineConfig`          |\n| Engine class     | `XxxEngine`     | `FSDPEngine`, `ArchonEngine`              |\n| Workflow class   | `XxxWorkflow`   | `RLVRWorkflow`, `MultiTurnWorkflow`       |\n| Reward function  | `xxx_reward_fn` | `gsm8k_reward_fn`, `geometry3k_reward_fn` |\n\n**Logging**: `areal.utils.logging.getLogger(name)` with **PascalCase** names -- never\n`print` or `logging.__name__`. Per-rank format: `[{Component} Rank {N}]`. Register new\nloggers with color in `areal/utils/logging.py`.\n\n**Performance**:\n\n- No GPU-CPU sync in hot paths (`.item()`, `.tolist()`, `print(tensor)`).\n- Batch ops over Python loops on tensor elements.\n- Explicit `dtype`/`device`; `torch.Size` assertions for shape validation.\n\n**Typing & imports**: explicit type hints; reuse `areal/api/cli_args.py` dataclasses; no\nwildcard imports; heavy optional deps inside functions.\n\n**Async**: rollout workflows must stay non-blocking (`await` + `aiofiles`); no sync I/O\nin `arun_episode`.\n\n______________________________________________________________________\n\n## Domain experts & skills\n\nFire the appropriate **expert subagent** or **load a skill** based on what you're\nworking on. Experts are read-only consultants with deep domain knowledge; skills are\nstep-by-step implementation guides.\n\n| Working on...                | Fire subagent      | Load skill          |\n| ---------------------------- | ------------------ | ------------------- |\n| FSDP engine code             | `fsdp-expert`      | --                  |\n| Archon engine / new model    | `archon-expert`    | `add-archon-model`  |\n| Megatron engine code         | `megatron-expert`  | --                  |\n| RL algorithms / PPO / GRPO   | `algorithm-expert` | --                  |\n| Launcher / scheduler / infra | `launcher-expert`  | `debug-distributed` |\n| New reward function          | --                 | `add-reward`        |\n| New dataset loader           | --                 | `add-dataset`       |\n| New rollout workflow         | --                 | `add-workflow`      |\n| Unit tests                   | --                 | `add-unit-tests`    |\n| Distributed debugging        | --                 | `debug-distributed` |\n\n**How to invoke experts and skills** (platform-specific):\n\n| Platform | Fire expert subagent                                                               | Load skill                                         |\n| -------- | ---------------------------------------------------------------------------------- | -------------------------------------------------- |\n| OpenCode | `task(subagent_type=\"<name>\", load_skills=[], run_in_background=true, prompt=\"…\")` | `skill(name=\"<name>\")` or `load_skills=[\"<name>\"]` |\n| Codex    | Invoke registered subagent by canonical name (see `.codex/config.toml`)            | Reference `.agents/skills/<name>/SKILL.md`         |\n\n**Harness layout**:\n\n| Component         | OpenCode                                | Codex                                                  |\n| ----------------- | --------------------------------------- | ------------------------------------------------------ |\n| Root instructions | `AGENTS.md`                             | `AGENTS.md`                                            |\n| Agent configs     | `.opencode/agents/*.md` (frontmatter)   | `.codex/config.toml` + `.codex/agents/*.toml` + `*.md` |\n| Skills            | `.opencode/skills/` + `.agents/skills/` | `.agents/skills/<name>/SKILL.md`                       |\n\nDirectly executable workflows (both platforms): `add-workflow`, `review-pr`,\n`create-pr`, `translate-doc-zh`.\n\n______________________________________________________________________\n\n## Core concepts\n\n**Trainer** orchestrator (`areal/trainer/`, `PPOTrainer`, `SFTTrainer`): manages the\ntraining loop, dataset loading, and workflow execution. Entry point:\n`examples/math/gsm8k_rl.py`.\n\n**Rollout workflows** (`areal/workflow/`, `RolloutWorkflow.arun_episode`): define how\nepisodes are generated. Use `add-workflow` skill for step-by-step guide.\n\n**Engines**: *Inference engines* handle async generation via `engine.agenerate()` and\nmanage weight updates. *Training engines* consume rollout tensors, compute PPO/GRPO\nupdates, and broadcast weight versions (FSDP2, Megatron, or Archon).\n\n**Weight versioning**: async workflows require version alignment via `WeightUpdateMeta`\n(`areal/api/engine_api.py`). Critical for correctness across distributed training.\n\n**Observability**: emit metrics via `stats_tracker.get()`, persist artifacts under\n`dump_dir`, checkpoint via `areal/utils/saver.py` / `recover.py`.\n\n**Launcher / scheduler**: training requires cluster setup (local / Ray / Slurm) via\nconfigs in `areal/infra/launcher/`. See `launcher-expert` for deployment guidance.\n\n______________________________________________________________________\n\n## API & config rules\n\n*Applies to: `areal/api/**`*\n\n- **Field ordering**: required -> common optional -> rare optional -> internal (`_`\n  prefix).\n- **Validation**: `__post_init__` with `ValueError` and clear message.\n- **Backward compat**: add fields with defaults; deprecate before removing; avoid type\n  changes.\n- **CLI**: use `Literal` for enum choices; all public configs need docstrings with\n  constraints.\n\n______________________________________________________________________\n\n## Distributed code rules\n\n*Applies to: `areal/engine/**`, `areal/experimental/**`*\n\n- Never create global process groups at module level; always pass `process_group`\n  explicitly.\n- `dist.get_rank(group)` not `dist.get_rank()` when group matters.\n- DeviceMesh dimensions must match `ArchonParallelDims`: `dp_shard`, `tp`, `cp`, `ep`,\n  `etp`.\n- All-reduce: all ranks must call. Broadcast: explicit `src`. Barrier: debugging only.\n\n| Issue         | Cause                            | Fix                       |\n| ------------- | -------------------------------- | ------------------------- |\n| Hang          | Mismatched collective calls      | All ranks call same op    |\n| Wrong results | Incorrect `ReduceOp`             | Check SUM vs MEAN         |\n| OOM           | Unsharded tensor on wrong device | Verify DTensor placements |\n\nDebug env vars: `TORCH_DISTRIBUTED_DEBUG=DETAIL`, `NCCL_DEBUG=INFO`,\n`CUDA_LAUNCH_BLOCKING=1`. See the `debug-distributed` skill for the full workflow.\n\n______________________________________________________________________\n\n## Testing rules\n\n*Applies to: `**/tests/**`, `test_*.py`*\n\n| Marker                                  | When                             |\n| --------------------------------------- | -------------------------------- |\n| `@pytest.mark.slow`                     | > 10s (excluded from default CI) |\n| `@pytest.mark.slow` + `@pytest.mark.ci` | Slow but must run in CI          |\n| `@pytest.mark.asyncio`                  | Async tests                      |\n\n- Naming: `test_<what>_<condition>_<expected>()` with Arrange/Act/Assert.\n- GPU: skip gracefully (`@pytest.mark.skipif(not CUDA_AVAILABLE, reason=\"...\")`).\n- Distributed mocking: `torch.distributed.fake_pg`; don't mock FSDP/DTensor internals.\n- Assertions: `torch.testing.assert_close()` with explicit `rtol`/`atol`; prefer\n  `tmp_path`, `monkeypatch`.\n\n| Suite       | Command                       | GPU       |\n| ----------- | ----------------------------- | --------- |\n| Unit        | `pytest tests/test_*.py`      | No        |\n| GRPO        | `pytest tests/grpo/`          | Yes       |\n| FSDP        | `pytest tests/test_fsdp_*.py` | Yes       |\n| Distributed | `pytest tests/torchrun/`      | Multi-GPU |\n\n______________________________________________________________________\n\n## Collaboration & review\n\n- **Branches**: kebab-case (`feature/multi-turn-metrics`, `bugfix/fsdp-weight-sync`).\n- **Commits**: Conventional Commits (e.g., `feat:`, `fix:`, `docs:`, `gov:`), ~72 char\n  subject, imperative voice. Squash WIP before PR.\n- **Pre-merge**: full pre-commit stack; doc-only edits need at least `mdformat --check`.\n- **PRs**: tie to issue, highlight risk areas, list test commands executed, note skipped\n  suites with reasons.\n\n| Skill                | Purpose                                                |\n| -------------------- | ------------------------------------------------------ |\n| `create-pr`          | Rebase, squash, and create or update a PR              |\n| `commit-conventions` | Commit message conventions to load before `git commit` |\n| `review-pr`          | Dynamic PR review with targeted expert consultation    |\n| `translate-doc-zh`   | Translate English docs to Chinese                      |\n\n______________________________________________________________________\n\n## Reference material\n\n- **Docs portal**: <https://areal-project.github.io/AReaL/>\n- **Quickstart**: `docs/tutorial/quickstart.md`\n- **Architecture**: `docs/tutorial/gsm8k_grpo.md`\n- **Customization**: `docs/customization/*.md`\n- **Algorithms**: `docs/algorithms/*.md`\n- **Best practices**: `docs/best_practices/*.md`\n- **CLI reference**: `docs/cli_reference.md`\n- **Agent workflow**: `docs/customization/agent.md`\n","category":"root","tokens":3155}]}