{"owner":"axolotl-ai-cloud","repo":"axolotl","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":["AGENTS.md"],"skills":{"AGENTS.md":"# Axolotl\n\nFine-tuning framework for LLMs. Config-driven: every training run is defined by a single YAML file.\n\n## Tech Stack\n\nPython, PyTorch, HuggingFace Transformers, TRL, PEFT (LoRA/QLoRA), DeepSpeed, FSDP, vLLM (for GRPO generation).\n\n## Commands\n\n```bash\naxolotl train config.yaml              # Train (single or multi-GPU, auto-detected)\naxolotl preprocess config.yaml         # Tokenize dataset and validate config\naxolotl preprocess config.yaml --debug # Inspect tokenized samples and label masking\naxolotl inference config.yaml          # Interactive inference\naxolotl merge-lora config.yaml         # Merge LoRA adapter into base model\naxolotl vllm-serve config.yaml         # Start vLLM server for GRPO/EBFT training\naxolotl fetch examples                 # Download example configs\naxolotl agent-docs                     # Show agent-optimized docs (bundled with pip package)\naxolotl agent-docs grpo                # Topic-specific agent reference\naxolotl config-schema                  # Dump config JSON schema\n```\n\n## Training Methods\n\n| Method | Config Key | When to Use |\n|--------|-----------|-------------|\n| SFT | *(default)* | Input-output pairs, instruction tuning |\n| DPO/IPO | `rl: dpo` / `rl: dpo, dpo_loss_type: [\"ipo\"]` | Paired preference data (chosen vs rejected) |\n| KTO | `rl: kto` | Unpaired binary preference labels |\n| ORPO | `rl: orpo` | Single-stage alignment, no ref model |\n| GRPO | `rl: grpo` | RL with verifiable reward functions (math, code) |\n| EBFT | `rl: ebft` | Feature-matching rewards from internal representations |\n\nAgent-specific references:\n- [docs/agents/sft.md](docs/agents/sft.md) — supervised fine-tuning\n- [docs/agents/preference_tuning.md](docs/agents/preference_tuning.md) — DPO, IPO, KTO, ORPO, SimPO\n- [docs/agents/grpo.md](docs/agents/grpo.md) — GRPO online RL with reward functions\n- [docs/agents/reward_modelling.md](docs/agents/reward_modelling.md) — outcome and process reward models\n- [docs/agents/pretraining.md](docs/agents/pretraining.md) — continual pretraining\n- [docs/agents/model_architectures.md](docs/agents/model_architectures.md) — model-specific quirks (Gemma4, Qwen3.5 MoE, etc.)\n- [docs/agents/new_model_support.md](docs/agents/new_model_support.md) — debugging and adding support for new model architectures\n\n## Config Pattern\n\nAll training is config-driven. A YAML file specifies model, adapter, dataset(s), and hyperparameters:\n\n```yaml\nbase_model: meta-llama/Llama-3.1-8B-Instruct\nadapter: lora                    # or qlora, or omit for full fine-tune\ndatasets:\n  - path: my_dataset\n    type: chat_template          # prompt strategy (see docs/dataset-formats/)\noutput_dir: ./outputs/lora-out\n```\n\nConfig schema: `src/axolotl/utils/schemas/config.py` (AxolotlInputConfig).\n\n## Project Structure\n\n```\nsrc/axolotl/\n  cli/                           # CLI entry points (train, preprocess, inference, merge_lora, vllm_serve)\n  core/\n    builders/                    # TrainerBuilder classes (causal.py for SFT, rl.py for RLHF)\n    trainers/                    # Trainer classes, mixins (optimizer, scheduler, packing)\n      dpo/                       # DPO trainer and config\n      grpo/                      # GRPO trainer and sampler\n  loaders/                       # Model, tokenizer, adapter, processor loading\n  prompt_strategies/             # Dataset format handlers (chat_template, alpaca, dpo/, kto/, orpo/)\n  utils/schemas/                 # Pydantic config schemas (config, model, training, peft, trl, fsdp)\n  integrations/                  # Plugins (liger, cut_cross_entropy, swanlab, nemo_gym)\n  monkeypatch/                   # Runtime patches for HF transformers\n\nexamples/                        # Example YAML configs by model (llama-3/, qwen2/, mistral/, ebft/)\ndeepspeed_configs/               # DeepSpeed JSON configs (zero2, zero3)\ndocs/                            # Quarto documentation site\n```\n\n## Linting & Tests\n\nThe repo pins CI tool versions in `.pre-commit-config.yaml` — never run system `ruff`/`mypy`.\n\n- `pre-commit run --all-files` — ruff, ruff-format, mypy, bandit at the CI-pinned versions\n- `uvx ruff@<rev> check --fix && uvx ruff@<rev> format` — auto-fix with the pinned ruff (`<rev>` = the `ruff-pre-commit` rev in `.pre-commit-config.yaml`)\n- `pytest -m 'not slow' --ignore=tests/e2e tests/` — CPU suite\n\nSetup, CI matrix, GPU e2e, skip-CI keywords: [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md).\n\n## Code Conventions\n\n- Config-driven: features are toggled via YAML, not code changes\n- Prompt strategies: `src/axolotl/prompt_strategies/` — each `type:` value maps to a function\n- Plugin system: `plugins:` list in config loads integration modules\n- Trainer mixins: `core/trainers/mixins/` for composable trainer behaviors\n- Schemas: all config validation via Pydantic in `utils/schemas/`\n\n## Comment Style\n\n- Default to no comment. Only add one when the WHY is non-obvious (hidden constraint, subtle invariant, workaround for a specific bug).\n- Don't explain WHAT the code does — names and types already do that.\n- Don't reference the current task, PR, or callers (e.g. \"added for X\", \"used by Y\", \"fixes #123\"). Those belong in commit messages / PR descriptions and rot fast.\n- Prefer one short line max.\n- Don't add planning/decision/analysis markdown files unless explicitly requested.\n\n## Key Documentation\n\n- [Getting Started](docs/getting-started.qmd) — quickstart tutorial\n- [Choosing a Method](docs/choosing_method.qmd) — SFT vs DPO vs GRPO decision guide\n- [Support Matrix](docs/support-matrix.qmd) — what Axolotl supports, feature couplings, and known gaps\n- [Config Reference](docs/config-reference.qmd) — all config options\n- [Dataset Formats](docs/dataset-formats/) — chat_template, alpaca, input_output, completion\n- [RLHF](docs/rlhf.qmd) — DPO, KTO, ORPO, GRPO, EBFT configs and dataset formats\n- [GRPO Deep Dive](docs/grpo.qmd) — async training, custom rewards, scaling\n- [vLLM Serving](docs/vllm_serving.qmd) — vLLM setup for GRPO/EBFT\n- [Multi-GPU](docs/multi-gpu.qmd) — FSDP and DeepSpeed\n- [Training Stability](docs/training_stability.qmd) — debugging loss, NaN, OOM\n- [Debugging](docs/debugging.qmd) — VSCode setup, Docker debugging\n"},"files":{"AGENTS.md":"# Axolotl\n\nFine-tuning framework for LLMs. Config-driven: every training run is defined by a single YAML file.\n\n## Tech Stack\n\nPython, PyTorch, HuggingFace Transformers, TRL, PEFT (LoRA/QLoRA), DeepSpeed, FSDP, vLLM (for GRPO generation).\n\n## Commands\n\n```bash\naxolotl train config.yaml              # Train (single or multi-GPU, auto-detected)\naxolotl preprocess config.yaml         # Tokenize dataset and validate config\naxolotl preprocess config.yaml --debug # Inspect tokenized samples and label masking\naxolotl inference config.yaml          # Interactive inference\naxolotl merge-lora config.yaml         # Merge LoRA adapter into base model\naxolotl vllm-serve config.yaml         # Start vLLM server for GRPO/EBFT training\naxolotl fetch examples                 # Download example configs\naxolotl agent-docs                     # Show agent-optimized docs (bundled with pip package)\naxolotl agent-docs grpo                # Topic-specific agent reference\naxolotl config-schema                  # Dump config JSON schema\n```\n\n## Training Methods\n\n| Method | Config Key | When to Use |\n|--------|-----------|-------------|\n| SFT | *(default)* | Input-output pairs, instruction tuning |\n| DPO/IPO | `rl: dpo` / `rl: dpo, dpo_loss_type: [\"ipo\"]` | Paired preference data (chosen vs rejected) |\n| KTO | `rl: kto` | Unpaired binary preference labels |\n| ORPO | `rl: orpo` | Single-stage alignment, no ref model |\n| GRPO | `rl: grpo` | RL with verifiable reward functions (math, code) |\n| EBFT | `rl: ebft` | Feature-matching rewards from internal representations |\n\nAgent-specific references:\n- [docs/agents/sft.md](docs/agents/sft.md) — supervised fine-tuning\n- [docs/agents/preference_tuning.md](docs/agents/preference_tuning.md) — DPO, IPO, KTO, ORPO, SimPO\n- [docs/agents/grpo.md](docs/agents/grpo.md) — GRPO online RL with reward functions\n- [docs/agents/reward_modelling.md](docs/agents/reward_modelling.md) — outcome and process reward models\n- [docs/agents/pretraining.md](docs/agents/pretraining.md) — continual pretraining\n- [docs/agents/model_architectures.md](docs/agents/model_architectures.md) — model-specific quirks (Gemma4, Qwen3.5 MoE, etc.)\n- [docs/agents/new_model_support.md](docs/agents/new_model_support.md) — debugging and adding support for new model architectures\n\n## Config Pattern\n\nAll training is config-driven. A YAML file specifies model, adapter, dataset(s), and hyperparameters:\n\n```yaml\nbase_model: meta-llama/Llama-3.1-8B-Instruct\nadapter: lora                    # or qlora, or omit for full fine-tune\ndatasets:\n  - path: my_dataset\n    type: chat_template          # prompt strategy (see docs/dataset-formats/)\noutput_dir: ./outputs/lora-out\n```\n\nConfig schema: `src/axolotl/utils/schemas/config.py` (AxolotlInputConfig).\n\n## Project Structure\n\n```\nsrc/axolotl/\n  cli/                           # CLI entry points (train, preprocess, inference, merge_lora, vllm_serve)\n  core/\n    builders/                    # TrainerBuilder classes (causal.py for SFT, rl.py for RLHF)\n    trainers/                    # Trainer classes, mixins (optimizer, scheduler, packing)\n      dpo/                       # DPO trainer and config\n      grpo/                      # GRPO trainer and sampler\n  loaders/                       # Model, tokenizer, adapter, processor loading\n  prompt_strategies/             # Dataset format handlers (chat_template, alpaca, dpo/, kto/, orpo/)\n  utils/schemas/                 # Pydantic config schemas (config, model, training, peft, trl, fsdp)\n  integrations/                  # Plugins (liger, cut_cross_entropy, swanlab, nemo_gym)\n  monkeypatch/                   # Runtime patches for HF transformers\n\nexamples/                        # Example YAML configs by model (llama-3/, qwen2/, mistral/, ebft/)\ndeepspeed_configs/               # DeepSpeed JSON configs (zero2, zero3)\ndocs/                            # Quarto documentation site\n```\n\n## Linting & Tests\n\nThe repo pins CI tool versions in `.pre-commit-config.yaml` — never run system `ruff`/`mypy`.\n\n- `pre-commit run --all-files` — ruff, ruff-format, mypy, bandit at the CI-pinned versions\n- `uvx ruff@<rev> check --fix && uvx ruff@<rev> format` — auto-fix with the pinned ruff (`<rev>` = the `ruff-pre-commit` rev in `.pre-commit-config.yaml`)\n- `pytest -m 'not slow' --ignore=tests/e2e tests/` — CPU suite\n\nSetup, CI matrix, GPU e2e, skip-CI keywords: [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md).\n\n## Code Conventions\n\n- Config-driven: features are toggled via YAML, not code changes\n- Prompt strategies: `src/axolotl/prompt_strategies/` — each `type:` value maps to a function\n- Plugin system: `plugins:` list in config loads integration modules\n- Trainer mixins: `core/trainers/mixins/` for composable trainer behaviors\n- Schemas: all config validation via Pydantic in `utils/schemas/`\n\n## Comment Style\n\n- Default to no comment. Only add one when the WHY is non-obvious (hidden constraint, subtle invariant, workaround for a specific bug).\n- Don't explain WHAT the code does — names and types already do that.\n- Don't reference the current task, PR, or callers (e.g. \"added for X\", \"used by Y\", \"fixes #123\"). Those belong in commit messages / PR descriptions and rot fast.\n- Prefer one short line max.\n- Don't add planning/decision/analysis markdown files unless explicitly requested.\n\n## Key Documentation\n\n- [Getting Started](docs/getting-started.qmd) — quickstart tutorial\n- [Choosing a Method](docs/choosing_method.qmd) — SFT vs DPO vs GRPO decision guide\n- [Support Matrix](docs/support-matrix.qmd) — what Axolotl supports, feature couplings, and known gaps\n- [Config Reference](docs/config-reference.qmd) — all config options\n- [Dataset Formats](docs/dataset-formats/) — chat_template, alpaca, input_output, completion\n- [RLHF](docs/rlhf.qmd) — DPO, KTO, ORPO, GRPO, EBFT configs and dataset formats\n- [GRPO Deep Dive](docs/grpo.qmd) — async training, custom rewards, scaling\n- [vLLM Serving](docs/vllm_serving.qmd) — vLLM setup for GRPO/EBFT\n- [Multi-GPU](docs/multi-gpu.qmd) — FSDP and DeepSpeed\n- [Training Stability](docs/training_stability.qmd) — debugging loss, NaN, OOM\n- [Debugging](docs/debugging.qmd) — VSCode setup, Docker debugging\n"},"items":[{"name":"AGENTS.md","path":"AGENTS.md","title":"AGENTS.md","content":"# Axolotl\n\nFine-tuning framework for LLMs. Config-driven: every training run is defined by a single YAML file.\n\n## Tech Stack\n\nPython, PyTorch, HuggingFace Transformers, TRL, PEFT (LoRA/QLoRA), DeepSpeed, FSDP, vLLM (for GRPO generation).\n\n## Commands\n\n```bash\naxolotl train config.yaml              # Train (single or multi-GPU, auto-detected)\naxolotl preprocess config.yaml         # Tokenize dataset and validate config\naxolotl preprocess config.yaml --debug # Inspect tokenized samples and label masking\naxolotl inference config.yaml          # Interactive inference\naxolotl merge-lora config.yaml         # Merge LoRA adapter into base model\naxolotl vllm-serve config.yaml         # Start vLLM server for GRPO/EBFT training\naxolotl fetch examples                 # Download example configs\naxolotl agent-docs                     # Show agent-optimized docs (bundled with pip package)\naxolotl agent-docs grpo                # Topic-specific agent reference\naxolotl config-schema                  # Dump config JSON schema\n```\n\n## Training Methods\n\n| Method | Config Key | When to Use |\n|--------|-----------|-------------|\n| SFT | *(default)* | Input-output pairs, instruction tuning |\n| DPO/IPO | `rl: dpo` / `rl: dpo, dpo_loss_type: [\"ipo\"]` | Paired preference data (chosen vs rejected) |\n| KTO | `rl: kto` | Unpaired binary preference labels |\n| ORPO | `rl: orpo` | Single-stage alignment, no ref model |\n| GRPO | `rl: grpo` | RL with verifiable reward functions (math, code) |\n| EBFT | `rl: ebft` | Feature-matching rewards from internal representations |\n\nAgent-specific references:\n- [docs/agents/sft.md](docs/agents/sft.md) — supervised fine-tuning\n- [docs/agents/preference_tuning.md](docs/agents/preference_tuning.md) — DPO, IPO, KTO, ORPO, SimPO\n- [docs/agents/grpo.md](docs/agents/grpo.md) — GRPO online RL with reward functions\n- [docs/agents/reward_modelling.md](docs/agents/reward_modelling.md) — outcome and process reward models\n- [docs/agents/pretraining.md](docs/agents/pretraining.md) — continual pretraining\n- [docs/agents/model_architectures.md](docs/agents/model_architectures.md) — model-specific quirks (Gemma4, Qwen3.5 MoE, etc.)\n- [docs/agents/new_model_support.md](docs/agents/new_model_support.md) — debugging and adding support for new model architectures\n\n## Config Pattern\n\nAll training is config-driven. A YAML file specifies model, adapter, dataset(s), and hyperparameters:\n\n```yaml\nbase_model: meta-llama/Llama-3.1-8B-Instruct\nadapter: lora                    # or qlora, or omit for full fine-tune\ndatasets:\n  - path: my_dataset\n    type: chat_template          # prompt strategy (see docs/dataset-formats/)\noutput_dir: ./outputs/lora-out\n```\n\nConfig schema: `src/axolotl/utils/schemas/config.py` (AxolotlInputConfig).\n\n## Project Structure\n\n```\nsrc/axolotl/\n  cli/                           # CLI entry points (train, preprocess, inference, merge_lora, vllm_serve)\n  core/\n    builders/                    # TrainerBuilder classes (causal.py for SFT, rl.py for RLHF)\n    trainers/                    # Trainer classes, mixins (optimizer, scheduler, packing)\n      dpo/                       # DPO trainer and config\n      grpo/                      # GRPO trainer and sampler\n  loaders/                       # Model, tokenizer, adapter, processor loading\n  prompt_strategies/             # Dataset format handlers (chat_template, alpaca, dpo/, kto/, orpo/)\n  utils/schemas/                 # Pydantic config schemas (config, model, training, peft, trl, fsdp)\n  integrations/                  # Plugins (liger, cut_cross_entropy, swanlab, nemo_gym)\n  monkeypatch/                   # Runtime patches for HF transformers\n\nexamples/                        # Example YAML configs by model (llama-3/, qwen2/, mistral/, ebft/)\ndeepspeed_configs/               # DeepSpeed JSON configs (zero2, zero3)\ndocs/                            # Quarto documentation site\n```\n\n## Linting & Tests\n\nThe repo pins CI tool versions in `.pre-commit-config.yaml` — never run system `ruff`/`mypy`.\n\n- `pre-commit run --all-files` — ruff, ruff-format, mypy, bandit at the CI-pinned versions\n- `uvx ruff@<rev> check --fix && uvx ruff@<rev> format` — auto-fix with the pinned ruff (`<rev>` = the `ruff-pre-commit` rev in `.pre-commit-config.yaml`)\n- `pytest -m 'not slow' --ignore=tests/e2e tests/` — CPU suite\n\nSetup, CI matrix, GPU e2e, skip-CI keywords: [.github/CONTRIBUTING.md](.github/CONTRIBUTING.md).\n\n## Code Conventions\n\n- Config-driven: features are toggled via YAML, not code changes\n- Prompt strategies: `src/axolotl/prompt_strategies/` — each `type:` value maps to a function\n- Plugin system: `plugins:` list in config loads integration modules\n- Trainer mixins: `core/trainers/mixins/` for composable trainer behaviors\n- Schemas: all config validation via Pydantic in `utils/schemas/`\n\n## Comment Style\n\n- Default to no comment. Only add one when the WHY is non-obvious (hidden constraint, subtle invariant, workaround for a specific bug).\n- Don't explain WHAT the code does — names and types already do that.\n- Don't reference the current task, PR, or callers (e.g. \"added for X\", \"used by Y\", \"fixes #123\"). Those belong in commit messages / PR descriptions and rot fast.\n- Prefer one short line max.\n- Don't add planning/decision/analysis markdown files unless explicitly requested.\n\n## Key Documentation\n\n- [Getting Started](docs/getting-started.qmd) — quickstart tutorial\n- [Choosing a Method](docs/choosing_method.qmd) — SFT vs DPO vs GRPO decision guide\n- [Support Matrix](docs/support-matrix.qmd) — what Axolotl supports, feature couplings, and known gaps\n- [Config Reference](docs/config-reference.qmd) — all config options\n- [Dataset Formats](docs/dataset-formats/) — chat_template, alpaca, input_output, completion\n- [RLHF](docs/rlhf.qmd) — DPO, KTO, ORPO, GRPO, EBFT configs and dataset formats\n- [GRPO Deep Dive](docs/grpo.qmd) — async training, custom rewards, scaling\n- [vLLM Serving](docs/vllm_serving.qmd) — vLLM setup for GRPO/EBFT\n- [Multi-GPU](docs/multi-gpu.qmd) — FSDP and DeepSpeed\n- [Training Stability](docs/training_stability.qmd) — debugging loss, NaN, OOM\n- [Debugging](docs/debugging.qmd) — VSCode setup, Docker debugging\n","category":"root","tokens":1546}]}