{"owner":"hiyouga","repo":"LlamaFactory","hasSkills":true,"hasMcp":false,"mcpConfig":null,"found":[".github/copilot-instructions.md"],"skills":{".github/copilot-instructions.md":"# GitHub Copilot Instructions for LLaMA Factory\n\n## Project Overview\n\nLLaMA Factory is an efficient fine-tuning framework for 100+ large language models (LLMs). It provides:\n- Support for various models: LLaMA, LLaVA, Mistral, Qwen, DeepSeek, Yi, Gemma, ChatGLM, Phi, etc.\n- Multiple training methods: pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO\n- Scalable resources: 16-bit full-tuning, freeze-tuning, LoRA and QLoRA variants\n- Advanced algorithms: GaLore, BAdam, APOLLO, Adam-mini, Muon, OFT, DoRA, etc.\n- Web UI (LLaMA Board) and CLI interfaces\n\n### Architecture Versions\n\nLLaMA Factory has two parallel architectures that can be switched via the `USE_V1` environment variable:\n\n**v0 (default)** - File hierarchy:\n- `api`, `webui` → `chat`, `eval`, `train` → `data`, `model` → `hparams` → `extras`\n\n**v1** - File hierarchy:\n- `trainers` → `core` → `accelerator`, `plugins`, `config` → `utils`\n\nSet `USE_V1=1` to enable v1 architecture.\n\n## Code Structure\n\n### v0 Architecture (Default)\n\n- `src/llamafactory/` - Main package directory\n  - `api/` - OpenAI-style API implementation\n  - `chat/` - Chat interface implementation\n  - `cli.py` - Command-line interface\n  - `data/` - Data processing and dataset handling\n  - `eval/` - Model evaluation utilities\n  - `extras/` - Additional utilities and helpers\n  - `hparams/` - Hyperparameter definitions\n  - `model/` - Model loading, patching, and utilities\n  - `train/` - Training pipeline implementation\n  - `webui/` - Gradio-based web interface\n- `src/train.py` - Training entry script (delegates to `llamafactory.train.tuner`)\n- `src/webui.py` - Web UI entry script (delegates to `llamafactory.webui.interface`)\n- `src/api.py` - API server entry script (delegates to `llamafactory.api.app`)\n- `tests/` - Test suite\n- `examples/` - Example configurations for various training scenarios\n- `data/` - Dataset definitions and examples\n\n### v1 Architecture (USE_V1=1)\n\n- `src/llamafactory/v1/` - Version 1 package directory\n  - `trainers/` - Training implementations\n  - `core/` - Core training utilities\n  - `accelerator/` - Acceleration and distributed training\n  - `plugins/` - Pluggable components (model, data, sampler, trainer)\n  - `config/` - Configuration management\n  - `utils/` - Utility functions\n\n## Development Practices\n\n### Code Style\n\n- Follow the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html)\n- Use ruff for linting and formatting\n- Line length: 119 characters\n- Indentation: 4 spaces\n- Quote style: double quotes\n- Use Google-style docstrings for documentation\n\n### Import Organization\n\n- Known first-party: `llamafactory`\n- Known third-party: `accelerate`, `datasets`, `gradio`, `numpy`, `peft`, `torch`, `transformers`, `trl`\n- Use 2 blank lines after imports\n\n### Quality Checks\n\nBefore committing code, run:\n```bash\nmake style      # Auto-fix style issues\nmake quality    # Check code quality\nmake test       # Run test suite\n```\n\nOr use the combined command:\n```bash\nmake commit     # Run pre-commit hooks\n```\n\n### Testing\n\n- Use pytest for testing\n- Tests are located in `tests/` and `tests_v1/` directories\n- Run tests with: `make test` (which runs `WANDB_DISABLED=true pytest -vv --import-mode=importlib tests/ tests_v1/`)\n- Disable wandb during testing to avoid external dependencies\n- **Note**: Training configurations require GPU machines, so training is typically not tested end-to-end. Use `make test` to validate file-level functionality.\n\n### Building\n\nBuild the package with:\n```bash\npip3 install build && python3 -m build\n```\n\n### License\n\n- All source files must include the Apache 2.0 license header\n- Check license headers with: `make license`\n\n## Common Patterns\n\n### Configuration Files\n\n- Training configurations are typically YAML or JSON files in `examples/` directory\n- Hyperparameters are defined using dataclasses in `src/llamafactory/hparams/`\n\n### Model Support\n\n- New model support is added through model patches in `src/llamafactory/model/`\n- Visual models use the visual utilities in `src/llamafactory/model/model_utils/visual.py`\n- Quantization support is in `src/llamafactory/model/model_utils/quantization.py`\n\n### Data Processing\n\n- Dataset definitions are in `data/dataset_info.json`\n- Data templates and processors are in `src/llamafactory/data/`\n\n### Training\n\n- Training pipelines are in `src/llamafactory/train/`\n- Support for different training methods: SFT, DPO, PPO, RM, PT, KTO, ORPO\n\n## Key Dependencies\n\n- Python >= 3.9.0\n- PyTorch and transformers for model handling\n- datasets for data processing\n- peft for parameter-efficient fine-tuning\n- accelerate for distributed training\n- gradio for web UI\n- trl for reinforcement learning\n- Optional: vllm/sglang for inference, flash-attention-2, unsloth, liger-kernel\n\n## Entry Points\n\n- **CLI Training**: `llamafactory-cli train --config examples/train_lora/llama3_lora_sft.yaml`\n- **Web UI**: `llamafactory-cli webui` or `python src/webui.py`\n- **API Server**: `llamafactory-cli api` or `python src/api.py`\n- **Chat Interface**: `llamafactory-cli chat --model_name_or_path MODEL_PATH`\n\n## Environment Setup\n\nFor development:\n```bash\npip install -e \".[dev]\"\n```\n\n## Important Notes\n\n- The project supports multiple backends: default PyTorch, vLLM, SGLang\n- Megatron-core training is supported via mcore_adapter\n- SwanLab and W&B are supported for experiment tracking\n- Docker support is available with pre-built images\n- Day-0/Day-1 support for latest cutting-edge models\n- Multi-modal support for vision and audio understanding tasks\n\n## Contribution Guidelines\n\n1. Fork the repository\n2. Create a development branch\n3. Set up development environment with `pip install -e \".[dev]\"`\n4. Make changes following the style guide\n5. Run quality checks: `make style && make quality`\n6. Run tests: `make test`\n7. Submit a pull request\n\n## Common Commands\n\n- `make style` - Format code\n- `make quality` - Run linters\n- `make test` - Run tests\n- `make commit` - Install and run pre-commit hooks\n- `make license` - Check license headers\n"},"files":{".github/copilot-instructions.md":"# GitHub Copilot Instructions for LLaMA Factory\n\n## Project Overview\n\nLLaMA Factory is an efficient fine-tuning framework for 100+ large language models (LLMs). It provides:\n- Support for various models: LLaMA, LLaVA, Mistral, Qwen, DeepSeek, Yi, Gemma, ChatGLM, Phi, etc.\n- Multiple training methods: pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO\n- Scalable resources: 16-bit full-tuning, freeze-tuning, LoRA and QLoRA variants\n- Advanced algorithms: GaLore, BAdam, APOLLO, Adam-mini, Muon, OFT, DoRA, etc.\n- Web UI (LLaMA Board) and CLI interfaces\n\n### Architecture Versions\n\nLLaMA Factory has two parallel architectures that can be switched via the `USE_V1` environment variable:\n\n**v0 (default)** - File hierarchy:\n- `api`, `webui` → `chat`, `eval`, `train` → `data`, `model` → `hparams` → `extras`\n\n**v1** - File hierarchy:\n- `trainers` → `core` → `accelerator`, `plugins`, `config` → `utils`\n\nSet `USE_V1=1` to enable v1 architecture.\n\n## Code Structure\n\n### v0 Architecture (Default)\n\n- `src/llamafactory/` - Main package directory\n  - `api/` - OpenAI-style API implementation\n  - `chat/` - Chat interface implementation\n  - `cli.py` - Command-line interface\n  - `data/` - Data processing and dataset handling\n  - `eval/` - Model evaluation utilities\n  - `extras/` - Additional utilities and helpers\n  - `hparams/` - Hyperparameter definitions\n  - `model/` - Model loading, patching, and utilities\n  - `train/` - Training pipeline implementation\n  - `webui/` - Gradio-based web interface\n- `src/train.py` - Training entry script (delegates to `llamafactory.train.tuner`)\n- `src/webui.py` - Web UI entry script (delegates to `llamafactory.webui.interface`)\n- `src/api.py` - API server entry script (delegates to `llamafactory.api.app`)\n- `tests/` - Test suite\n- `examples/` - Example configurations for various training scenarios\n- `data/` - Dataset definitions and examples\n\n### v1 Architecture (USE_V1=1)\n\n- `src/llamafactory/v1/` - Version 1 package directory\n  - `trainers/` - Training implementations\n  - `core/` - Core training utilities\n  - `accelerator/` - Acceleration and distributed training\n  - `plugins/` - Pluggable components (model, data, sampler, trainer)\n  - `config/` - Configuration management\n  - `utils/` - Utility functions\n\n## Development Practices\n\n### Code Style\n\n- Follow the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html)\n- Use ruff for linting and formatting\n- Line length: 119 characters\n- Indentation: 4 spaces\n- Quote style: double quotes\n- Use Google-style docstrings for documentation\n\n### Import Organization\n\n- Known first-party: `llamafactory`\n- Known third-party: `accelerate`, `datasets`, `gradio`, `numpy`, `peft`, `torch`, `transformers`, `trl`\n- Use 2 blank lines after imports\n\n### Quality Checks\n\nBefore committing code, run:\n```bash\nmake style      # Auto-fix style issues\nmake quality    # Check code quality\nmake test       # Run test suite\n```\n\nOr use the combined command:\n```bash\nmake commit     # Run pre-commit hooks\n```\n\n### Testing\n\n- Use pytest for testing\n- Tests are located in `tests/` and `tests_v1/` directories\n- Run tests with: `make test` (which runs `WANDB_DISABLED=true pytest -vv --import-mode=importlib tests/ tests_v1/`)\n- Disable wandb during testing to avoid external dependencies\n- **Note**: Training configurations require GPU machines, so training is typically not tested end-to-end. Use `make test` to validate file-level functionality.\n\n### Building\n\nBuild the package with:\n```bash\npip3 install build && python3 -m build\n```\n\n### License\n\n- All source files must include the Apache 2.0 license header\n- Check license headers with: `make license`\n\n## Common Patterns\n\n### Configuration Files\n\n- Training configurations are typically YAML or JSON files in `examples/` directory\n- Hyperparameters are defined using dataclasses in `src/llamafactory/hparams/`\n\n### Model Support\n\n- New model support is added through model patches in `src/llamafactory/model/`\n- Visual models use the visual utilities in `src/llamafactory/model/model_utils/visual.py`\n- Quantization support is in `src/llamafactory/model/model_utils/quantization.py`\n\n### Data Processing\n\n- Dataset definitions are in `data/dataset_info.json`\n- Data templates and processors are in `src/llamafactory/data/`\n\n### Training\n\n- Training pipelines are in `src/llamafactory/train/`\n- Support for different training methods: SFT, DPO, PPO, RM, PT, KTO, ORPO\n\n## Key Dependencies\n\n- Python >= 3.9.0\n- PyTorch and transformers for model handling\n- datasets for data processing\n- peft for parameter-efficient fine-tuning\n- accelerate for distributed training\n- gradio for web UI\n- trl for reinforcement learning\n- Optional: vllm/sglang for inference, flash-attention-2, unsloth, liger-kernel\n\n## Entry Points\n\n- **CLI Training**: `llamafactory-cli train --config examples/train_lora/llama3_lora_sft.yaml`\n- **Web UI**: `llamafactory-cli webui` or `python src/webui.py`\n- **API Server**: `llamafactory-cli api` or `python src/api.py`\n- **Chat Interface**: `llamafactory-cli chat --model_name_or_path MODEL_PATH`\n\n## Environment Setup\n\nFor development:\n```bash\npip install -e \".[dev]\"\n```\n\n## Important Notes\n\n- The project supports multiple backends: default PyTorch, vLLM, SGLang\n- Megatron-core training is supported via mcore_adapter\n- SwanLab and W&B are supported for experiment tracking\n- Docker support is available with pre-built images\n- Day-0/Day-1 support for latest cutting-edge models\n- Multi-modal support for vision and audio understanding tasks\n\n## Contribution Guidelines\n\n1. Fork the repository\n2. Create a development branch\n3. Set up development environment with `pip install -e \".[dev]\"`\n4. Make changes following the style guide\n5. Run quality checks: `make style && make quality`\n6. Run tests: `make test`\n7. Submit a pull request\n\n## Common Commands\n\n- `make style` - Format code\n- `make quality` - Run linters\n- `make test` - Run tests\n- `make commit` - Install and run pre-commit hooks\n- `make license` - Check license headers\n"},"items":[{"name":"copilot-instructions.md","path":".github/copilot-instructions.md","title":"copilot-instructions.md","content":"# GitHub Copilot Instructions for LLaMA Factory\n\n## Project Overview\n\nLLaMA Factory is an efficient fine-tuning framework for 100+ large language models (LLMs). It provides:\n- Support for various models: LLaMA, LLaVA, Mistral, Qwen, DeepSeek, Yi, Gemma, ChatGLM, Phi, etc.\n- Multiple training methods: pre-training, supervised fine-tuning, reward modeling, PPO, DPO, KTO, ORPO\n- Scalable resources: 16-bit full-tuning, freeze-tuning, LoRA and QLoRA variants\n- Advanced algorithms: GaLore, BAdam, APOLLO, Adam-mini, Muon, OFT, DoRA, etc.\n- Web UI (LLaMA Board) and CLI interfaces\n\n### Architecture Versions\n\nLLaMA Factory has two parallel architectures that can be switched via the `USE_V1` environment variable:\n\n**v0 (default)** - File hierarchy:\n- `api`, `webui` → `chat`, `eval`, `train` → `data`, `model` → `hparams` → `extras`\n\n**v1** - File hierarchy:\n- `trainers` → `core` → `accelerator`, `plugins`, `config` → `utils`\n\nSet `USE_V1=1` to enable v1 architecture.\n\n## Code Structure\n\n### v0 Architecture (Default)\n\n- `src/llamafactory/` - Main package directory\n  - `api/` - OpenAI-style API implementation\n  - `chat/` - Chat interface implementation\n  - `cli.py` - Command-line interface\n  - `data/` - Data processing and dataset handling\n  - `eval/` - Model evaluation utilities\n  - `extras/` - Additional utilities and helpers\n  - `hparams/` - Hyperparameter definitions\n  - `model/` - Model loading, patching, and utilities\n  - `train/` - Training pipeline implementation\n  - `webui/` - Gradio-based web interface\n- `src/train.py` - Training entry script (delegates to `llamafactory.train.tuner`)\n- `src/webui.py` - Web UI entry script (delegates to `llamafactory.webui.interface`)\n- `src/api.py` - API server entry script (delegates to `llamafactory.api.app`)\n- `tests/` - Test suite\n- `examples/` - Example configurations for various training scenarios\n- `data/` - Dataset definitions and examples\n\n### v1 Architecture (USE_V1=1)\n\n- `src/llamafactory/v1/` - Version 1 package directory\n  - `trainers/` - Training implementations\n  - `core/` - Core training utilities\n  - `accelerator/` - Acceleration and distributed training\n  - `plugins/` - Pluggable components (model, data, sampler, trainer)\n  - `config/` - Configuration management\n  - `utils/` - Utility functions\n\n## Development Practices\n\n### Code Style\n\n- Follow the [Google Python Style Guide](https://google.github.io/styleguide/pyguide.html)\n- Use ruff for linting and formatting\n- Line length: 119 characters\n- Indentation: 4 spaces\n- Quote style: double quotes\n- Use Google-style docstrings for documentation\n\n### Import Organization\n\n- Known first-party: `llamafactory`\n- Known third-party: `accelerate`, `datasets`, `gradio`, `numpy`, `peft`, `torch`, `transformers`, `trl`\n- Use 2 blank lines after imports\n\n### Quality Checks\n\nBefore committing code, run:\n```bash\nmake style      # Auto-fix style issues\nmake quality    # Check code quality\nmake test       # Run test suite\n```\n\nOr use the combined command:\n```bash\nmake commit     # Run pre-commit hooks\n```\n\n### Testing\n\n- Use pytest for testing\n- Tests are located in `tests/` and `tests_v1/` directories\n- Run tests with: `make test` (which runs `WANDB_DISABLED=true pytest -vv --import-mode=importlib tests/ tests_v1/`)\n- Disable wandb during testing to avoid external dependencies\n- **Note**: Training configurations require GPU machines, so training is typically not tested end-to-end. Use `make test` to validate file-level functionality.\n\n### Building\n\nBuild the package with:\n```bash\npip3 install build && python3 -m build\n```\n\n### License\n\n- All source files must include the Apache 2.0 license header\n- Check license headers with: `make license`\n\n## Common Patterns\n\n### Configuration Files\n\n- Training configurations are typically YAML or JSON files in `examples/` directory\n- Hyperparameters are defined using dataclasses in `src/llamafactory/hparams/`\n\n### Model Support\n\n- New model support is added through model patches in `src/llamafactory/model/`\n- Visual models use the visual utilities in `src/llamafactory/model/model_utils/visual.py`\n- Quantization support is in `src/llamafactory/model/model_utils/quantization.py`\n\n### Data Processing\n\n- Dataset definitions are in `data/dataset_info.json`\n- Data templates and processors are in `src/llamafactory/data/`\n\n### Training\n\n- Training pipelines are in `src/llamafactory/train/`\n- Support for different training methods: SFT, DPO, PPO, RM, PT, KTO, ORPO\n\n## Key Dependencies\n\n- Python >= 3.9.0\n- PyTorch and transformers for model handling\n- datasets for data processing\n- peft for parameter-efficient fine-tuning\n- accelerate for distributed training\n- gradio for web UI\n- trl for reinforcement learning\n- Optional: vllm/sglang for inference, flash-attention-2, unsloth, liger-kernel\n\n## Entry Points\n\n- **CLI Training**: `llamafactory-cli train --config examples/train_lora/llama3_lora_sft.yaml`\n- **Web UI**: `llamafactory-cli webui` or `python src/webui.py`\n- **API Server**: `llamafactory-cli api` or `python src/api.py`\n- **Chat Interface**: `llamafactory-cli chat --model_name_or_path MODEL_PATH`\n\n## Environment Setup\n\nFor development:\n```bash\npip install -e \".[dev]\"\n```\n\n## Important Notes\n\n- The project supports multiple backends: default PyTorch, vLLM, SGLang\n- Megatron-core training is supported via mcore_adapter\n- SwanLab and W&B are supported for experiment tracking\n- Docker support is available with pre-built images\n- Day-0/Day-1 support for latest cutting-edge models\n- Multi-modal support for vision and audio understanding tasks\n\n## Contribution Guidelines\n\n1. Fork the repository\n2. Create a development branch\n3. Set up development environment with `pip install -e \".[dev]\"`\n4. Make changes following the style guide\n5. Run quality checks: `make style && make quality`\n6. Run tests: `make test`\n7. Submit a pull request\n\n## Common Commands\n\n- `make style` - Format code\n- `make quality` - Run linters\n- `make test` - Run tests\n- `make commit` - Install and run pre-commit hooks\n- `make license` - Check license headers\n","category":".github","tokens":1512}]}