{"owner":"Orchestra-Research","repo":"AI-Research-SKILLs","hasSkills":true,"totalSkillsCount":466,"totalTokensCount":163955,"categories":["anthropic-skill","claude-rule","marketplace","generic","plugin-manifest"],"hasMcp":false,"mcpConfig":null,"found":["01-model-architecture/mamba/SKILL.md","01-model-architecture/rwkv/SKILL.md","01-model-architecture/nanogpt/SKILL.md","01-model-architecture/torchtitan/SKILL.md","01-model-architecture/litgpt/SKILL.md","02-tokenization/huggingface-tokenizers/SKILL.md","0-autoresearch-skill/SKILL.md","03-fine-tuning/unsloth/SKILL.md","03-fine-tuning/llama-factory/SKILL.md","02-tokenization/sentencepiece/SKILL.md","03-fine-tuning/axolotl/SKILL.md","03-fine-tuning/peft/SKILL.md","04-mechanistic-interpretability/transformer-lens/SKILL.md","04-mechanistic-interpretability/saelens/SKILL.md","04-mechanistic-interpretability/pyvene/SKILL.md","04-mechanistic-interpretability/nnsight/SKILL.md","05-data-processing/nemo-curator/SKILL.md","05-data-processing/ray-data/SKILL.md","06-post-training/verl/SKILL.md","06-post-training/trl-fine-tuning/SKILL.md","06-post-training/torchforge/SKILL.md","06-post-training/slime/SKILL.md","06-post-training/simpo/SKILL.md","06-post-training/openrlhf/SKILL.md","06-post-training/miles/SKILL.md","06-post-training/grpo-rl-training/SKILL.md","07-safety-alignment/constitutional-ai/SKILL.md","07-safety-alignment/llamaguard/SKILL.md","07-safety-alignment/nemo-guardrails/SKILL.md","07-safety-alignment/prompt-guard/SKILL.md","08-distributed-training/ray-train/SKILL.md","08-distributed-training/pytorch-lightning/SKILL.md","08-distributed-training/pytorch-fsdp2/SKILL.md","08-distributed-training/megatron-core/SKILL.md","08-distributed-training/deepspeed/SKILL.md","08-distributed-training/accelerate/SKILL.md","09-infrastructure/lambda-labs/SKILL.md","09-infrastructure/modal/SKILL.md","09-infrastructure/skypilot/SKILL.md","10-optimization/awq/SKILL.md","10-optimization/bitsandbytes/SKILL.md","10-optimization/flash-attention/SKILL.md","10-optimization/gguf/SKILL.md","10-optimization/gptq/SKILL.md","10-optimization/hqq/SKILL.md","10-optimization/ml-training-recipes/SKILL.md","11-evaluation/bigcode-evaluation-harness/SKILL.md","11-evaluation/lm-evaluation-harness/SKILL.md","11-evaluation/nemo-evaluator/SKILL.md","12-inference-serving/llama-cpp/SKILL.md","12-inference-serving/sglang/SKILL.md","12-inference-serving/tensorrt-llm/SKILL.md","12-inference-serving/vllm/SKILL.md","13-mlops/mlflow/SKILL.md","13-mlops/swanlab/SKILL.md","13-mlops/tensorboard/SKILL.md","13-mlops/weights-and-biases/SKILL.md","14-agents/a-evolve/SKILL.md","14-agents/autogpt/SKILL.md","14-agents/crewai/SKILL.md","14-agents/langchain/SKILL.md","14-agents/llamaindex/SKILL.md","15-rag/chroma/SKILL.md","15-rag/faiss/SKILL.md","15-rag/pinecone/SKILL.md","15-rag/qdrant/SKILL.md","15-rag/sentence-transformers/SKILL.md","16-prompt-engineering/dspy/SKILL.md","16-prompt-engineering/guidance/SKILL.md","16-prompt-engineering/instructor/SKILL.md","16-prompt-engineering/outlines/SKILL.md","17-observability/langsmith/SKILL.md","17-observability/phoenix/SKILL.md","18-multimodal/audiocraft/SKILL.md","18-multimodal/blip-2/SKILL.md","18-multimodal/clip/SKILL.md","18-multimodal/cosmos-policy/SKILL.md","18-multimodal/llava/SKILL.md","18-multimodal/openpi/SKILL.md","18-multimodal/openvla-oft/SKILL.md","18-multimodal/segment-anything/SKILL.md","18-multimodal/stable-diffusion/SKILL.md","18-multimodal/whisper/SKILL.md","19-emerging-techniques/knowledge-distillation/SKILL.md","19-emerging-techniques/long-context/SKILL.md","19-emerging-techniques/model-merging/SKILL.md","19-emerging-techniques/model-pruning/SKILL.md","19-emerging-techniques/moe-training/SKILL.md","19-emerging-techniques/speculative-decoding/SKILL.md","20-ml-paper-writing/systems-paper-writing/SKILL.md","20-ml-paper-writing/presenting-conference-talks/SKILL.md","20-ml-paper-writing/ml-paper-writing/SKILL.md","20-ml-paper-writing/academic-plotting/SKILL.md","21-research-ideation/creative-thinking-for-research/SKILL.md","21-research-ideation/brainstorming-research-ideas/SKILL.md","22-agent-native-research-artifact/rigor-reviewer/SKILL.md","22-agent-native-research-artifact/research-manager/SKILL.md","22-agent-native-research-artifact/compiler/SKILL.md","CLAUDE.md",".claude-plugin/marketplace.json","0-autoresearch-skill/README.md","0-autoresearch-skill/references/agent-continuity.md","0-autoresearch-skill/references/progress-reporting.md","0-autoresearch-skill/references/skill-routing.md","packages/ai-research-skills/package.json","packages/ai-research-skills/package-lock.json","packages/ai-research-skills/README.md","01-model-architecture/litgpt/README.md","01-model-architecture/mamba/README.md","01-model-architecture/nanogpt/README.md","01-model-architecture/rwkv/README.md","01-model-architecture/torchtitan/README.md","02-tokenization/huggingface-tokenizers/README.md","02-tokenization/sentencepiece/README.md","03-fine-tuning/axolotl/README.md","03-fine-tuning/llama-factory/README.md","03-fine-tuning/peft/README.md","03-fine-tuning/unsloth/README.md","04-mechanistic-interpretability/nnsight/README.md","04-mechanistic-interpretability/pyvene/README.md","04-mechanistic-interpretability/saelens/README.md","04-mechanistic-interpretability/transformer-lens/README.md","05-data-processing/nemo-curator/README.md","05-data-processing/ray-data/README.md","06-post-training/grpo-rl-training/README.md","06-post-training/miles/README.md","06-post-training/openrlhf/README.md","06-post-training/simpo/README.md","06-post-training/slime/README.md","06-post-training/torchforge/README.md","06-post-training/trl-fine-tuning/README.md","06-post-training/verl/README.md","07-safety-alignment/constitutional-ai/README.md","07-safety-alignment/llamaguard/README.md","07-safety-alignment/nemo-guardrails/README.md","07-safety-alignment/prompt-guard/README.md","08-distributed-training/accelerate/README.md","08-distributed-training/deepspeed/README.md","08-distributed-training/megatron-core/README.md","08-distributed-training/pytorch-fsdp2/README.md","08-distributed-training/pytorch-lightning/README.md","08-distributed-training/ray-train/README.md","09-infrastructure/lambda-labs/README.md","09-infrastructure/modal/README.md","09-infrastructure/skypilot/README.md","10-optimization/awq/README.md","10-optimization/bitsandbytes/README.md","10-optimization/flash-attention/README.md","10-optimization/gguf/README.md","10-optimization/gptq/README.md","10-optimization/hqq/README.md","10-optimization/ml-training-recipes/README.md","11-evaluation/bigcode-evaluation-harness/README.md","11-evaluation/lm-evaluation-harness/README.md","11-evaluation/nemo-evaluator/README.md","12-inference-serving/llama-cpp/README.md","12-inference-serving/sglang/README.md","12-inference-serving/tensorrt-llm/README.md","12-inference-serving/vllm/README.md","13-mlops/mlflow/README.md","13-mlops/tensorboard/README.md","13-mlops/weights-and-biases/README.md","14-agents/autogpt/README.md","14-agents/crewai/README.md","14-agents/langchain/README.md","14-agents/llamaindex/README.md","15-rag/chroma/README.md","15-rag/faiss/README.md","15-rag/pinecone/README.md","15-rag/qdrant/README.md","15-rag/sentence-transformers/README.md","16-prompt-engineering/dspy/README.md","16-prompt-engineering/guidance/README.md","16-prompt-engineering/instructor/README.md","16-prompt-engineering/outlines/README.md","17-observability/langsmith/README.md","17-observability/phoenix/README.md","18-multimodal/audiocraft/README.md","18-multimodal/blip-2/README.md","18-multimodal/clip/README.md","18-multimodal/cosmos-policy/README.md","18-multimodal/llava/README.md","18-multimodal/openpi/README.md","18-multimodal/openvla-oft/README.md","18-multimodal/segment-anything/README.md","18-multimodal/stable-diffusion/README.md","18-multimodal/whisper/README.md","19-emerging-techniques/knowledge-distillation/README.md","19-emerging-techniques/long-context/README.md","19-emerging-techniques/model-merging/README.md","19-emerging-techniques/model-pruning/README.md","19-emerging-techniques/moe-training/README.md","19-emerging-techniques/speculative-decoding/README.md","20-ml-paper-writing/ml-paper-writing/README.md","20-ml-paper-writing/academic-plotting/README.md","20-ml-paper-writing/systems-paper-writing/README.md","20-ml-paper-writing/presenting-conference-talks/README.md","21-research-ideation/brainstorming-research-ideas/README.md","21-research-ideation/creative-thinking-for-research/README.md","22-agent-native-research-artifact/compiler/README.md","22-agent-native-research-artifact/research-manager/README.md","22-agent-native-research-artifact/rigor-reviewer/README.md","01-model-architecture/litgpt/references/custom-models.md","01-model-architecture/litgpt/references/distributed-training.md","01-model-architecture/litgpt/references/supported-models.md","01-model-architecture/litgpt/references/training-recipes.md","01-model-architecture/mamba/references/architecture-details.md","01-model-architecture/mamba/references/benchmarks.md","01-model-architecture/mamba/references/training-guide.md","01-model-architecture/nanogpt/references/architecture.md","01-model-architecture/nanogpt/references/data.md","01-model-architecture/nanogpt/references/training.md","01-model-architecture/rwkv/references/architecture-details.md","01-model-architecture/rwkv/references/rwkv7.md","01-model-architecture/rwkv/references/state-management.md","01-model-architecture/torchtitan/references/checkpoint.md","01-model-architecture/torchtitan/references/custom-models.md","01-model-architecture/torchtitan/references/float8.md","01-model-architecture/torchtitan/references/fsdp.md","02-tokenization/huggingface-tokenizers/references/algorithms.md","02-tokenization/huggingface-tokenizers/references/integration.md","02-tokenization/huggingface-tokenizers/references/pipeline.md","02-tokenization/huggingface-tokenizers/references/training.md","02-tokenization/sentencepiece/references/algorithms.md","02-tokenization/sentencepiece/references/training.md","03-fine-tuning/unsloth/references/index.md","03-fine-tuning/unsloth/references/llms.md","03-fine-tuning/unsloth/references/llms-full.md","03-fine-tuning/unsloth/references/llms-txt.md","03-fine-tuning/peft/references/advanced-usage.md","03-fine-tuning/peft/references/troubleshooting.md","03-fine-tuning/llama-factory/references/_images.md","03-fine-tuning/llama-factory/references/advanced.md","03-fine-tuning/llama-factory/references/getting_started.md","03-fine-tuning/llama-factory/references/index.md","03-fine-tuning/llama-factory/references/other.md","03-fine-tuning/axolotl/references/api.md","03-fine-tuning/axolotl/references/dataset-formats.md","03-fine-tuning/axolotl/references/index.md","03-fine-tuning/axolotl/references/other.md","04-mechanistic-interpretability/transformer-lens/references/api.md","04-mechanistic-interpretability/transformer-lens/references/README.md","04-mechanistic-interpretability/transformer-lens/references/tutorials.md","04-mechanistic-interpretability/saelens/references/api.md","04-mechanistic-interpretability/saelens/references/README.md","04-mechanistic-interpretability/saelens/references/tutorials.md","04-mechanistic-interpretability/pyvene/references/api.md","04-mechanistic-interpretability/pyvene/references/README.md","04-mechanistic-interpretability/pyvene/references/tutorials.md","04-mechanistic-interpretability/nnsight/references/api.md","04-mechanistic-interpretability/nnsight/references/README.md","04-mechanistic-interpretability/nnsight/references/tutorials.md","05-data-processing/nemo-curator/references/deduplication.md","05-data-processing/nemo-curator/references/filtering.md","05-data-processing/ray-data/references/integration.md","05-data-processing/ray-data/references/transformations.md","06-post-training/verl/references/api-reference.md","06-post-training/verl/references/troubleshooting.md","06-post-training/trl-fine-tuning/references/dpo-variants.md","06-post-training/trl-fine-tuning/references/online-rl.md","06-post-training/trl-fine-tuning/references/reward-modeling.md","06-post-training/trl-fine-tuning/references/sft-training.md","06-post-training/torchforge/references/api-reference.md","06-post-training/torchforge/references/troubleshooting.md","06-post-training/slime/references/api-reference.md","06-post-training/slime/references/troubleshooting.md","06-post-training/simpo/references/datasets.md","06-post-training/simpo/references/hyperparameters.md","06-post-training/simpo/references/loss-functions.md","06-post-training/openrlhf/references/algorithm-comparison.md","06-post-training/openrlhf/references/custom-rewards.md","06-post-training/openrlhf/references/hybrid-engine.md","06-post-training/openrlhf/references/multi-node-training.md","06-post-training/miles/references/api-reference.md","06-post-training/miles/references/troubleshooting.md","08-distributed-training/ray-train/references/multi-node.md","08-distributed-training/pytorch-lightning/references/callbacks.md","08-distributed-training/pytorch-lightning/references/distributed.md","08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md","08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md","08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md","08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md","08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md","08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md","08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md","08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md","08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md","08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md","08-distributed-training/megatron-core/references/benchmarks.md","08-distributed-training/megatron-core/references/parallelism-guide.md","08-distributed-training/megatron-core/references/production-examples.md","08-distributed-training/megatron-core/references/training-recipes.md","08-distributed-training/deepspeed/references/08.md","08-distributed-training/deepspeed/references/09.md","08-distributed-training/deepspeed/references/2020.md","08-distributed-training/deepspeed/references/2023.md","08-distributed-training/deepspeed/references/assets.md","08-distributed-training/deepspeed/references/index.md","08-distributed-training/deepspeed/references/mii.md","08-distributed-training/deepspeed/references/other.md","08-distributed-training/deepspeed/references/tutorials.md","08-distributed-training/accelerate/references/custom-plugins.md","08-distributed-training/accelerate/references/megatron-integration.md","08-distributed-training/accelerate/references/performance.md","09-infrastructure/lambda-labs/references/advanced-usage.md","09-infrastructure/lambda-labs/references/troubleshooting.md","09-infrastructure/modal/references/advanced-usage.md","09-infrastructure/modal/references/troubleshooting.md","09-infrastructure/skypilot/references/advanced-usage.md","09-infrastructure/skypilot/references/troubleshooting.md","10-optimization/awq/references/advanced-usage.md","10-optimization/awq/references/troubleshooting.md","10-optimization/bitsandbytes/references/memory-optimization.md","10-optimization/bitsandbytes/references/qlora-training.md","10-optimization/bitsandbytes/references/quantization-formats.md","10-optimization/flash-attention/references/benchmarks.md","10-optimization/flash-attention/references/transformers-integration.md","10-optimization/gguf/references/advanced-usage.md","10-optimization/gguf/references/troubleshooting.md","10-optimization/gptq/references/calibration.md","10-optimization/gptq/references/integration.md","10-optimization/gptq/references/troubleshooting.md","10-optimization/hqq/references/advanced-usage.md","10-optimization/hqq/references/troubleshooting.md","10-optimization/ml-training-recipes/references/architecture.md","10-optimization/ml-training-recipes/references/biomedical.md","10-optimization/ml-training-recipes/references/domain-specific.md","10-optimization/ml-training-recipes/references/experiment-loop.md","10-optimization/ml-training-recipes/references/optimizers.md","10-optimization/ml-training-recipes/references/scaling-and-selection.md","11-evaluation/bigcode-evaluation-harness/references/benchmarks.md","11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md","11-evaluation/bigcode-evaluation-harness/references/issues.md","11-evaluation/lm-evaluation-harness/references/api-evaluation.md","11-evaluation/lm-evaluation-harness/references/benchmark-guide.md","11-evaluation/lm-evaluation-harness/references/custom-tasks.md","11-evaluation/lm-evaluation-harness/references/distributed-eval.md","11-evaluation/nemo-evaluator/references/adapter-system.md","11-evaluation/nemo-evaluator/references/configuration.md","11-evaluation/nemo-evaluator/references/custom-benchmarks.md","11-evaluation/nemo-evaluator/references/execution-backends.md","12-inference-serving/llama-cpp/references/optimization.md","12-inference-serving/llama-cpp/references/quantization.md","12-inference-serving/llama-cpp/references/server.md","12-inference-serving/sglang/references/deployment.md","12-inference-serving/sglang/references/radix-attention.md","12-inference-serving/sglang/references/structured-generation.md","12-inference-serving/tensorrt-llm/references/multi-gpu.md","12-inference-serving/tensorrt-llm/references/optimization.md","12-inference-serving/tensorrt-llm/references/serving.md","12-inference-serving/vllm/references/optimization.md","12-inference-serving/vllm/references/quantization.md","12-inference-serving/vllm/references/server-deployment.md","12-inference-serving/vllm/references/troubleshooting.md","13-mlops/mlflow/references/deployment.md","13-mlops/mlflow/references/model-registry.md","13-mlops/mlflow/references/tracking.md","13-mlops/swanlab/references/integrations.md","13-mlops/swanlab/references/visualization.md","13-mlops/tensorboard/references/integrations.md","13-mlops/tensorboard/references/profiling.md","13-mlops/tensorboard/references/visualization.md","13-mlops/weights-and-biases/references/artifacts.md","13-mlops/weights-and-biases/references/integrations.md","13-mlops/weights-and-biases/references/sweeps.md","14-agents/a-evolve/references/api.md","14-agents/a-evolve/references/architecture.md","14-agents/a-evolve/references/design-patterns.md","14-agents/a-evolve/references/examples.md","14-agents/a-evolve/references/issues.md","14-agents/a-evolve/references/README.md","14-agents/a-evolve/references/releases.md","14-agents/a-evolve/references/tutorials.md","14-agents/autogpt/references/advanced-usage.md","14-agents/autogpt/references/troubleshooting.md","14-agents/crewai/references/flows.md","14-agents/crewai/references/tools.md","14-agents/crewai/references/troubleshooting.md","14-agents/langchain/references/agents.md","14-agents/langchain/references/integration.md","14-agents/langchain/references/rag.md","14-agents/llamaindex/references/agents.md","14-agents/llamaindex/references/data_connectors.md","14-agents/llamaindex/references/query_engines.md","15-rag/chroma/references/integration.md","15-rag/faiss/references/index_types.md","15-rag/pinecone/references/deployment.md","15-rag/qdrant/references/advanced-usage.md","15-rag/qdrant/references/troubleshooting.md","15-rag/sentence-transformers/references/models.md","16-prompt-engineering/dspy/references/examples.md","16-prompt-engineering/dspy/references/modules.md","16-prompt-engineering/dspy/references/optimizers.md","16-prompt-engineering/guidance/references/backends.md","16-prompt-engineering/guidance/references/constraints.md","16-prompt-engineering/guidance/references/examples.md","16-prompt-engineering/instructor/references/examples.md","16-prompt-engineering/instructor/references/providers.md","16-prompt-engineering/instructor/references/validation.md","16-prompt-engineering/outlines/references/backends.md","16-prompt-engineering/outlines/references/examples.md","16-prompt-engineering/outlines/references/json_generation.md","17-observability/langsmith/references/advanced-usage.md","17-observability/langsmith/references/troubleshooting.md","17-observability/phoenix/references/advanced-usage.md","17-observability/phoenix/references/troubleshooting.md","18-multimodal/audiocraft/references/advanced-usage.md","18-multimodal/audiocraft/references/troubleshooting.md","18-multimodal/blip-2/references/advanced-usage.md","18-multimodal/blip-2/references/troubleshooting.md","18-multimodal/clip/references/applications.md","18-multimodal/cosmos-policy/references/libero-commands.md","18-multimodal/cosmos-policy/references/robocasa-commands.md","18-multimodal/llava/references/training.md","18-multimodal/openpi/references/checkpoints-and-env-map.md","18-multimodal/openpi/references/config-recipes.md","18-multimodal/openpi/references/pytorch-gotchas.md","18-multimodal/openpi/references/remote-client-pattern.md","18-multimodal/openpi/references/training-debugging.md","18-multimodal/openvla-oft/references/aloha-workflow.md","18-multimodal/openvla-oft/references/config-troubleshooting.md","18-multimodal/openvla-oft/references/libero-workflow.md","18-multimodal/openvla-oft/references/paper-and-checkpoints.md","18-multimodal/segment-anything/references/advanced-usage.md","18-multimodal/segment-anything/references/troubleshooting.md","18-multimodal/stable-diffusion/references/advanced-usage.md","18-multimodal/stable-diffusion/references/troubleshooting.md","18-multimodal/whisper/references/languages.md","19-emerging-techniques/knowledge-distillation/references/minillm.md","19-emerging-techniques/long-context/references/extension_methods.md","19-emerging-techniques/long-context/references/fine_tuning.md","19-emerging-techniques/long-context/references/rope.md","19-emerging-techniques/model-merging/references/coefficient-tuning.md","19-emerging-techniques/model-merging/references/evaluation.md","19-emerging-techniques/model-merging/references/examples.md","19-emerging-techniques/model-merging/references/methods.md","19-emerging-techniques/model-pruning/references/wanda.md","19-emerging-techniques/moe-training/references/architectures.md","19-emerging-techniques/moe-training/references/inference.md","19-emerging-techniques/moe-training/references/training.md","19-emerging-techniques/speculative-decoding/references/lookahead.md","19-emerging-techniques/speculative-decoding/references/medusa.md","20-ml-paper-writing/systems-paper-writing/references/checklist.md","20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md","20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md","20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md","20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md","20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md","20-ml-paper-writing/ml-paper-writing/references/checklists.md","20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md","20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md","20-ml-paper-writing/ml-paper-writing/references/sources.md","20-ml-paper-writing/ml-paper-writing/references/writing-guide.md","20-ml-paper-writing/academic-plotting/references/data-visualization.md","20-ml-paper-writing/academic-plotting/references/diagram-generation.md","20-ml-paper-writing/academic-plotting/references/style-guide.md","22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md","22-agent-native-research-artifact/research-manager/references/event-taxonomy.md","22-agent-native-research-artifact/research-manager/references/provenance-tags.md","22-agent-native-research-artifact/research-manager/references/session-protocol.md","22-agent-native-research-artifact/compiler/references/ara-schema.md","22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md","22-agent-native-research-artifact/compiler/references/validation-checklist.md"],"skills":{"01-model-architecture/mamba/SKILL.md":"---\nname: mamba-architecture\ndescription: State-space model with O(n) complexity vs Transformers' O(n²). 5× faster inference, million-token sequences, no KV cache. Selective SSM with hardware-aware design. Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head). Models 130M-2.8B on HuggingFace.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Mamba, State Space Models, SSM, Linear Complexity, Long Context, Efficient Inference, Hardware-Aware, Alternative To Transformers]\ndependencies: [mamba-ssm, torch, transformers, causal-conv1d]\n---\n\n# Mamba - Selective State Space Models\n\n## Quick start\n\nMamba is a state-space model architecture achieving O(n) linear complexity for sequence modeling.\n\n**Installation**:\n```bash\n# Install causal-conv1d (optional, for efficiency)\npip install causal-conv1d>=1.4.0\n\n# Install Mamba\npip install mamba-ssm\n# Or both together\npip install mamba-ssm[causal-conv1d]\n```\n\n**Prerequisites**: Linux, NVIDIA GPU, PyTorch 1.12+, CUDA 11.6+\n\n**Basic usage** (Mamba block):\n```python\nimport torch\nfrom mamba_ssm import Mamba\n\nbatch, length, dim = 2, 64, 16\nx = torch.randn(batch, length, dim).to(\"cuda\")\n\nmodel = Mamba(\n d_model=dim, # Model dimension\n d_state=16, # SSM state dimension\n d_conv=4, # Conv1d kernel size\n expand=2 # Expansion factor\n).to(\"cuda\")\n\ny = model(x) # O(n) complexity!\nassert y.shape == x.shape\n```\n\n## Common workflows\n\n### Workflow 1: Language model with Mamba-2\n\n**Complete LM with generation**:\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nfrom mamba_ssm.models.config_mamba import MambaConfig\nimport torch\n\n# Configure Mamba-2 LM\nconfig = MambaConfig(\n d_model=1024, # Hidden dimension\n n_layer=24, # Number of layers\n vocab_size=50277, # Vocabulary size\n ssm_cfg=dict(\n layer=\"Mamba2\", # Use Mamba-2\n d_state=128, # Larger state for Mamba-2\n headdim=64, # Head dimension\n ngroups=1 # Number of groups\n )\n)\n\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\n\n# Generate text\ninput_ids = torch.randint(0, 1000, (1, 20), device=\"cuda\", dtype=torch.long)\noutput = model.generate(\n input_ids=input_ids,\n max_length=100,\n temperature=0.7,\n top_p=0.9\n)\n```\n\n### Workflow 2: Use pretrained Mamba models\n\n**Load from HuggingFace**:\n```python\nfrom transformers import AutoTokenizer\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\n\n# Load pretrained model\nmodel_name = \"state-spaces/mamba-2.8b\"\ntokenizer = AutoTokenizer.from_pretrained(\"EleutherAI/gpt-neox-20b\") # Use compatible tokenizer\nmodel = MambaLMHeadModel.from_pretrained(model_name, device=\"cuda\", dtype=torch.float16)\n\n# Generate\nprompt = \"The future of AI is\"\ninput_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(\"cuda\")\noutput_ids = model.generate(\n input_ids=input_ids,\n max_length=200,\n temperature=0.7,\n top_p=0.9,\n repetition_penalty=1.2\n)\ngenerated_text = tokenizer.decode(output_ids[0])\nprint(generated_text)\n```\n\n**Available models**:\n- `state-spaces/mamba-130m`\n- `state-spaces/mamba-370m`\n- `state-spaces/mamba-790m`\n- `state-spaces/mamba-1.4b`\n- `state-spaces/mamba-2.8b`\n\n### Workflow 3: Mamba-1 vs Mamba-2\n\n**Mamba-1** (smaller state):\n```python\nfrom mamba_ssm import Mamba\n\nmodel = Mamba(\n d_model=256,\n d_state=16, # Smaller state dimension\n d_conv=4,\n expand=2\n).to(\"cuda\")\n```\n\n**Mamba-2** (multi-head, larger state):\n```python\nfrom mamba_ssm import Mamba2\n\nmodel = Mamba2(\n d_model=256,\n d_state=128, # Larger state dimension\n d_conv=4,\n expand=2,\n headdim=64, # Head dimension for multi-head\n ngroups=1 # Parallel groups\n).to(\"cuda\")\n```\n\n**Key differences**:\n- **State size**: Mamba-1 (d_state=16) vs Mamba-2 (d_state=128)\n- **Architecture**: Mamba-2 has multi-head structure\n- **Normalization**: Mamba-2 uses RMSNorm\n- **Distributed**: Mamba-2 supports tensor parallelism\n\n### Workflow 4: Benchmark vs Transformers\n\n**Generation speed comparison**:\n```bash\n# Benchmark Mamba\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"state-spaces/mamba-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n\n# Benchmark Transformer\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"EleutherAI/pythia-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n```\n\n**Expected results**:\n- **Mamba**: 5× faster inference\n- **Memory**: No KV cache needed\n- **Scaling**: Linear with sequence length\n\n## When to use vs alternatives\n\n**Use Mamba when**:\n- Need long sequences (100K+ tokens)\n- Want faster inference than Transformers\n- Memory-constrained (no KV cache)\n- Building streaming applications\n- Linear scaling important\n\n**Advantages**:\n- **O(n) complexity**: Linear vs quadratic\n- **5× faster inference**: No attention overhead\n- **No KV cache**: Lower memory usage\n- **Million-token sequences**: Hardware-efficient\n- **Streaming**: Constant memory per token\n\n**Use alternatives instead**:\n- **Transformers**: Need best-in-class performance, have compute\n- **RWKV**: Want RNN+Transformer hybrid\n- **RetNet**: Need retention-based architecture\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or use gradient checkpointing:\n```python\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\nmodel.gradient_checkpointing_enable() # Enable checkpointing\n```\n\n**Issue: Slow installation**\n\nInstall binary wheels (not source):\n```bash\npip install mamba-ssm --no-build-isolation\n```\n\n**Issue: Missing causal-conv1d**\n\nInstall separately:\n```bash\npip install causal-conv1d>=1.4.0\n```\n\n**Issue: Model not loading from HuggingFace**\n\nUse `MambaLMHeadModel.from_pretrained` (not `AutoModel`):\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nmodel = MambaLMHeadModel.from_pretrained(\"state-spaces/mamba-2.8b\")\n```\n\n## Advanced topics\n\n**Selective SSM**: See [references/selective-ssm.md](references/selective-ssm.md) for mathematical formulation, state-space equations, and how selectivity enables O(n) complexity.\n\n**Mamba-2 architecture**: See [references/mamba2-details.md](references/mamba2-details.md) for multi-head structure, tensor parallelism, and distributed training setup.\n\n**Performance optimization**: See [references/performance.md](references/performance.md) for hardware-aware design, CUDA kernels, and memory efficiency techniques.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA with CUDA 11.6+\n- **VRAM**:\n - 130M model: 2GB\n - 370M model: 4GB\n - 790M model: 8GB\n - 1.4B model: 14GB\n - 2.8B model: 28GB (FP16)\n- **Inference**: 5× faster than Transformers\n- **Memory**: No KV cache (lower than Transformers)\n\n**Performance** (vs Transformers):\n- **Speed**: 5× faster inference\n- **Memory**: 50% less (no KV cache)\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (Mamba-1): https://arxiv.org/abs/2312.00752 (Dec 2023)\n- Paper (Mamba-2): https://arxiv.org/abs/2405.21060 (May 2024)\n- GitHub: https://github.com/state-spaces/mamba ⭐ 13,000+\n- Models: https://huggingface.co/state-spaces\n- Docs: Repository README and wiki\n\n\n","01-model-architecture/rwkv/SKILL.md":"---\nname: rwkv-architecture\ndescription: RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [RWKV, Model Architecture, RNN, Transformer Hybrid, Linear Complexity, Infinite Context, Efficient Inference, Linux Foundation, Alternative Architecture]\ndependencies: [rwkv, torch, transformers]\n---\n\n# RWKV - Receptance Weighted Key Value\n\n## Quick start\n\nRWKV (RwaKuv) combines Transformer parallelization (training) with RNN efficiency (inference).\n\n**Installation**:\n```bash\n# Install PyTorch\npip install torch --upgrade --extra-index-url https://download.pytorch.org/whl/cu121\n\n# Install dependencies\npip install pytorch-lightning==1.9.5 deepspeed wandb ninja --upgrade\n\n# Install RWKV\npip install rwkv\n```\n\n**Basic usage** (GPT mode + RNN mode):\n```python\nimport os\nfrom rwkv.model import RWKV\n\nos.environ[\"RWKV_JIT_ON\"] = '1'\nos.environ[\"RWKV_CUDA_ON\"] = '1' # Use CUDA kernel for speed\n\n# Load model\nmodel = RWKV(\n model='/path/to/RWKV-4-Pile-1B5-20220903-8040',\n strategy='cuda fp16'\n)\n\n# GPT mode (parallel processing)\nout, state = model.forward([187, 510, 1563, 310, 247], None)\nprint(out.detach().cpu().numpy()) # Logits\n\n# RNN mode (sequential processing, same result)\nout, state = model.forward([187, 510], None) # First 2 tokens\nout, state = model.forward([1563], state) # Next token\nout, state = model.forward([310, 247], state) # Last tokens\nprint(out.detach().cpu().numpy()) # Same logits as above!\n```\n\n## Common workflows\n\n### Workflow 1: Text generation (streaming)\n\n**Efficient token-by-token generation**:\n```python\nfrom rwkv.model import RWKV\nfrom rwkv.utils import PIPELINE\n\nmodel = RWKV(model='RWKV-4-Pile-14B-20230313-ctx8192-test1050', strategy='cuda fp16')\npipeline = PIPELINE(model, \"20B_tokenizer.json\")\n\n# Initial prompt\nprompt = \"The future of AI is\"\nstate = None\n\n# Generate token by token\nfor token in prompt:\n out, state = pipeline.model.forward(pipeline.encode(token), state)\n\n# Continue generation\nfor _ in range(100):\n out, state = pipeline.model.forward(None, state)\n token = pipeline.sample_logits(out)\n print(pipeline.decode(token), end='', flush=True)\n```\n\n**Key advantage**: Constant memory per token (no growing KV cache)\n\n### Workflow 2: Long context processing (infinite context)\n\n**Process million-token sequences**:\n```python\nmodel = RWKV(model='RWKV-4-Pile-14B', strategy='cuda fp16')\n\n# Process very long document\nstate = None\nlong_document = load_document() # e.g., 1M tokens\n\n# Stream through entire document\nfor chunk in chunks(long_document, chunk_size=1024):\n out, state = model.forward(chunk, state)\n\n# State now contains information from entire 1M token document\n# Memory usage: O(1) (constant, not O(n)!)\n```\n\n### Workflow 3: Fine-tuning RWKV\n\n**Standard fine-tuning workflow**:\n```python\n# Training script\nimport pytorch_lightning as pl\nfrom rwkv.model import RWKV\nfrom rwkv.trainer import RWKVTrainer\n\n# Configure model\nconfig = {\n 'n_layer': 24,\n 'n_embd': 1024,\n 'vocab_size': 50277,\n 'ctx_len': 1024\n}\n\n# Setup trainer\ntrainer = pl.Trainer(\n accelerator='gpu',\n devices=8,\n precision='bf16',\n strategy='deepspeed_stage_2',\n max_epochs=1\n)\n\n# Train\nmodel = RWKV(config)\ntrainer.fit(model, train_dataloader)\n```\n\n### Workflow 4: RWKV vs Transformer comparison\n\n**Memory comparison** (1M token sequence):\n```python\n# Transformer (GPT)\n# Memory: O(n²) for attention\n# KV cache: 1M × hidden_dim × n_layers × 2 (keys + values)\n# Example: 1M × 4096 × 24 × 2 = ~400GB (impractical!)\n\n# RWKV\n# Memory: O(1) per token\n# State: hidden_dim × n_layers = 4096 × 24 = ~400KB\n# 1,000,000× more efficient!\n```\n\n**Speed comparison** (inference):\n```python\n# Transformer: O(n) per token (quadratic overall)\n# First token: 1 computation\n# Second token: 2 computations\n# ...\n# 1000th token: 1000 computations\n\n# RWKV: O(1) per token (linear overall)\n# Every token: 1 computation\n# 1000th token: 1 computation (same as first!)\n```\n\n## When to use vs alternatives\n\n**Use RWKV when**:\n- Need very long context (100K+ tokens)\n- Want constant memory usage\n- Building streaming applications\n- Need RNN efficiency with Transformer performance\n- Memory-constrained deployment\n\n**Key advantages**:\n- **Linear time**: O(n) vs O(n²) for Transformers\n- **No KV cache**: Constant memory per token\n- **Infinite context**: No fixed window limit\n- **Parallelizable training**: Like GPT\n- **Sequential inference**: Like RNN\n\n**Use alternatives instead**:\n- **Transformers**: Need absolute best performance, have compute\n- **Mamba**: Want state-space models\n- **RetNet**: Need retention mechanism\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: Out of memory during training**\n\nUse gradient checkpointing and DeepSpeed:\n```python\ntrainer = pl.Trainer(\n strategy='deepspeed_stage_3', # Full ZeRO-3\n precision='bf16'\n)\n```\n\n**Issue: Slow inference**\n\nEnable CUDA kernel:\n```python\nos.environ[\"RWKV_CUDA_ON\"] = '1'\n```\n\n**Issue: Model not loading**\n\nCheck model path and strategy:\n```python\nmodel = RWKV(\n model='/absolute/path/to/model.pth',\n strategy='cuda fp16' # Or 'cpu fp32' for CPU\n)\n```\n\n**Issue: State management in RNN mode**\n\nAlways pass state between forward calls:\n```python\n# WRONG: State lost\nout1, _ = model.forward(tokens1, None)\nout2, _ = model.forward(tokens2, None) # No context from tokens1!\n\n# CORRECT: State preserved\nout1, state = model.forward(tokens1, None)\nout2, state = model.forward(tokens2, state) # Has context from tokens1\n```\n\n## Advanced topics\n\n**Time-mixing and channel-mixing**: See [references/architecture-details.md](references/architecture-details.md) for WKV operation, time-decay mechanism, and receptance gates.\n\n**State management**: See [references/state-management.md](references/state-management.md) for att_x_prev, att_kv, ffn_x_prev states, and numerical stability considerations.\n\n**RWKV-7 improvements**: See [references/rwkv7.md](references/rwkv7.md) for latest architectural improvements (March 2025) and multimodal capabilities.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.6+) or CPU\n- **VRAM** (FP16):\n - 169M model: 1GB\n - 430M model: 2GB\n - 1.5B model: 4GB\n - 3B model: 8GB\n - 7B model: 16GB\n - 14B model: 32GB\n- **Inference**: O(1) memory per token\n- **Training**: Parallelizable like GPT\n\n**Performance** (vs Transformers):\n- **Speed**: Similar training, faster inference\n- **Memory**: 1000× less for long sequences\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (RWKV): https://arxiv.org/abs/2305.13048 (May 2023)\n- Paper (RWKV-7): https://arxiv.org/abs/2503.14456 (March 2025)\n- GitHub: https://github.com/BlinkDL/RWKV-LM ⭐ 12,000+\n- Docs: https://wiki.rwkv.com/\n- Models: https://huggingface.co/BlinkDL\n- Linux Foundation AI: Official project\n- Production: Microsoft Windows, Office integration, NeMo support\n\n\n","01-model-architecture/nanogpt/SKILL.md":"---\nname: nanogpt\ndescription: Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, NanoGPT, GPT-2, Educational, Andrej Karpathy, Transformer, Minimalist, From Scratch, Training]\ndependencies: [torch, transformers, datasets, tiktoken, wandb]\n---\n\n# nanoGPT - Minimalist GPT Training\n\n## Quick start\n\nnanoGPT is a simplified GPT implementation designed for learning and experimentation.\n\n**Installation**:\n```bash\npip install torch numpy transformers datasets tiktoken wandb tqdm\n```\n\n**Train on Shakespeare** (CPU-friendly):\n```bash\n# Prepare data\npython data/shakespeare_char/prepare.py\n\n# Train (5 minutes on CPU)\npython train.py config/train_shakespeare_char.py\n\n# Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Output**:\n```\nROMEO:\nWhat say'st thou? Shall I speak, and be a man?\n\nJULIET:\nI am afeard, and yet I'll speak; for thou art\nOne that hath been a man, and yet I know not\nWhat thou art.\n```\n\n## Common workflows\n\n### Workflow 1: Character-level Shakespeare\n\n**Complete training pipeline**:\n```bash\n# Step 1: Prepare data (creates train.bin, val.bin)\npython data/shakespeare_char/prepare.py\n\n# Step 2: Train small model\npython train.py config/train_shakespeare_char.py\n\n# Step 3: Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Config** (`config/train_shakespeare_char.py`):\n```python\n# Model config\nn_layer = 6 # 6 transformer layers\nn_head = 6 # 6 attention heads\nn_embd = 384 # 384-dim embeddings\nblock_size = 256 # 256 char context\n\n# Training config\nbatch_size = 64\nlearning_rate = 1e-3\nmax_iters = 5000\neval_interval = 500\n\n# Hardware\ndevice = 'cpu' # Or 'cuda'\ncompile = False # Set True for PyTorch 2.0\n```\n\n**Training time**: ~5 minutes (CPU), ~1 minute (GPU)\n\n### Workflow 2: Reproduce GPT-2 (124M)\n\n**Multi-GPU training on OpenWebText**:\n```bash\n# Step 1: Prepare OpenWebText (takes ~1 hour)\npython data/openwebtext/prepare.py\n\n# Step 2: Train GPT-2 124M with DDP (8 GPUs)\ntorchrun --standalone --nproc_per_node=8 \\\n train.py config/train_gpt2.py\n\n# Step 3: Sample from trained model\npython sample.py --out_dir=out\n```\n\n**Config** (`config/train_gpt2.py`):\n```python\n# GPT-2 (124M) architecture\nn_layer = 12\nn_head = 12\nn_embd = 768\nblock_size = 1024\ndropout = 0.0\n\n# Training\nbatch_size = 12\ngradient_accumulation_steps = 5 * 8 # Total batch ~0.5M tokens\nlearning_rate = 6e-4\nmax_iters = 600000\nlr_decay_iters = 600000\n\n# System\ncompile = True # PyTorch 2.0\n```\n\n**Training time**: ~4 days (8× A100)\n\n### Workflow 3: Fine-tune pretrained GPT-2\n\n**Start from OpenAI checkpoint**:\n```python\n# In train.py or config\ninit_from = 'gpt2' # Options: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n\n# Model loads OpenAI weights automatically\npython train.py config/finetune_shakespeare.py\n```\n\n**Example config** (`config/finetune_shakespeare.py`):\n```python\n# Start from GPT-2\ninit_from = 'gpt2'\n\n# Dataset\ndataset = 'shakespeare_char'\nbatch_size = 1\nblock_size = 1024\n\n# Fine-tuning\nlearning_rate = 3e-5 # Lower LR for fine-tuning\nmax_iters = 2000\nwarmup_iters = 100\n\n# Regularization\nweight_decay = 1e-1\n```\n\n### Workflow 4: Custom dataset\n\n**Train on your own text**:\n```python\n# data/custom/prepare.py\nimport numpy as np\n\n# Load your data\nwith open('my_data.txt', 'r') as f:\n text = f.read()\n\n# Create character mappings\nchars = sorted(list(set(text)))\nstoi = {ch: i for i, ch in enumerate(chars)}\nitos = {i: ch for i, ch in enumerate(chars)}\n\n# Tokenize\ndata = np.array([stoi[ch] for ch in text], dtype=np.uint16)\n\n# Split train/val\nn = len(data)\ntrain_data = data[:int(n*0.9)]\nval_data = data[int(n*0.9):]\n\n# Save\ntrain_data.tofile('data/custom/train.bin')\nval_data.tofile('data/custom/val.bin')\n```\n\n**Train**:\n```bash\npython data/custom/prepare.py\npython train.py --dataset=custom\n```\n\n## When to use vs alternatives\n\n**Use nanoGPT when**:\n- Learning how GPT works\n- Experimenting with transformer variants\n- Teaching/education purposes\n- Quick prototyping\n- Limited compute (can run on CPU)\n\n**Simplicity advantages**:\n- **~300 lines**: Entire model in `model.py`\n- **~300 lines**: Training loop in `train.py`\n- **Hackable**: Easy to modify\n- **No abstractions**: Pure PyTorch\n\n**Use alternatives instead**:\n- **HuggingFace Transformers**: Production use, many models\n- **Megatron-LM**: Large-scale distributed training\n- **LitGPT**: More architectures, production-ready\n- **PyTorch Lightning**: Need high-level framework\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or context length:\n```python\nbatch_size = 1 # Reduce from 12\nblock_size = 512 # Reduce from 1024\ngradient_accumulation_steps = 40 # Increase to maintain effective batch\n```\n\n**Issue: Training too slow**\n\nEnable compilation (PyTorch 2.0+):\n```python\ncompile = True # 2× speedup\n```\n\nUse mixed precision:\n```python\ndtype = 'bfloat16' # Or 'float16'\n```\n\n**Issue: Poor generation quality**\n\nTrain longer:\n```python\nmax_iters = 10000 # Increase from 5000\n```\n\nLower temperature:\n```python\n# In sample.py\ntemperature = 0.7 # Lower from 1.0\ntop_k = 200 # Add top-k sampling\n```\n\n**Issue: Can't load GPT-2 weights**\n\nInstall transformers:\n```bash\npip install transformers\n```\n\nCheck model name:\n```python\ninit_from = 'gpt2' # Valid: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n```\n\n## Advanced topics\n\n**Model architecture**: See [references/architecture.md](references/architecture.md) for GPT block structure, multi-head attention, and MLP layers explained simply.\n\n**Training loop**: See [references/training.md](references/training.md) for learning rate schedule, gradient accumulation, and distributed data parallel setup.\n\n**Data preparation**: See [references/data.md](references/data.md) for tokenization strategies (character-level vs BPE) and binary format details.\n\n## Hardware requirements\n\n- **Shakespeare (char-level)**:\n - CPU: 5 minutes\n - GPU (T4): 1 minute\n - VRAM: <1GB\n\n- **GPT-2 (124M)**:\n - 1× A100: ~1 week\n - 8× A100: ~4 days\n - VRAM: ~16GB per GPU\n\n- **GPT-2 Medium (350M)**:\n - 8× A100: ~2 weeks\n - VRAM: ~40GB per GPU\n\n**Performance**:\n- With `compile=True`: 2× speedup\n- With `dtype=bfloat16`: 50% memory reduction\n\n## Resources\n\n- GitHub: https://github.com/karpathy/nanoGPT ⭐ 48,000+\n- Video: \"Let's build GPT\" by Andrej Karpathy\n- Paper: \"Attention is All You Need\" (Vaswani et al.)\n- OpenWebText: https://huggingface.co/datasets/Skylion007/openwebtext\n- Educational: Best for understanding transformers from scratch\n\n\n","01-model-architecture/torchtitan/SKILL.md":"---\nname: distributed-llm-pretraining-torchtitan\ndescription: Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Distributed Training, TorchTitan, FSDP2, Tensor Parallel, Pipeline Parallel, Context Parallel, Float8, Llama, Pretraining]\ndependencies: [torch>=2.6.0, torchtitan>=0.2.0, torchao>=0.5.0]\n---\n\n# TorchTitan - PyTorch Native Distributed LLM Pretraining\n\n## Quick start\n\nTorchTitan is PyTorch's official platform for large-scale LLM pretraining with composable 4D parallelism (FSDP2, TP, PP, CP), achieving 65%+ speedups over baselines on H100 GPUs.\n\n**Installation**:\n```bash\n# From PyPI (stable)\npip install torchtitan\n\n# From source (latest features, requires PyTorch nightly)\ngit clone https://github.com/pytorch/torchtitan\ncd torchtitan\npip install -r requirements.txt\n```\n\n**Download tokenizer**:\n```bash\n# Get HF token from https://huggingface.co/settings/tokens\npython scripts/download_hf_assets.py --repo_id meta-llama/Llama-3.1-8B --assets tokenizer --hf_token=...\n```\n\n**Start training on 8 GPUs**:\n```bash\nCONFIG_FILE=\"./torchtitan/models/llama3/train_configs/llama3_8b.toml\" ./run_train.sh\n```\n\n## Common workflows\n\n### Workflow 1: Pretrain Llama 3.1 8B on single node\n\nCopy this checklist:\n\n```\nSingle Node Pretraining:\n- [ ] Step 1: Download tokenizer\n- [ ] Step 2: Configure training\n- [ ] Step 3: Launch training\n- [ ] Step 4: Monitor and checkpoint\n```\n\n**Step 1: Download tokenizer**\n\n```bash\npython scripts/download_hf_assets.py \\\n --repo_id meta-llama/Llama-3.1-8B \\\n --assets tokenizer \\\n --hf_token=YOUR_HF_TOKEN\n```\n\n**Step 2: Configure training**\n\nEdit or create a TOML config file:\n\n```toml\n# llama3_8b_custom.toml\n[job]\ndump_folder = \"./outputs\"\ndescription = \"Llama 3.1 8B training\"\n\n[model]\nname = \"llama3\"\nflavor = \"8B\"\nhf_assets_path = \"./assets/hf/Llama-3.1-8B\"\n\n[optimizer]\nname = \"AdamW\"\nlr = 3e-4\n\n[lr_scheduler]\nwarmup_steps = 200\n\n[training]\nlocal_batch_size = 2\nseq_len = 8192\nmax_norm = 1.0\nsteps = 1000\ndataset = \"c4\"\n\n[parallelism]\ndata_parallel_shard_degree = -1 # Use all GPUs for FSDP\n\n[activation_checkpoint]\nmode = \"selective\"\nselective_ac_option = \"op\"\n\n[checkpoint]\nenable = true\nfolder = \"checkpoint\"\ninterval = 500\n```\n\n**Step 3: Launch training**\n\n```bash\n# 8 GPUs on single node\nCONFIG_FILE=\"./llama3_8b_custom.toml\" ./run_train.sh\n\n# Or explicitly with torchrun\ntorchrun --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_8b_custom.toml\n```\n\n**Step 4: Monitor and checkpoint**\n\nTensorBoard logs are saved to `./outputs/tb/`:\n```bash\ntensorboard --logdir ./outputs/tb\n```\n\n### Workflow 2: Multi-node training with SLURM\n\n```\nMulti-Node Training:\n- [ ] Step 1: Configure parallelism for scale\n- [ ] Step 2: Set up SLURM script\n- [ ] Step 3: Submit job\n- [ ] Step 4: Resume from checkpoint\n```\n\n**Step 1: Configure parallelism for scale**\n\nFor 70B model on 256 GPUs (32 nodes):\n```toml\n[parallelism]\ndata_parallel_shard_degree = 32 # FSDP across 32 ranks\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 1 # No PP for 70B\ncontext_parallel_degree = 1 # Increase for long sequences\n```\n\n**Step 2: Set up SLURM script**\n\n```bash\n#!/bin/bash\n#SBATCH --job-name=llama70b\n#SBATCH --nodes=32\n#SBATCH --ntasks-per-node=8\n#SBATCH --gpus-per-node=8\n\nsrun torchrun \\\n --nnodes=32 \\\n --nproc_per_node=8 \\\n --rdzv_backend=c10d \\\n --rdzv_endpoint=$MASTER_ADDR:$MASTER_PORT \\\n -m torchtitan.train \\\n --job.config_file ./llama3_70b.toml\n```\n\n**Step 3: Submit job**\n\n```bash\nsbatch multinode_trainer.slurm\n```\n\n**Step 4: Resume from checkpoint**\n\nTraining auto-resumes if checkpoint exists in configured folder.\n\n### Workflow 3: Enable Float8 training for H100s\n\nFloat8 provides 30-50% speedup on H100 GPUs.\n\n```\nFloat8 Training:\n- [ ] Step 1: Install torchao\n- [ ] Step 2: Configure Float8\n- [ ] Step 3: Launch with compile\n```\n\n**Step 1: Install torchao**\n\n```bash\nUSE_CPP=0 pip install git+https://github.com/pytorch/ao.git\n```\n\n**Step 2: Configure Float8**\n\nAdd to your TOML config:\n```toml\n[model]\nconverters = [\"quantize.linear.float8\"]\n\n[quantize.linear.float8]\nenable_fsdp_float8_all_gather = true\nprecompute_float8_dynamic_scale_for_fsdp = true\nfilter_fqns = [\"output\"] # Exclude output layer\n\n[compile]\nenable = true\ncomponents = [\"model\", \"loss\"]\n```\n\n**Step 3: Launch with compile**\n\n```bash\nCONFIG_FILE=\"./llama3_8b.toml\" ./run_train.sh \\\n --model.converters=\"quantize.linear.float8\" \\\n --quantize.linear.float8.enable_fsdp_float8_all_gather \\\n --compile.enable\n```\n\n### Workflow 4: 4D parallelism for 405B models\n\n```\n4D Parallelism (FSDP + TP + PP + CP):\n- [ ] Step 1: Create seed checkpoint\n- [ ] Step 2: Configure 4D parallelism\n- [ ] Step 3: Launch on 512 GPUs\n```\n\n**Step 1: Create seed checkpoint**\n\nRequired for consistent initialization across PP stages:\n```bash\nNGPU=1 CONFIG_FILE=./llama3_405b.toml ./run_train.sh \\\n --checkpoint.enable \\\n --checkpoint.create_seed_checkpoint \\\n --parallelism.data_parallel_shard_degree 1 \\\n --parallelism.tensor_parallel_degree 1 \\\n --parallelism.pipeline_parallel_degree 1\n```\n\n**Step 2: Configure 4D parallelism**\n\n```toml\n[parallelism]\ndata_parallel_shard_degree = 8 # FSDP\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 8 # PP across nodes\ncontext_parallel_degree = 1 # CP for long sequences\n\n[training]\nlocal_batch_size = 32\nseq_len = 8192\n```\n\n**Step 3: Launch on 512 GPUs**\n\n```bash\n# 64 nodes x 8 GPUs = 512 GPUs\nsrun torchrun --nnodes=64 --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_405b.toml\n```\n\n## When to use vs alternatives\n\n**Use TorchTitan when:**\n- Pretraining LLMs from scratch (8B to 405B+)\n- Need PyTorch-native solution without third-party dependencies\n- Require composable 4D parallelism (FSDP2, TP, PP, CP)\n- Training on H100s with Float8 support\n- Want interoperable checkpoints with torchtune/HuggingFace\n\n**Use alternatives instead:**\n- **Megatron-LM**: Maximum performance for NVIDIA-only deployments\n- **DeepSpeed**: Broader ZeRO optimization ecosystem, inference support\n- **Axolotl/TRL**: Fine-tuning rather than pretraining\n- **LitGPT**: Educational, smaller-scale training\n\n## Common issues\n\n**Issue: Out of memory on large models**\n\nEnable activation checkpointing and reduce batch size:\n```toml\n[activation_checkpoint]\nmode = \"full\" # Instead of \"selective\"\n\n[training]\nlocal_batch_size = 1\n```\n\nOr use gradient accumulation:\n```toml\n[training]\nlocal_batch_size = 1\nglobal_batch_size = 32 # Accumulates gradients\n```\n\n**Issue: TP causes high memory with async collectives**\n\nSet environment variable:\n```bash\nexport TORCH_NCCL_AVOID_RECORD_STREAMS=1\n```\n\n**Issue: Float8 training not faster**\n\nFloat8 only benefits large GEMMs. Filter small layers:\n```toml\n[quantize.linear.float8]\nfilter_fqns = [\"attention.wk\", \"attention.wv\", \"output\", \"auto_filter_small_kn\"]\n```\n\n**Issue: Checkpoint loading fails after parallelism change**\n\nUse DCP's resharding capability:\n```bash\n# Convert sharded checkpoint to single file\npython -m torch.distributed.checkpoint.format_utils \\\n dcp_to_torch checkpoint/step-1000 checkpoint.pt\n```\n\n**Issue: Pipeline parallelism initialization**\n\nCreate seed checkpoint first (see Workflow 4, Step 1).\n\n## Supported models\n\n| Model | Sizes | Status |\n|-------|-------|--------|\n| Llama 3.1 | 8B, 70B, 405B | Production |\n| Llama 4 | Various | Experimental |\n| DeepSeek V3 | 16B, 236B, 671B (MoE) | Experimental |\n| GPT-OSS | 20B, 120B (MoE) | Experimental |\n| Qwen 3 | Various | Experimental |\n| Flux | Diffusion | Experimental |\n\n## Performance benchmarks (H100)\n\n| Model | GPUs | Parallelism | TPS/GPU | Techniques |\n|-------|------|-------------|---------|------------|\n| Llama 8B | 8 | FSDP | 5,762 | Baseline |\n| Llama 8B | 8 | FSDP+compile+FP8 | 8,532 | +48% |\n| Llama 70B | 256 | FSDP+TP+AsyncTP | 876 | 2D parallel |\n| Llama 405B | 512 | FSDP+TP+PP | 128 | 3D parallel |\n\n## Advanced topics\n\n**FSDP2 configuration**: See [references/fsdp.md](references/fsdp.md) for detailed FSDP2 vs FSDP1 comparison and ZeRO equivalents.\n\n**Float8 training**: See [references/float8.md](references/float8.md) for tensorwise vs rowwise scaling recipes.\n\n**Checkpointing**: See [references/checkpoint.md](references/checkpoint.md) for HuggingFace conversion and async checkpointing.\n\n**Adding custom models**: See [references/custom-models.md](references/custom-models.md) for TrainSpec protocol.\n\n## Resources\n\n- GitHub: https://github.com/pytorch/torchtitan\n- Paper: https://arxiv.org/abs/2410.06511\n- ICLR 2025: https://iclr.cc/virtual/2025/poster/29620\n- PyTorch Forum: https://discuss.pytorch.org/c/distributed/torchtitan/44\n\n","01-model-architecture/litgpt/SKILL.md":"---\nname: implementing-llms-litgpt\ndescription: Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, LitGPT, Lightning AI, LLM Implementation, LoRA, QLoRA, Fine-Tuning, Llama, Gemma, Phi, Mistral, Educational]\ndependencies: [litgpt, torch, transformers]\n---\n\n# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft/phi-2\")\n\n# Generate text\nresult = llm.generate(\n \"What is the capital of France?\",\n max_new_tokens=50,\n temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama/Meta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft/phi-2\n\n# Download Gemma 2B\nlitgpt download google/gemma-2b\n```\n\nModels are saved to `checkpoints/` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n {\n \"instruction\": \"What is the capital of France?\",\n \"input\": \"\",\n \"output\": \"The capital of France is Paris.\"\n },\n {\n \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n \"input\": \"\",\n \"output\": \"Hola, ¿cómo estás?\"\n }\n]\n```\n\nSave as `data/my_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n meta-llama/Meta-Llama-3-8B \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --train.max_steps 1000 \\\n --train.learning_rate 2e-5 \\\n --train.micro_batch_size 1 \\\n --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --lora_alpha 32 \\\n --lora_dropout 0.05 \\\n --train.max_steps 1000 \\\n --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out/finetune/` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out/finetune/logs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out/finetune/lightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality/size tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\ # LoRA rank (8-64, higher=more capacity)\n --lora_alpha 32 \\ # LoRA scaling (typically 2×r)\n --lora_dropout 0.05 \\ # Prevent overfitting\n --lora_query true \\ # Apply LoRA to query projection\n --lora_key false \\ # Usually not needed\n --lora_value true \\ # Apply LoRA to value projection\n --lora_projection true \\ # Apply LoRA to output projection\n --lora_mlp false \\ # Usually not needed\n --lora_head false # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --train.epochs 3 \\\n --train.learning_rate 1e-4 \\\n --train.micro_batch_size 4 \\\n --train.global_batch_size 32 \\\n --out_dir out/phi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n out/phi2-lora/final \\\n --out_dir out/phi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out/phi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts/prepare_dataset.py \\\n --source_path data/my_corpus.txt \\\n --checkpoint_dir checkpoints/tokenizer \\\n --destination_path data/pretrain \\\n --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config/pythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n --config config/pythia-160m.yaml \\\n --data.data_dir data/pretrain \\\n --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir data/pretrain \\\n --devices 8 \\\n --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir /shared/data/pretrain \\\n --devices 8 \\\n --num_nodes 8 \\\n --train.global_batch_size 512 \\\n --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out/phi2-lora/final\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --dtype bfloat16 \\\n --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --quantize bnb.nf4-dq # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts/convert_lit_checkpoint.py \\\n --checkpoint_path out/phi2-lora/final \\\n --output_path models/phi2.gguf \\\n --model_name microsoft/phi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out/phi2-lora/final\")\n\n@app.post(\"/generate\")\ndef generate(prompt: str, max_tokens: int = 100):\n result = llm.generate(\n prompt,\n max_new_tokens=max_tokens,\n temperature=0.7\n )\n return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl/TRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n ... \\\n --train.gradient_accumulation_iters 4 # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30/40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32 # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama/Meta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints/\n# Should see: meta-llama/Meta-Llama-3-8B/\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8 # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\ # Disable this\n--lora_mlp false # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references/supported-models.md](references/supported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references/training-recipes.md](references/training-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references/distributed-training.md](references/distributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references/custom-models.md](references/custom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n - Inference (Phi-2): 6GB\n - LoRA fine-tuning (7B): 16GB\n - Full fine-tuning (7B): 40GB+\n - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https://github.com/Lightning-AI/litgpt\n- Docs: https://lightning.ai/docs/litgpt\n- Tutorials: https://lightning.ai/docs/litgpt/tutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n","02-tokenization/huggingface-tokenizers/SKILL.md":"---\nname: huggingface-tokenizers\ndescription: Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Tokenization, HuggingFace, BPE, WordPiece, Unigram, Fast Tokenization, Rust, Custom Tokenizer, Alignment Tracking, Production]\ndependencies: [tokenizers, transformers, datasets]\n---\n\n# HuggingFace Tokenizers - Fast Tokenization for NLP\n\nFast, production-ready tokenizers with Rust performance and Python ease-of-use.\n\n## When to use HuggingFace Tokenizers\n\n**Use HuggingFace Tokenizers when:**\n- Need extremely fast tokenization (<20s per GB of text)\n- Training custom tokenizers from scratch\n- Want alignment tracking (token → original text position)\n- Building production NLP pipelines\n- Need to tokenize large corpora efficiently\n\n**Performance**:\n- **Speed**: <20 seconds to tokenize 1GB on CPU\n- **Implementation**: Rust core with Python/Node.js bindings\n- **Efficiency**: 10-100× faster than pure Python implementations\n\n**Use alternatives instead**:\n- **SentencePiece**: Language-independent, used by T5/ALBERT\n- **tiktoken**: OpenAI's BPE tokenizer for GPT models\n- **transformers AutoTokenizer**: Loading pretrained only (uses this library internally)\n\n## Quick start\n\n### Installation\n\n```bash\n# Install tokenizers\npip install tokenizers\n\n# With transformers integration\npip install tokenizers transformers\n```\n\n### Load pretrained tokenizer\n\n```python\nfrom tokenizers import Tokenizer\n\n# Load from HuggingFace Hub\ntokenizer = Tokenizer.from_pretrained(\"bert-base-uncased\")\n\n# Encode text\noutput = tokenizer.encode(\"Hello, how are you?\")\nprint(output.tokens) # ['hello', ',', 'how', 'are', 'you', '?']\nprint(output.ids) # [7592, 1010, 2129, 2024, 2017, 1029]\n\n# Decode back\ntext = tokenizer.decode(output.ids)\nprint(text) # \"hello, how are you?\"\n```\n\n### Train custom BPE tokenizer\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import BPE\nfrom tokenizers.trainers import BpeTrainer\nfrom tokenizers.pre_tokenizers import Whitespace\n\n# Initialize tokenizer with BPE model\ntokenizer = Tokenizer(BPE(unk_token=\"[UNK]\"))\ntokenizer.pre_tokenizer = Whitespace()\n\n# Configure trainer\ntrainer = BpeTrainer(\n vocab_size=30000,\n special_tokens=[\"[UNK]\", \"[CLS]\", \"[SEP]\", \"[PAD]\", \"[MASK]\"],\n min_frequency=2\n)\n\n# Train on files\nfiles = [\"train.txt\", \"validation.txt\"]\ntokenizer.train(files, trainer)\n\n# Save\ntokenizer.save(\"my-tokenizer.json\")\n```\n\n**Training time**: ~1-2 minutes for 100MB corpus, ~10-20 minutes for 1GB\n\n### Batch encoding with padding\n\n```python\n# Enable padding\ntokenizer.enable_padding(pad_id=3, pad_token=\"[PAD]\")\n\n# Encode batch\ntexts = [\"Hello world\", \"This is a longer sentence\"]\nencodings = tokenizer.encode_batch(texts)\n\nfor encoding in encodings:\n print(encoding.ids)\n# [101, 7592, 2088, 102, 3, 3, 3]\n# [101, 2023, 2003, 1037, 2936, 6251, 102]\n```\n\n## Tokenization algorithms\n\n### BPE (Byte-Pair Encoding)\n\n**How it works**:\n1. Start with character-level vocabulary\n2. Find most frequent character pair\n3. Merge into new token, add to vocabulary\n4. Repeat until vocabulary size reached\n\n**Used by**: GPT-2, GPT-3, RoBERTa, BART, DeBERTa\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import BPE\nfrom tokenizers.trainers import BpeTrainer\nfrom tokenizers.pre_tokenizers import ByteLevel\n\ntokenizer = Tokenizer(BPE(unk_token=\"<|endoftext|>\"))\ntokenizer.pre_tokenizer = ByteLevel()\n\ntrainer = BpeTrainer(\n vocab_size=50257,\n special_tokens=[\"<|endoftext|>\"],\n min_frequency=2\n)\n\ntokenizer.train(files=[\"data.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Handles OOV words well (breaks into subwords)\n- Flexible vocabulary size\n- Good for morphologically rich languages\n\n**Trade-offs**:\n- Tokenization depends on merge order\n- May split common words unexpectedly\n\n### WordPiece\n\n**How it works**:\n1. Start with character vocabulary\n2. Score merge pairs: `frequency(pair) / (frequency(first) × frequency(second))`\n3. Merge highest scoring pair\n4. Repeat until vocabulary size reached\n\n**Used by**: BERT, DistilBERT, MobileBERT\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import WordPiece\nfrom tokenizers.trainers import WordPieceTrainer\nfrom tokenizers.pre_tokenizers import Whitespace\nfrom tokenizers.normalizers import BertNormalizer\n\ntokenizer = Tokenizer(WordPiece(unk_token=\"[UNK]\"))\ntokenizer.normalizer = BertNormalizer(lowercase=True)\ntokenizer.pre_tokenizer = Whitespace()\n\ntrainer = WordPieceTrainer(\n vocab_size=30522,\n special_tokens=[\"[UNK]\", \"[CLS]\", \"[SEP]\", \"[PAD]\", \"[MASK]\"],\n continuing_subword_prefix=\"##\"\n)\n\ntokenizer.train(files=[\"corpus.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Prioritizes meaningful merges (high score = semantically related)\n- Used successfully in BERT (state-of-the-art results)\n\n**Trade-offs**:\n- Unknown words become `[UNK]` if no subword match\n- Saves vocabulary, not merge rules (larger files)\n\n### Unigram\n\n**How it works**:\n1. Start with large vocabulary (all substrings)\n2. Compute loss for corpus with current vocabulary\n3. Remove tokens with minimal impact on loss\n4. Repeat until vocabulary size reached\n\n**Used by**: ALBERT, T5, mBART, XLNet (via SentencePiece)\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import Unigram\nfrom tokenizers.trainers import UnigramTrainer\n\ntokenizer = Tokenizer(Unigram())\n\ntrainer = UnigramTrainer(\n vocab_size=8000,\n special_tokens=[\"\", \"\", \"\"],\n unk_token=\"\"\n)\n\ntokenizer.train(files=[\"data.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Probabilistic (finds most likely tokenization)\n- Works well for languages without word boundaries\n- Handles diverse linguistic contexts\n\n**Trade-offs**:\n- Computationally expensive to train\n- More hyperparameters to tune\n\n## Tokenization pipeline\n\nComplete pipeline: **Normalization → Pre-tokenization → Model → Post-processing**\n\n### Normalization\n\nClean and standardize text:\n\n```python\nfrom tokenizers.normalizers import NFD, StripAccents, Lowercase, Sequence\n\ntokenizer.normalizer = Sequence([\n NFD(), # Unicode normalization (decompose)\n Lowercase(), # Convert to lowercase\n StripAccents() # Remove accents\n])\n\n# Input: \"Héllo WORLD\"\n# After normalization: \"hello world\"\n```\n\n**Common normalizers**:\n- `NFD`, `NFC`, `NFKD`, `NFKC` - Unicode normalization forms\n- `Lowercase()` - Convert to lowercase\n- `StripAccents()` - Remove accents (é → e)\n- `Strip()` - Remove whitespace\n- `Replace(pattern, content)` - Regex replacement\n\n### Pre-tokenization\n\nSplit text into word-like units:\n\n```python\nfrom tokenizers.pre_tokenizers import Whitespace, Punctuation, Sequence, ByteLevel\n\n# Split on whitespace and punctuation\ntokenizer.pre_tokenizer = Sequence([\n Whitespace(),\n Punctuation()\n])\n\n# Input: \"Hello, world!\"\n# After pre-tokenization: [\"Hello\", \",\", \"world\", \"!\"]\n```\n\n**Common pre-tokenizers**:\n- `Whitespace()` - Split on spaces, tabs, newlines\n- `ByteLevel()` - GPT-2 style byte-level splitting\n- `Punctuation()` - Isolate punctuation\n- `Digits(individual_digits=True)` - Split digits individually\n- `Metaspace()` - Replace spaces with ▁ (SentencePiece style)\n\n### Post-processing\n\nAdd special tokens for model input:\n\n```python\nfrom tokenizers.processors import TemplateProcessing\n\n# BERT-style: [CLS] sentence [SEP]\ntokenizer.post_processor = TemplateProcessing(\n single=\"[CLS] $A [SEP]\",\n pair=\"[CLS] $A [SEP] $B [SEP]\",\n special_tokens=[\n (\"[CLS]\", 1),\n (\"[SEP]\", 2),\n ],\n)\n```\n\n**Common patterns**:\n```python\n# GPT-2: sentence <|endoftext|>\nTemplateProcessing(\n single=\"$A <|endoftext|>\",\n special_tokens=[(\"<|endoftext|>\", 50256)]\n)\n\n# RoBERTa: sentence \nTemplateProcessing(\n single=\" $A \",\n pair=\" $A $B \",\n special_tokens=[(\"\", 0), (\"\", 2)]\n)\n```\n\n## Alignment tracking\n\nTrack token positions in original text:\n\n```python\noutput = tokenizer.encode(\"Hello, world!\")\n\n# Get token offsets\nfor token, offset in zip(output.tokens, output.offsets):\n start, end = offset\n print(f\"{token:10} → [{start:2}, {end:2}): {text[start:end]!r}\")\n\n# Output:\n# hello → [ 0, 5): 'Hello'\n# , → [ 5, 6): ','\n# world → [ 7, 12): 'world'\n# ! → [12, 13): '!'\n```\n\n**Use cases**:\n- Named entity recognition (map predictions back to text)\n- Question answering (extract answer spans)\n- Token classification (align labels to original positions)\n\n## Integration with transformers\n\n### Load with AutoTokenizer\n\n```python\nfrom transformers import AutoTokenizer\n\n# AutoTokenizer automatically uses fast tokenizers\ntokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n\n# Check if using fast tokenizer\nprint(tokenizer.is_fast) # True\n\n# Access underlying tokenizers.Tokenizer\nfast_tokenizer = tokenizer.backend_tokenizer\nprint(type(fast_tokenizer)) # \n```\n\n### Convert custom tokenizer to transformers\n\n```python\nfrom tokenizers import Tokenizer\nfrom transformers import PreTrainedTokenizerFast\n\n# Train custom tokenizer\ntokenizer = Tokenizer(BPE())\n# ... train tokenizer ...\ntokenizer.save(\"my-tokenizer.json\")\n\n# Wrap for transformers\ntransformers_tokenizer = PreTrainedTokenizerFast(\n tokenizer_file=\"my-tokenizer.json\",\n unk_token=\"[UNK]\",\n pad_token=\"[PAD]\",\n cls_token=\"[CLS]\",\n sep_token=\"[SEP]\",\n mask_token=\"[MASK]\"\n)\n\n# Use like any transformers tokenizer\noutputs = transformers_tokenizer(\n \"Hello world\",\n padding=True,\n truncation=True,\n max_length=512,\n return_tensors=\"pt\"\n)\n```\n\n## Common patterns\n\n### Train from iterator (large datasets)\n\n```python\nfrom datasets import load_dataset\n\n# Load dataset\ndataset = load_dataset(\"wikitext\", \"wikitext-103-raw-v1\", split=\"train\")\n\n# Create batch iterator\ndef batch_iterator(batch_size=1000):\n for i in range(0, len(dataset), batch_size):\n yield dataset[i:i + batch_size][\"text\"]\n\n# Train tokenizer\ntokenizer.train_from_iterator(\n batch_iterator(),\n trainer=trainer,\n length=len(dataset) # For progress bar\n)\n```\n\n**Performance**: Processes 1GB in ~10-20 minutes\n\n### Enable truncation and padding\n\n```python\n# Enable truncation\ntokenizer.enable_truncation(max_length=512)\n\n# Enable padding\ntokenizer.enable_padding(\n pad_id=tokenizer.token_to_id(\"[PAD]\"),\n pad_token=\"[PAD]\",\n length=512 # Fixed length, or None for batch max\n)\n\n# Encode with both\noutput = tokenizer.encode(\"This is a long sentence that will be truncated...\")\nprint(len(output.ids)) # 512\n```\n\n### Multi-processing\n\n```python\nfrom tokenizers import Tokenizer\nfrom multiprocessing import Pool\n\n# Load tokenizer\ntokenizer = Tokenizer.from_file(\"tokenizer.json\")\n\ndef encode_batch(texts):\n return tokenizer.encode_batch(texts)\n\n# Process large corpus in parallel\nwith Pool(8) as pool:\n # Split corpus into chunks\n chunk_size = 1000\n chunks = [corpus[i:i+chunk_size] for i in range(0, len(corpus), chunk_size)]\n\n # Encode in parallel\n results = pool.map(encode_batch, chunks)\n```\n\n**Speedup**: 5-8× with 8 cores\n\n## Performance benchmarks\n\n### Training speed\n\n| Corpus Size | BPE (30k vocab) | WordPiece (30k) | Unigram (8k) |\n|-------------|-----------------|-----------------|--------------|\n| 10 MB | 15 sec | 18 sec | 25 sec |\n| 100 MB | 1.5 min | 2 min | 4 min |\n| 1 GB | 15 min | 20 min | 40 min |\n\n**Hardware**: 16-core CPU, tested on English Wikipedia\n\n### Tokenization speed\n\n| Implementation | 1 GB corpus | Throughput |\n|----------------|-------------|---------------|\n| Pure Python | ~20 minutes | ~50 MB/min |\n| HF Tokenizers | ~15 seconds | ~4 GB/min |\n| **Speedup** | **80×** | **80×** |\n\n**Test**: English text, average sentence length 20 words\n\n### Memory usage\n\n| Task | Memory |\n|-------------------------|---------|\n| Load tokenizer | ~10 MB |\n| Train BPE (30k vocab) | ~200 MB |\n| Encode 1M sentences | ~500 MB |\n\n## Supported models\n\nPre-trained tokenizers available via `from_pretrained()`:\n\n**BERT family**:\n- `bert-base-uncased`, `bert-large-cased`\n- `distilbert-base-uncased`\n- `roberta-base`, `roberta-large`\n\n**GPT family**:\n- `gpt2`, `gpt2-medium`, `gpt2-large`\n- `distilgpt2`\n\n**T5 family**:\n- `t5-small`, `t5-base`, `t5-large`\n- `google/flan-t5-xxl`\n\n**Other**:\n- `facebook/bart-base`, `facebook/mbart-large-cc25`\n- `albert-base-v2`, `albert-xlarge-v2`\n- `xlm-roberta-base`, `xlm-roberta-large`\n\nBrowse all: https://huggingface.co/models?library=tokenizers\n\n## References\n\n- **[Training Guide](references/training.md)** - Train custom tokenizers, configure trainers, handle large datasets\n- **[Algorithms Deep Dive](references/algorithms.md)** - BPE, WordPiece, Unigram explained in detail\n- **[Pipeline Components](references/pipeline.md)** - Normalizers, pre-tokenizers, post-processors, decoders\n- **[Transformers Integration](references/integration.md)** - AutoTokenizer, PreTrainedTokenizerFast, special tokens\n\n## Resources\n\n- **Docs**: https://huggingface.co/docs/tokenizers\n- **GitHub**: https://github.com/huggingface/tokenizers ⭐ 9,000+\n- **Version**: 0.20.0+\n- **Course**: https://huggingface.co/learn/nlp-course/chapter6/1\n- **Paper**: BPE (Sennrich et al., 2016), WordPiece (Schuster & Nakajima, 2012)\n\n\n","0-autoresearch-skill/SKILL.md":"---\nname: autoresearch\ndescription: Orchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for execution, supports continuous agent operation via Claude Code /loop and OpenClaw heartbeat, and produces research presentations and papers. Use when starting a research project, running autonomous experiments, or managing a multi-hypothesis research effort.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Autonomous Research, Two-Loop Architecture, Experiment Orchestration, Research Synthesis, Project Management]\n---\n\n# Autoresearch\n\nAutonomous research orchestration for AI coding agents. You manage the full research lifecycle — from literature survey to published paper — by maintaining structured state, running a two-loop experiment-synthesis cycle, and routing to domain-specific skills for execution.\n\nYou are a research project manager, not a domain expert. You orchestrate; the domain skills execute.\n\n**This runs fully autonomously.** Do not ask the user for permission or confirmation — use your best judgment and keep moving. Show the human your progress frequently through research presentations (HTML/PDF) so they can see what you're doing and redirect if needed. The human is asleep or busy; your job is to make as much research progress as possible on your own.\n\n## Getting Started\n\nUsers arrive in different states. Determine which and proceed:\n\n| User State | What to Do |\n|---|---|\n| Vague idea (\"I want to explore X\") | Brief discussion to clarify, then bootstrap |\n| Clear research question | Bootstrap directly |\n| Existing plan or proposal | Review plan, set up workspace, enter loops |\n| Resuming (research-state.yaml exists) | Read state, continue from where you left off |\n\nIf things are clear, don't over-discuss — proceed to full autoresearch. Most users want you to just start researching.\n\n**Step 0 — before anything else**: Set up the agent continuity loop. See [Agent Continuity](#agent-continuity-mandatory--set-up-first). This is MANDATORY. Without it, the research stops after one cycle.\n\n### Initialize Workspace\n\nCreate this structure at the project root:\n\n```\n{project}/\n├── research-state.yaml # Central state tracking\n├── research-log.md # Decision timeline\n├── findings.md # Evolving narrative synthesis\n├── literature/ # Papers, survey notes\n├── src/ # Reusable code (utils, plotting, shared modules)\n├── data/ # Raw result data (CSVs, JSONs, checkpoints)\n├── experiments/ # Per-hypothesis work\n│ └── {hypothesis-slug}/\n│ ├── protocol.md # What, why, and prediction\n│ ├── code/ # Experiment-specific code\n│ ├── results/ # Raw outputs, metrics, logs\n│ └── analysis.md # What we learned\n├── to_human/ # Progress presentations and reports for human review\n└── paper/ # Final paper (via ml-paper-writing)\n```\n\n- **`src/`**: When you write useful code (plotting functions, data loaders, evaluation helpers), move it here so it can be reused across experiments. Don't duplicate code in every experiment directory.\n- **`data/`**: Save raw result data (metric CSVs, training logs, small outputs) here in a structured way. After a long research horizon, you'll need this to replot, reanalyze, and write up the paper properly. Name files descriptively (e.g., `trajectory_H1_runs001-010.csv`). Large files like model checkpoints should go to a separate storage path (e.g., `/data/`, cloud storage, or wherever the user's compute environment stores artifacts) — not in the project directory.\n\nInitialize `research-state.yaml`, `research-log.md`, and `findings.md` from [templates/](templates/). Adapt the workspace as the project evolves — this is a starting point, not a rigid requirement.\n\n## The Two-Loop Architecture\n\nThis is the core engine. Everything else supports it.\n\n```\nBOOTSTRAP (once, lightweight)\n Scope question → search literature → form initial hypotheses\n\nINNER LOOP (fast, autonomous, repeating)\n Pick hypothesis → experiment → measure → record → learn → next\n Goal: run constrained experiments with clear measurable outcomes\n\nOUTER LOOP (periodic, reflective)\n Review results → find patterns → update findings.md →\n new hypotheses → decide direction\n Goal: synthesize understanding, find the story — this is where novelty comes from\n\nFINALIZE (when concluding)\n Write paper via ml-paper-writing → final presentation → archive\n```\n\nThe inner loop runs tight experiment cycles with clear measurable outcomes. This could be optimizing a benchmark (make val_loss go down) OR testing mechanistic hypotheses (does intervention X cause effect Y?). The outer loop steps back to ask: what do these results *mean*? What patterns emerge? What's the story? Research is open-ended — the two loops let you both optimize and discover.\n\nThere is no rigid boundary between the two loops — you decide when enough inner loop results have accumulated to warrant reflection. Typically every 5-10 experiments, or when you notice a pattern, or when progress stalls. The agent's judgment drives the rhythm.\n\n### Research is Non-Linear\n\nThe two-loop structure is a rhythm, not a railroad. At any point during research you can and should:\n\n- **Return to literature** when results surprise you, assumptions break, or you need context for a new direction — always save what you find to `literature/`\n- **Brainstorm new ideas** using `21-research-ideation/` skills when you're stuck or when results open unexpected questions\n- **Pivot the question entirely** if experiments reveal the original question was wrong or less interesting than what you found\n\nThis is normal. Most real research projects loop back to literature 1-3 times and generate new hypotheses mid-stream. Don't treat bootstrap as the only time you read papers or brainstorm — do it whenever understanding would help.\n\n## Bootstrap: Literature and Hypotheses\n\nBefore entering the loops, understand the landscape. Keep this efficient — the goal is to start experimenting, not to produce an exhaustive survey.\n\n1. **Search literature** for the research question. Use multiple sources — never stop at one:\n - **Exa MCP** (`web_search_exa`) if available — best for broad discovery and finding relevant papers quickly\n - **Semantic Scholar** (`pip install semanticscholar`) — best for ML/AI papers, citation graphs, and specific paper lookup. See `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code examples\n - **arXiv** (`pip install arxiv`) — best for recent preprints and open-access papers\n - **CrossRef** — best for DOI lookup and BibTeX retrieval\n - Keep searching until you have good coverage. If one source comes up empty, try another with different keywords\n\n **Save everything to `literature/`**: For every paper you find, save a summary to `literature/` — title, authors, year, key findings, relevance to your question, and the URL/DOI. Create one file per paper and a running `literature/survey.md` with all summaries. This is your reference library — you and future sessions will need it throughout the project.\n\n2. **Identify gaps** from the literature\n - What's been tried? What hasn't? Where do existing methods break?\n - What do Discussion sections flag as future work?\n\n3. **Form initial hypotheses** — invoke `21-research-ideation/` skills\n - `brainstorming-research-ideas` for structured diverge-converge workflow\n - `creative-thinking-for-research` for deeper cognitive frameworks\n - Each hypothesis must be testable with a clear prediction\n\n4. **Define the evaluation**\n - Set the proxy metric and baseline before running experiments\n - The metric should be computable quickly (minutes, not hours)\n - Lock evaluation criteria upfront to prevent unconscious metric gaming\n\n5. **Record** in research-state.yaml, log the bootstrap in research-log.md\n\n## The Inner Loop\n\nRapid iteration with clear measurable outcomes. Two flavors:\n\n- **Optimization**: make a metric go up/down (val_loss, accuracy, throughput). Think Karpathy's autoresearch.\n- **Discovery**: test mechanistic hypotheses about why something works. The metric is a measurement (does grokking happen faster? does entropy increase before forgetting?), not just a target to optimize.\n\n```\n1. Pick the highest-priority untested hypothesis\n2. Write a protocol: what change, what prediction, why\n Lock it: commit to git BEFORE running (research(protocol): {hypothesis})\n This creates temporal proof your plan existed before results\n3. Run the experiment (invoke the relevant domain skill)\n4. Sanity check before trusting results:\n - Did training converge? No NaN/Inf?\n - Does baseline reproduce expected performance?\n - Data loading correct? (spot-check a few samples)\n5. Measure the proxy metric\n6. Record in experiments/{hypothesis-slug}/\n Label clearly: CONFIRMATORY (in your protocol) vs EXPLORATORY (discovered during execution)\n7. If positive: keep, note WHY it worked\n8. If negative: this is progress — note what it rules out and what it suggests\n9. Update research-state.yaml\n10. If stuck: search literature or invoke ideation skills — don't just keep trying random things\n```\n\n**Never stop.** Even if something fails, find a path forward. Debug, adjust, simplify, or pivot — but keep the research moving. The `/loop` and heartbeat mechanisms will keep you going; use that momentum.\n\n### Route to Domain Skills\n\nWhen you need domain-specific execution, search the skills library:\n\n| Research Activity | Look In |\n|---|---|\n| Data preparation | `05-data-processing/` |\n| Model training / fine-tuning | `01-model-architecture/`, `03-fine-tuning/`, `06-post-training/` |\n| Distributed training | `08-distributed-training/` |\n| Optimization (quantization, attention) | `10-optimization/` |\n| Evaluation / benchmarks | `11-evaluation/` |\n| Inference / serving | `12-inference-serving/` |\n| Interpretability analysis | `04-mechanistic-interpretability/` |\n| Experiment tracking (W&B, MLflow) | `13-mlops/` |\n| Cloud compute | `09-infrastructure/` |\n\nRead the relevant SKILL.md before starting — it has workflows, common issues, and code examples. See [references/skill-routing.md](references/skill-routing.md) for a complete guide.\n\n### Track the Experiment Trajectory\n\nMaintain a running record of measurable outcomes across experiments:\n\n```json\n{\n \"experiment_id\": \"run_014\",\n \"hypothesis\": \"H3\",\n \"metric_value\": 0.847,\n \"baseline\": 0.812,\n \"delta\": \"+0.035\",\n \"wall_time_min\": 23,\n \"change_summary\": \"Added cosine annealing warmup schedule\"\n}\n```\n\nThis trajectory produces the optimization plot (like Karpathy's progress chart) — include it in progress reports. Humans love seeing the upward curve.\n\n## The Outer Loop\n\nStep back from individual experiments. Synthesize.\n\n```\n1. Review all results since last reflection\n2. Cluster by type: what kinds of changes worked? Which didn't?\n3. Ask WHY — identify the mechanism behind successes and failures\n4. Update findings.md with current understanding\n5. Search literature if results were surprising or assumptions need revisiting\n6. Generate new hypotheses if warranted (invoke 21-research-ideation/ skills)\n7. Decide direction (see criteria below)\n8. Update research-state.yaml with new direction\n9. Log the reflection in research-log.md\n10. If there's something meaningful, generate a progress presentation\n```\n\n### Deciding Direction\n\nDon't just pick randomly — use these criteria:\n\n**DEEPEN** — a supported result raises follow-up questions\n- Does the effect hold under different conditions? What's the mechanism?\n- Action: generate sub-hypotheses (H1.1, H1.2) → back to inner loop\n\n**BROADEN** — current results are solid, but adjacent questions are untested\n- New questions emerged. The current contribution is clear but more is possible.\n- Action: generate new root hypotheses → back to inner loop\n\n**PIVOT** — results invalidate key assumptions or something more interesting appeared\n- A core assumption was wrong, or an unexpected finding is more promising than the original question.\n- Action: return to literature with new questions → re-bootstrap\n\n**CONCLUDE** — sufficient evidence for a contribution\n- At least one hypothesis is strongly supported (or a coherent set of negative results)\n- Key ablations completed, error analysis done\n- findings.md reads like a paper backbone — a human could write the abstract from it\n- No critical open questions that would change the story\n\nNote: coherent negative results are a valid contribution. \"X does NOT work because Y\" is publishable if the reasoning is rigorous.\n\n### findings.md Is Your Project Memory\n\nThis file serves two purposes: it's the research narrative for humans AND your accumulated knowledge base as an agent. Read it at the start of every session, /loop tick, or heartbeat to remember what you've learned.\n\nAfter every outer loop, update it to answer:\n\n- What do we know so far? (Current Understanding)\n- What patterns explain our results? (Patterns and Insights)\n- What specific things did we learn not to repeat? (Lessons and Constraints)\n- What remains open? (Open Questions)\n\nThe \"Lessons and Constraints\" section is especially important — it captures specific actionable learnings like \"weight decay > 0.1 diverges at this scale\" or \"baseline only reproduces with batch_size=64.\" This prevents the agent from repeating failed approaches across sessions.\n\n**Quality test**: After 30 inner loop experiments, a human should be able to read findings.md and write a paper abstract from it. If they can't, the outer loop isn't synthesizing — it's just logging.\n\n## Agent Continuity (MANDATORY — Set Up First)\n\n**Before doing anything else**, set up the wall-clock loop. This is what keeps the research running continuously. Without it, the agent stops after one cycle.\n\n### Claude Code\n\n**First action — run this immediately:**\n\n```\n/loop 20m Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a report in to_human/ and opening the HTML/PDF. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\n```\n\nThis fires every 20 minutes regardless of what's happening. It's a rhythm that keeps you working — not a research phase boundary. If your previous work isn't done, just continue it.\n\n### OpenClaw\n\n**First action — set up a 20-minute cron job in the current session. You MUST do this before any research work.**\n\nUse the `cron.add` tool to create a recurring job bound to this chat session:\n\n```json\n{\n \"name\": \"autoresearch-loop\",\n \"schedule\": { \"kind\": \"every\", \"everyMs\": 1200000 },\n \"sessionTarget\": \"current\",\n \"payload\": {\n \"kind\": \"agentTurn\",\n \"message\": \"Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a PDF report in to_human/ and sending it to the user via Telegram, WhatsApp, or Slack. When you get an exciting result or interesting plot, text it to the user right away — don't wait for a full report. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\"\n }\n}\n```\n\nKey details:\n- `sessionTarget: \"current\"` binds the cron to this chat session so it maintains conversation context across ticks\n- `everyMs: 1200000` = 20 minutes\n- After creation, verify with `cron.list` that the job exists and is enabled\n- If the cron fires while you're mid-experiment, just continue — the tick is a nudge, not a restart\n\n### What the Loop Does\n\nThe `/loop` and cron job are purely **wall-clock rhythm**. They are completely separate from your research loops (inner/outer). On each tick:\n\n1. Read `research-state.yaml` and `findings.md` — remember where you are\n2. Check if anything is broken (failed experiments, stalled training, errors)\n3. If on track → keep working on whatever you were doing\n4. If stuck or something's wrong → step back, diagnose, fix, then continue\n5. Never idle. Always be making progress.\n\n## Progress Reporting\n\nWhen you have something meaningful to share, create a research presentation — not just a status dashboard, but a compelling story.\n\n**When to report** (your judgment):\n- After an outer loop that found a significant pattern\n- When the optimization trajectory shows clear progress (include the plot!)\n- After a pivot in direction\n- Before requesting human input on a decision\n- When concluding\n\n**What to include** (adapt to what's compelling):\n- The research question and why it matters\n- Key results with visualizations (plots, metric tables)\n- The optimization trajectory chart (metric over experiments)\n- What was tried and why (selective, not exhaustive)\n- Current understanding (the findings narrative)\n- What's planned next\n\nFor Claude Code: generate HTML and `open` it. If HTML fails to open or render, convert to PDF as fallback (use `weasyprint`, `playwright pdf`, or `wkhtmltopdf`). For OpenClaw: generate PDF directly.\n\nSee [references/progress-reporting.md](references/progress-reporting.md) for template scaffolding and the optimization plot approach. Use the template as a starting point — be creative with what you show.\n\n## Git Protocol\n\nCommit at natural research milestones:\n\n| When | Message Pattern |\n|---|---|\n| Workspace initialized | `research(init): {project} — {question}` |\n| Experiment protocol locked | `research(protocol): {hypothesis}` |\n| Significant results | `research(results): {hypothesis} — {outcome}` |\n| Outer loop direction change | `research(reflect): {direction} — {reason}` |\n| Paper draft complete | `research(paper): {title}` |\n\n**Hard rule**: Protocol commits MUST precede result commits. Never combine them. The git history is your lightweight pre-registration — it proves what you planned before you saw results. Don't commit after every experiment — commit when there's meaningful progress.\n\n## Concluding: Paper Writing\n\nWhen the outer loop decides to CONCLUDE:\n\n1. Ensure findings.md has a clear, well-supported narrative\n2. Study 2-3 top related papers to learn their format, style, and section structure\n3. Invoke the `20-ml-paper-writing` skill — it has LaTeX templates for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, and systems venues\n4. Feed it the accumulated literature, experimental results, and findings\n5. Follow its citation verification workflow — never hallucinate references\n6. Generate a final comprehensive research presentation\n\nProceed autonomously through the writing process. If the ml-paper-writing skill suggests human collaboration points, adapt and keep going — produce the best draft you can. The human will review and provide feedback.\n\n## Research Discipline\n\nPrinciples to enforce continuously — not tied to any specific phase:\n\n- **Lock before you run**: Commit your experiment protocol to git before executing. This proves your plan existed before you saw results. Never combine protocol + results in one commit.\n- **Confirmatory vs exploratory**: Results matching your locked protocol are confirmatory. Everything else is exploratory — interesting but requiring more skepticism.\n- **Negative results are progress**: A refuted hypothesis tells you something. Log what it rules out and what it suggests. Don't treat it as failure.\n- **Sanity check before analysis**: Verify training converged, baselines reproduce, and data is correct before trusting your primary metric.\n- **Return to literature when confused**: Don't guess — search. If results surprise you or assumptions break, go find papers. Use Exa MCP for discovery, Semantic Scholar for specific ML/AI paper lookup, arXiv for preprints.\n- **Never stop**: Don't wait for human approval on routine decisions. If a skill or tool suggests collaboration, adapt and keep going. Find the best path forward autonomously. The human will see your progress reports and can redirect if needed.\n- **Use whatever compute is available**: Adapt to the user's environment — local GPU, cluster job submission, cloud instances, or just CPU. If no GPU is available, use CPU and adjust experiment scale accordingly. Don't block on compute availability.\n\n## Quality Standards\n\n**Good agent behavior:**\n- Hypotheses have mechanistic reasoning (\"X because Y, predicting Z\"), not just \"try X\"\n- findings.md builds a coherent narrative, not a flat list of results\n- Negative results are recorded with what they rule out\n- The agent updates its model when experiments contradict expectations\n- Progress reports tell a research story with compelling visualizations\n\n**Bad agent behavior:**\n- Pure hyperparameter sweeps without interpretation\n- findings.md is just experiment logs copy-pasted\n- Agent never revisits its assumptions after failures\n- Optimizing metrics without understanding why changes work\n\n## When to Use vs Alternatives\n\n**Use autoresearch when:**\n- You have a research question explorable through experiments\n- There's a measurable proxy metric for inner loop optimization\n- The real contribution requires synthesis beyond the metric\n- You want continuous autonomous research operation\n\n**Use individual domain skills instead when:**\n- You have a specific one-off task (train a model, run eval, write a paper)\n- No iterative experimentation needed\n\n## Common Issues\n\n**Inner loop stalls (no metric improvement)**\nRun an outer loop. Is the metric the right one? Is the search space exhausted? Consider broadening or pivoting. Search literature for new approaches.\n\n**Stuck and not making progress**\nDon't keep trying random changes. Step back: search literature for related work, invoke `21-research-ideation/` brainstorming skills, or run an outer loop reflection. Being stuck means you need new information or a new perspective, not more experiments.\n\n**Results contradict baseline expectations**\nInvestigate, don't ignore. Return to literature — your protocol might have an error, the published baseline may be wrong, or conditions differ. Update findings.md with what you learn.\n\n**Agent loses context between ticks**\nEnsure research-state.yaml and findings.md are updated after every action. These files are your memory across sessions.\n\n**Can't find relevant papers**\nTry multiple approaches in order: Exa MCP for broad search, Semantic Scholar for specific ML/AI paper lookup (`pip install semanticscholar`), arXiv for preprints (`pip install arxiv`). Check `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code. Note: Google Scholar has no official API — use Semantic Scholar instead for programmatic search.\n\n**No GPU available**\nUse CPU and scale experiments down. Many research tasks (analysis, interpretability, small model training) run fine on CPU. Adjust experiment design to fit available compute rather than blocking.\n\n**Experiments take longer than /loop interval**\nNormal. On the next tick, check if it finished. If not, keep waiting or do something else useful (update notes, search papers). Adjust interval if needed.\n\n**Not sure when to conclude**\nThree questions: Do you have a strongly supported finding? Can you explain WHY it works? Would findings.md make a convincing paper abstract? If yes to all: conclude.\n\n## Advanced Topics\n\n- **Detailed agent continuity**: [references/agent-continuity.md](references/agent-continuity.md)\n- **Progress presentation templates**: [references/progress-reporting.md](references/progress-reporting.md)\n- **Complete skill routing**: [references/skill-routing.md](references/skill-routing.md)\n","03-fine-tuning/unsloth/SKILL.md":"---\nname: unsloth\ndescription: Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, Unsloth, Fast Training, LoRA, QLoRA, Memory-Efficient, Optimization, Llama, Mistral, Gemma, Qwen]\ndependencies: [unsloth, torch, transformers, trl, datasets, peft]\n---\n\n# Unsloth Skill\n\nComprehensive assistance with unsloth development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with unsloth\n- Asking about unsloth features or APIs\n- Implementing unsloth solutions\n- Debugging unsloth code\n- Learning unsloth best practices\n\n## Quick Reference\n\n### Common Patterns\n\n*Quick reference patterns will be added as you use the skill.*\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **llms-txt.md** - Llms-Txt documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n\n\n\n","03-fine-tuning/llama-factory/SKILL.md":"---\nname: llama-factory\ndescription: Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, LLaMA Factory, LLM, WebUI, No-Code, QLoRA, LoRA, Multimodal, HuggingFace, Llama, Qwen, Gemma]\ndependencies: [llmtuner, torch, transformers, datasets, peft, accelerate, gradio]\n---\n\n# Llama-Factory Skill\n\nComprehensive assistance with llama-factory development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with llama-factory\n- Asking about llama-factory features or APIs\n- Implementing llama-factory solutions\n- Debugging llama-factory code\n- Learning llama-factory best practices\n\n## Quick Reference\n\n### Common Patterns\n\n*Quick reference patterns will be added as you use the skill.*\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **_images.md** - Images documentation\n- **advanced.md** - Advanced documentation\n- **getting_started.md** - Getting Started documentation\n- **other.md** - Other documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n","02-tokenization/sentencepiece/SKILL.md":"---\nname: sentencepiece\ndescription: Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Tokenization, SentencePiece, Language-Independent, BPE, Unigram, Multilingual, CJK Languages, Unicode, Deterministic, Google]\ndependencies: [sentencepiece, transformers]\n---\n\n# SentencePiece - Language-Independent Tokenization\n\nUnsupervised tokenizer that works on raw text without language-specific preprocessing.\n\n## When to use SentencePiece\n\n**Use SentencePiece when:**\n- Building multilingual models (no language-specific rules)\n- Working with CJK languages (Chinese, Japanese, Korean)\n- Need reproducible tokenization (deterministic vocabulary)\n- Want to train on raw text (no pre-tokenization needed)\n- Require lightweight deployment (6MB memory, 50k sentences/sec)\n\n**Performance**:\n- **Speed**: 50,000 sentences/sec\n- **Memory**: ~6MB for loaded model\n- **Languages**: All (language-independent)\n\n**Use alternatives instead**:\n- **HuggingFace Tokenizers**: Faster training, more flexibility\n- **tiktoken**: OpenAI models (GPT-3.5/4)\n- **BERT WordPiece**: English-centric tasks\n\n## Quick start\n\n### Installation\n\n```bash\n# Python\npip install sentencepiece\n\n# C++ (requires CMake)\ngit clone https://github.com/google/sentencepiece.git\ncd sentencepiece\nmkdir build && cd build\ncmake .. && make -j $(nproc)\nsudo make install\n```\n\n### Train model\n\n```bash\n# Command-line (BPE with 8000 vocab)\nspm_train --input=data.txt --model_prefix=m --vocab_size=8000 --model_type=bpe\n\n# Python API\nimport sentencepiece as spm\n\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='m',\n vocab_size=8000,\n model_type='bpe'\n)\n```\n\n**Training time**: ~1-2 minutes for 100MB corpus\n\n### Encode and decode\n\n```python\nimport sentencepiece as spm\n\n# Load model\nsp = spm.SentencePieceProcessor(model_file='m.model')\n\n# Encode to pieces\npieces = sp.encode('This is a test', out_type=str)\nprint(pieces) # ['▁This', '▁is', '▁a', '▁test']\n\n# Encode to IDs\nids = sp.encode('This is a test', out_type=int)\nprint(ids) # [284, 47, 11, 1243]\n\n# Decode\ntext = sp.decode(ids)\nprint(text) # \"This is a test\"\n```\n\n## Language-independent design\n\n### Whitespace as symbol (▁)\n\n```python\ntext = \"Hello world\"\npieces = sp.encode(text, out_type=str)\nprint(pieces) # ['▁Hello', '▁world']\n\n# Decode preserves spaces\ndecoded = sp.decode_pieces(pieces)\nprint(decoded) # \"Hello world\"\n```\n\n**Key principle**: Treat text as raw Unicode, whitespace = ▁ (meta symbol)\n\n## Tokenization algorithms\n\n### BPE (Byte-Pair Encoding)\n\n```python\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='bpe_model',\n vocab_size=16000,\n model_type='bpe'\n)\n```\n\n**Used by**: mBART\n\n### Unigram (default)\n\n```python\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='unigram_model',\n vocab_size=8000,\n model_type='unigram'\n)\n```\n\n**Used by**: T5, ALBERT, XLNet\n\n## Training configuration\n\n### Essential parameters\n\n```python\nspm.SentencePieceTrainer.train(\n input='corpus.txt',\n model_prefix='m',\n vocab_size=32000,\n model_type='unigram',\n character_coverage=0.9995, # 1.0 for CJK\n user_defined_symbols=['[SEP]', '[CLS]'],\n unk_piece='',\n num_threads=16\n)\n```\n\n### Character coverage\n\n| Language Type | Coverage | Rationale |\n|---------------|----------|-----------|\n| English | 0.9995 | Most common chars |\n| CJK (Chinese) | 1.0 | All characters needed |\n| Multilingual | 0.9995 | Balance |\n\n## Encoding options\n\n### Subword regularization\n\n```python\n# Sample different tokenizations\nfor _ in range(3):\n pieces = sp.encode('tokenization', out_type=str, enable_sampling=True, alpha=0.1)\n print(pieces)\n\n# Output (different each time):\n# ['▁token', 'ization']\n# ['▁tok', 'en', 'ization']\n```\n\n**Use case**: Data augmentation for robustness.\n\n## Common patterns\n\n### T5-style training\n\n```python\nspm.SentencePieceTrainer.train(\n input='c4_corpus.txt',\n model_prefix='t5',\n vocab_size=32000,\n model_type='unigram',\n user_defined_symbols=[f'' for i in range(100)],\n unk_id=2,\n eos_id=1,\n pad_id=0\n)\n```\n\n### Integration with transformers\n\n```python\nfrom transformers import T5Tokenizer\n\n# T5 uses SentencePiece internally\ntokenizer = T5Tokenizer.from_pretrained('t5-base')\ninputs = tokenizer('translate English to French: Hello', return_tensors='pt')\n```\n\n## Performance benchmarks\n\n### Training speed\n\n| Corpus | BPE (16k) | Unigram (8k) |\n|--------|-----------|--------------|\n| 100 MB | 1-2 min | 3-4 min |\n| 1 GB | 10-15 min | 30-40 min |\n\n### Tokenization speed\n\n- **SentencePiece**: 50,000 sentences/sec\n- **HF Tokenizers**: 200,000 sentences/sec (4× faster)\n\n## Supported models\n\n**T5 family**: `t5-base`, `t5-large` (32k vocab, Unigram)\n**ALBERT**: `albert-base-v2` (30k vocab, Unigram)\n**XLNet**: `xlnet-base-cased` (32k vocab, Unigram)\n**mBART**: `facebook/mbart-large-50` (250k vocab, BPE)\n\n## References\n\n- **[Training Guide](references/training.md)** - Detailed options, corpus preparation\n- **[Algorithms](references/algorithms.md)** - BPE vs Unigram, subword regularization\n\n## Resources\n\n- **GitHub**: https://github.com/google/sentencepiece ⭐ 10,000+\n- **Paper**: https://arxiv.org/abs/1808.06226 (EMNLP 2018)\n- **Version**: 0.2.0+\n\n\n","03-fine-tuning/axolotl/SKILL.md":"---\nname: axolotl\ndescription: Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, Axolotl, LLM, LoRA, QLoRA, DPO, KTO, ORPO, GRPO, YAML, HuggingFace, DeepSpeed, Multimodal]\ndependencies: [axolotl, torch, transformers, datasets, peft, accelerate, deepspeed]\n---\n\n# Axolotl Skill\n\nComprehensive assistance with axolotl development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with axolotl\n- Asking about axolotl features or APIs\n- Implementing axolotl solutions\n- Debugging axolotl code\n- Learning axolotl best practices\n\n## Quick Reference\n\n### Common Patterns\n\n**Pattern 1:** To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example:\n\n```\n./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3\n```\n\n**Pattern 2:** Configure your model to use FSDP in the Axolotl yaml. For example:\n\n```\nfsdp_version: 2\nfsdp_config:\n offload_params: true\n state_dict_type: FULL_STATE_DICT\n auto_wrap_policy: TRANSFORMER_BASED_WRAP\n transformer_layer_cls_to_wrap: LlamaDecoderLayer\n reshard_after_forward: true\n```\n\n**Pattern 3:** The context_parallel_size should be a divisor of the total number of GPUs. For example:\n\n```\ncontext_parallel_size\n```\n\n**Pattern 4:** For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and context_parallel_size=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU micro_batch_size is 2, the global batch size decreases from 16 to 4\n\n```\ncontext_parallel_size=4\n```\n\n**Pattern 5:** Setting save_compressed: true in your configuration enables saving models in a compressed format, which: - Reduces disk space usage by approximately 40% - Maintains compatibility with vLLM for accelerated inference - Maintains compatibility with llmcompressor for further optimization (example: quantization)\n\n```\nsave_compressed: true\n```\n\n**Pattern 6:** Note It is not necessary to place your integration in the integrations folder. It can be in any location, so long as it’s installed in a package in your python env. See this repo for an example: https://github.com/axolotl-ai-cloud/diff-transformer\n\n```\nintegrations\n```\n\n**Pattern 7:** Handle both single-example and batched data. - single example: sample[‘input_ids’] is a list[int] - batched data: sample[‘input_ids’] is a list[list[int]]\n\n```\nutils.trainer.drop_long_seq(sample, sequence_len=2048, min_sequence_len=2)\n```\n\n### Example Code Patterns\n\n**Example 1** (python):\n```python\ncli.cloud.modal_.ModalCloud(config, app=None)\n```\n\n**Example 2** (python):\n```python\ncli.cloud.modal_.run_cmd(cmd, run_folder, volumes=None)\n```\n\n**Example 3** (python):\n```python\ncore.trainers.base.AxolotlTrainer(\n *_args,\n bench_data_collator=None,\n eval_data_collator=None,\n dataset_tags=None,\n **kwargs,\n)\n```\n\n**Example 4** (python):\n```python\ncore.trainers.base.AxolotlTrainer.log(logs, start_time=None)\n```\n\n**Example 5** (python):\n```python\nprompt_strategies.input_output.RawInputOutputPrompter()\n```\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **api.md** - Api documentation\n- **dataset-formats.md** - Dataset-Formats documentation\n- **other.md** - Other documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n","03-fine-tuning/peft/SKILL.md":"---\nname: peft-fine-tuning\ndescription: Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, PEFT, LoRA, QLoRA, Parameter-Efficient, Adapters, Low-Rank, Memory Optimization, Multi-Adapter]\ndependencies: [peft>=0.13.0, transformers>=4.45.0, torch>=2.0.0, bitsandbytes>=0.43.0]\n---\n\n# PEFT (Parameter-Efficient Fine-Tuning)\n\nFine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods.\n\n## When to use PEFT\n\n**Use PEFT/LoRA when:**\n- Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100)\n- Need to train <1% parameters (6MB adapters vs 14GB full model)\n- Want fast iteration with multiple task-specific adapters\n- Deploying multiple fine-tuned variants from one base model\n\n**Use QLoRA (PEFT + quantization) when:**\n- Fine-tuning 70B models on single 24GB GPU\n- Memory is the primary constraint\n- Can accept ~5% quality trade-off vs full fine-tuning\n\n**Use full fine-tuning instead when:**\n- Training small models (<1B parameters)\n- Need maximum quality and have compute budget\n- Significant domain shift requires updating all weights\n\n## Quick start\n\n### Installation\n\n```bash\n# Basic installation\npip install peft\n\n# With quantization support (recommended)\npip install peft bitsandbytes\n\n# Full stack\npip install peft transformers accelerate bitsandbytes datasets\n```\n\n### LoRA fine-tuning (standard)\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer\nfrom peft import get_peft_model, LoraConfig, TaskType\nfrom datasets import load_dataset\n\n# Load base model\nmodel_name = \"meta-llama/Llama-3.1-8B\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=\"auto\", device_map=\"auto\")\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# LoRA configuration\nlora_config = LoraConfig(\n task_type=TaskType.CAUSAL_LM,\n r=16, # Rank (8-64, higher = more capacity)\n lora_alpha=32, # Scaling factor (typically 2*r)\n lora_dropout=0.05, # Dropout for regularization\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\"], # Attention layers\n bias=\"none\" # Don't train biases\n)\n\n# Apply LoRA\nmodel = get_peft_model(model, lora_config)\nmodel.print_trainable_parameters()\n# Output: trainable params: 13,631,488 || all params: 8,043,307,008 || trainable%: 0.17%\n\n# Prepare dataset\ndataset = load_dataset(\"databricks/databricks-dolly-15k\", split=\"train\")\n\ndef tokenize(example):\n text = f\"### Instruction:\\n{example['instruction']}\\n\\n### Response:\\n{example['response']}\"\n return tokenizer(text, truncation=True, max_length=512, padding=\"max_length\")\n\ntokenized = dataset.map(tokenize, remove_columns=dataset.column_names)\n\n# Training\ntraining_args = TrainingArguments(\n output_dir=\"./lora-llama\",\n num_train_epochs=3,\n per_device_train_batch_size=4,\n gradient_accumulation_steps=4,\n learning_rate=2e-4,\n fp16=True,\n logging_steps=10,\n save_strategy=\"epoch\"\n)\n\ntrainer = Trainer(\n model=model,\n args=training_args,\n train_dataset=tokenized,\n data_collator=lambda data: {\"input_ids\": torch.stack([f[\"input_ids\"] for f in data]),\n \"attention_mask\": torch.stack([f[\"attention_mask\"] for f in data]),\n \"labels\": torch.stack([f[\"input_ids\"] for f in data])}\n)\n\ntrainer.train()\n\n# Save adapter only (6MB vs 16GB)\nmodel.save_pretrained(\"./lora-llama-adapter\")\n```\n\n### QLoRA fine-tuning (memory-efficient)\n\n```python\nfrom transformers import AutoModelForCausalLM, BitsAndBytesConfig\nfrom peft import get_peft_model, LoraConfig, prepare_model_for_kbit_training\n\n# 4-bit quantization config\nbnb_config = BitsAndBytesConfig(\n load_in_4bit=True,\n bnb_4bit_quant_type=\"nf4\", # NormalFloat4 (best for LLMs)\n bnb_4bit_compute_dtype=\"bfloat16\", # Compute in bf16\n bnb_4bit_use_double_quant=True # Nested quantization\n)\n\n# Load quantized model\nmodel = AutoModelForCausalLM.from_pretrained(\n \"meta-llama/Llama-3.1-70B\",\n quantization_config=bnb_config,\n device_map=\"auto\"\n)\n\n# Prepare for training (enables gradient checkpointing)\nmodel = prepare_model_for_kbit_training(model)\n\n# LoRA config for QLoRA\nlora_config = LoraConfig(\n r=64, # Higher rank for 70B\n lora_alpha=128,\n lora_dropout=0.1,\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"],\n bias=\"none\",\n task_type=\"CAUSAL_LM\"\n)\n\nmodel = get_peft_model(model, lora_config)\n# 70B model now fits on single 24GB GPU!\n```\n\n## LoRA parameter selection\n\n### Rank (r) - capacity vs efficiency\n\n| Rank | Trainable Params | Memory | Quality | Use Case |\n|------|-----------------|--------|---------|----------|\n| 4 | ~3M | Minimal | Lower | Simple tasks, prototyping |\n| **8** | ~7M | Low | Good | **Recommended starting point** |\n| **16** | ~14M | Medium | Better | **General fine-tuning** |\n| 32 | ~27M | Higher | High | Complex tasks |\n| 64 | ~54M | High | Highest | Domain adaptation, 70B models |\n\n### Alpha (lora_alpha) - scaling factor\n\n```python\n# Rule of thumb: alpha = 2 * rank\nLoraConfig(r=16, lora_alpha=32) # Standard\nLoraConfig(r=16, lora_alpha=16) # Conservative (lower learning rate effect)\nLoraConfig(r=16, lora_alpha=64) # Aggressive (higher learning rate effect)\n```\n\n### Target modules by architecture\n\n```python\n# Llama / Mistral / Qwen\ntarget_modules = [\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"]\n\n# GPT-2 / GPT-Neo\ntarget_modules = [\"c_attn\", \"c_proj\", \"c_fc\"]\n\n# Falcon\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# BLOOM\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# Auto-detect all linear layers\ntarget_modules = \"all-linear\" # PEFT 0.6.0+\n```\n\n## Loading and merging adapters\n\n### Load trained adapter\n\n```python\nfrom peft import PeftModel, AutoPeftModelForCausalLM\nfrom transformers import AutoModelForCausalLM\n\n# Option 1: Load with PeftModel\nbase_model = AutoModelForCausalLM.from_pretrained(\"meta-llama/Llama-3.1-8B\")\nmodel = PeftModel.from_pretrained(base_model, \"./lora-llama-adapter\")\n\n# Option 2: Load directly (recommended)\nmodel = AutoPeftModelForCausalLM.from_pretrained(\n \"./lora-llama-adapter\",\n device_map=\"auto\"\n)\n```\n\n### Merge adapter into base model\n\n```python\n# Merge for deployment (no adapter overhead)\nmerged_model = model.merge_and_unload()\n\n# Save merged model\nmerged_model.save_pretrained(\"./llama-merged\")\ntokenizer.save_pretrained(\"./llama-merged\")\n\n# Push to Hub\nmerged_model.push_to_hub(\"username/llama-finetuned\")\n```\n\n### Multi-adapter serving\n\n```python\nfrom peft import PeftModel\n\n# Load base with first adapter\nmodel = AutoPeftModelForCausalLM.from_pretrained(\"./adapter-task1\")\n\n# Load additional adapters\nmodel.load_adapter(\"./adapter-task2\", adapter_name=\"task2\")\nmodel.load_adapter(\"./adapter-task3\", adapter_name=\"task3\")\n\n# Switch between adapters at runtime\nmodel.set_adapter(\"task1\") # Use task1 adapter\noutput1 = model.generate(**inputs)\n\nmodel.set_adapter(\"task2\") # Switch to task2\noutput2 = model.generate(**inputs)\n\n# Disable adapters (use base model)\nwith model.disable_adapter():\n base_output = model.generate(**inputs)\n```\n\n## PEFT methods comparison\n\n| Method | Trainable % | Memory | Speed | Best For |\n|--------|------------|--------|-------|----------|\n| **LoRA** | 0.1-1% | Low | Fast | General fine-tuning |\n| **QLoRA** | 0.1-1% | Very Low | Medium | Memory-constrained |\n| AdaLoRA | 0.1-1% | Low | Medium | Automatic rank selection |\n| IA3 | 0.01% | Minimal | Fastest | Few-shot adaptation |\n| Prefix Tuning | 0.1% | Low | Medium | Generation control |\n| Prompt Tuning | 0.001% | Minimal | Fast | Simple task adaptation |\n| P-Tuning v2 | 0.1% | Low | Medium | NLU tasks |\n\n### IA3 (minimal parameters)\n\n```python\nfrom peft import IA3Config\n\nia3_config = IA3Config(\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"down_proj\"],\n feedforward_modules=[\"down_proj\"]\n)\nmodel = get_peft_model(model, ia3_config)\n# Trains only 0.01% of parameters!\n```\n\n### Prefix Tuning\n\n```python\nfrom peft import PrefixTuningConfig\n\nprefix_config = PrefixTuningConfig(\n task_type=\"CAUSAL_LM\",\n num_virtual_tokens=20, # Prepended tokens\n prefix_projection=True # Use MLP projection\n)\nmodel = get_peft_model(model, prefix_config)\n```\n\n## Integration patterns\n\n### With TRL (SFTTrainer)\n\n```python\nfrom trl import SFTTrainer, SFTConfig\nfrom peft import LoraConfig\n\nlora_config = LoraConfig(r=16, lora_alpha=32, target_modules=\"all-linear\")\n\ntrainer = SFTTrainer(\n model=model,\n args=SFTConfig(output_dir=\"./output\", max_seq_length=512),\n train_dataset=dataset,\n peft_config=lora_config, # Pass LoRA config directly\n)\ntrainer.train()\n```\n\n### With Axolotl (YAML config)\n\n```yaml\n# axolotl config.yaml\nadapter: lora\nlora_r: 16\nlora_alpha: 32\nlora_dropout: 0.05\nlora_target_modules:\n - q_proj\n - v_proj\n - k_proj\n - o_proj\nlora_target_linear: true # Target all linear layers\n```\n\n### With vLLM (inference)\n\n```python\nfrom vllm import LLM\nfrom vllm.lora.request import LoRARequest\n\n# Load base model with LoRA support\nllm = LLM(model=\"meta-llama/Llama-3.1-8B\", enable_lora=True)\n\n# Serve with adapter\noutputs = llm.generate(\n prompts,\n lora_request=LoRARequest(\"adapter1\", 1, \"./lora-adapter\")\n)\n```\n\n## Performance benchmarks\n\n### Memory usage (Llama 3.1 8B)\n\n| Method | GPU Memory | Trainable Params |\n|--------|-----------|------------------|\n| Full fine-tuning | 60+ GB | 8B (100%) |\n| LoRA r=16 | 18 GB | 14M (0.17%) |\n| QLoRA r=16 | 6 GB | 14M (0.17%) |\n| IA3 | 16 GB | 800K (0.01%) |\n\n### Training speed (A100 80GB)\n\n| Method | Tokens/sec | vs Full FT |\n|--------|-----------|------------|\n| Full FT | 2,500 | 1x |\n| LoRA | 3,200 | 1.3x |\n| QLoRA | 2,100 | 0.84x |\n\n### Quality (MMLU benchmark)\n\n| Model | Full FT | LoRA | QLoRA |\n|-------|---------|------|-------|\n| Llama 2-7B | 45.3 | 44.8 | 44.1 |\n| Llama 2-13B | 54.8 | 54.2 | 53.5 |\n\n## Common issues\n\n### CUDA OOM during training\n\n```python\n# Solution 1: Enable gradient checkpointing\nmodel.gradient_checkpointing_enable()\n\n# Solution 2: Reduce batch size + increase accumulation\nTrainingArguments(\n per_device_train_batch_size=1,\n gradient_accumulation_steps=16\n)\n\n# Solution 3: Use QLoRA\nfrom transformers import BitsAndBytesConfig\nbnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\")\n```\n\n### Adapter not applying\n\n```python\n# Verify adapter is active\nprint(model.active_adapters) # Should show adapter name\n\n# Check trainable parameters\nmodel.print_trainable_parameters()\n\n# Ensure model in training mode\nmodel.train()\n```\n\n### Quality degradation\n\n```python\n# Increase rank\nLoraConfig(r=32, lora_alpha=64)\n\n# Target more modules\ntarget_modules = \"all-linear\"\n\n# Use more training data and epochs\nTrainingArguments(num_train_epochs=5)\n\n# Lower learning rate\nTrainingArguments(learning_rate=1e-4)\n```\n\n## Best practices\n\n1. **Start with r=8-16**, increase if quality insufficient\n2. **Use alpha = 2 * rank** as starting point\n3. **Target attention + MLP layers** for best quality/efficiency\n4. **Enable gradient checkpointing** for memory savings\n5. **Save adapters frequently** (small files, easy rollback)\n6. **Evaluate on held-out data** before merging\n7. **Use QLoRA for 70B+ models** on consumer hardware\n\n## References\n\n- **[Advanced Usage](references/advanced-usage.md)** - DoRA, LoftQ, rank stabilization, custom modules\n- **[Troubleshooting](references/troubleshooting.md)** - Common errors, debugging, optimization\n\n## Resources\n\n- **GitHub**: https://github.com/huggingface/peft\n- **Docs**: https://huggingface.co/docs/peft\n- **LoRA Paper**: arXiv:2106.09685\n- **QLoRA Paper**: arXiv:2305.14314\n- **Models**: https://huggingface.co/models?library=peft\n","04-mechanistic-interpretability/transformer-lens/SKILL.md":"# transformer-lens\n\nPath: `04-mechanistic-interpretability/transformer-lens/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/SKILL.md)","04-mechanistic-interpretability/saelens/SKILL.md":"# saelens\n\nPath: `04-mechanistic-interpretability/saelens/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/SKILL.md)","04-mechanistic-interpretability/pyvene/SKILL.md":"# pyvene\n\nPath: `04-mechanistic-interpretability/pyvene/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/SKILL.md)","04-mechanistic-interpretability/nnsight/SKILL.md":"# nnsight\n\nPath: `04-mechanistic-interpretability/nnsight/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/SKILL.md)","05-data-processing/nemo-curator/SKILL.md":"# nemo-curator\n\nPath: `05-data-processing/nemo-curator/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/SKILL.md)","05-data-processing/ray-data/SKILL.md":"# ray-data\n\nPath: `05-data-processing/ray-data/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/SKILL.md)","06-post-training/verl/SKILL.md":"# verl\n\nPath: `06-post-training/verl/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/SKILL.md)","06-post-training/trl-fine-tuning/SKILL.md":"# trl-fine-tuning\n\nPath: `06-post-training/trl-fine-tuning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/SKILL.md)","06-post-training/torchforge/SKILL.md":"# torchforge\n\nPath: `06-post-training/torchforge/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/SKILL.md)","06-post-training/slime/SKILL.md":"# slime\n\nPath: `06-post-training/slime/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/SKILL.md)","06-post-training/simpo/SKILL.md":"# simpo\n\nPath: `06-post-training/simpo/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/SKILL.md)","06-post-training/openrlhf/SKILL.md":"# openrlhf\n\nPath: `06-post-training/openrlhf/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/SKILL.md)","06-post-training/miles/SKILL.md":"# miles\n\nPath: `06-post-training/miles/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/SKILL.md)","06-post-training/grpo-rl-training/SKILL.md":"# grpo-rl-training\n\nPath: `06-post-training/grpo-rl-training/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/SKILL.md)","07-safety-alignment/constitutional-ai/SKILL.md":"# constitutional-ai\n\nPath: `07-safety-alignment/constitutional-ai/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/SKILL.md)","07-safety-alignment/llamaguard/SKILL.md":"# llamaguard\n\nPath: `07-safety-alignment/llamaguard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/SKILL.md)","07-safety-alignment/nemo-guardrails/SKILL.md":"# nemo-guardrails\n\nPath: `07-safety-alignment/nemo-guardrails/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/SKILL.md)","07-safety-alignment/prompt-guard/SKILL.md":"# prompt-guard\n\nPath: `07-safety-alignment/prompt-guard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/SKILL.md)","08-distributed-training/ray-train/SKILL.md":"# ray-train\n\nPath: `08-distributed-training/ray-train/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/SKILL.md)","08-distributed-training/pytorch-lightning/SKILL.md":"# pytorch-lightning\n\nPath: `08-distributed-training/pytorch-lightning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/SKILL.md)","08-distributed-training/pytorch-fsdp2/SKILL.md":"# pytorch-fsdp2\n\nPath: `08-distributed-training/pytorch-fsdp2/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/SKILL.md)","08-distributed-training/megatron-core/SKILL.md":"# megatron-core\n\nPath: `08-distributed-training/megatron-core/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/SKILL.md)","08-distributed-training/deepspeed/SKILL.md":"# deepspeed\n\nPath: `08-distributed-training/deepspeed/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/SKILL.md)","08-distributed-training/accelerate/SKILL.md":"# accelerate\n\nPath: `08-distributed-training/accelerate/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/SKILL.md)","09-infrastructure/lambda-labs/SKILL.md":"# lambda-labs\n\nPath: `09-infrastructure/lambda-labs/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/SKILL.md)","09-infrastructure/modal/SKILL.md":"# modal\n\nPath: `09-infrastructure/modal/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/SKILL.md)","09-infrastructure/skypilot/SKILL.md":"# skypilot\n\nPath: `09-infrastructure/skypilot/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/SKILL.md)","10-optimization/awq/SKILL.md":"# awq\n\nPath: `10-optimization/awq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/SKILL.md)","10-optimization/bitsandbytes/SKILL.md":"# bitsandbytes\n\nPath: `10-optimization/bitsandbytes/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/SKILL.md)","10-optimization/flash-attention/SKILL.md":"# flash-attention\n\nPath: `10-optimization/flash-attention/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/SKILL.md)","10-optimization/gguf/SKILL.md":"# gguf\n\nPath: `10-optimization/gguf/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/SKILL.md)","10-optimization/gptq/SKILL.md":"# gptq\n\nPath: `10-optimization/gptq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/SKILL.md)","10-optimization/hqq/SKILL.md":"# hqq\n\nPath: `10-optimization/hqq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/SKILL.md)","10-optimization/ml-training-recipes/SKILL.md":"# ml-training-recipes\n\nPath: `10-optimization/ml-training-recipes/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/SKILL.md)","11-evaluation/bigcode-evaluation-harness/SKILL.md":"# bigcode-evaluation-harness\n\nPath: `11-evaluation/bigcode-evaluation-harness/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/SKILL.md)","11-evaluation/lm-evaluation-harness/SKILL.md":"# lm-evaluation-harness\n\nPath: `11-evaluation/lm-evaluation-harness/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/SKILL.md)","11-evaluation/nemo-evaluator/SKILL.md":"# nemo-evaluator\n\nPath: `11-evaluation/nemo-evaluator/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/SKILL.md)","12-inference-serving/llama-cpp/SKILL.md":"# llama-cpp\n\nPath: `12-inference-serving/llama-cpp/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/SKILL.md)","12-inference-serving/sglang/SKILL.md":"# sglang\n\nPath: `12-inference-serving/sglang/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/SKILL.md)","12-inference-serving/tensorrt-llm/SKILL.md":"# tensorrt-llm\n\nPath: `12-inference-serving/tensorrt-llm/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/SKILL.md)","12-inference-serving/vllm/SKILL.md":"# vllm\n\nPath: `12-inference-serving/vllm/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/SKILL.md)","13-mlops/mlflow/SKILL.md":"# mlflow\n\nPath: `13-mlops/mlflow/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/SKILL.md)","13-mlops/swanlab/SKILL.md":"# swanlab\n\nPath: `13-mlops/swanlab/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/SKILL.md)","13-mlops/tensorboard/SKILL.md":"# tensorboard\n\nPath: `13-mlops/tensorboard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/SKILL.md)","13-mlops/weights-and-biases/SKILL.md":"# weights-and-biases\n\nPath: `13-mlops/weights-and-biases/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/SKILL.md)","14-agents/a-evolve/SKILL.md":"# a-evolve\n\nPath: `14-agents/a-evolve/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/SKILL.md)","14-agents/autogpt/SKILL.md":"# autogpt\n\nPath: `14-agents/autogpt/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/SKILL.md)","14-agents/crewai/SKILL.md":"# crewai\n\nPath: `14-agents/crewai/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/SKILL.md)","14-agents/langchain/SKILL.md":"# langchain\n\nPath: `14-agents/langchain/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/SKILL.md)","14-agents/llamaindex/SKILL.md":"# llamaindex\n\nPath: `14-agents/llamaindex/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/SKILL.md)","15-rag/chroma/SKILL.md":"# chroma\n\nPath: `15-rag/chroma/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/SKILL.md)","15-rag/faiss/SKILL.md":"# faiss\n\nPath: `15-rag/faiss/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/SKILL.md)","15-rag/pinecone/SKILL.md":"# pinecone\n\nPath: `15-rag/pinecone/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/SKILL.md)","15-rag/qdrant/SKILL.md":"# qdrant\n\nPath: `15-rag/qdrant/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/SKILL.md)","15-rag/sentence-transformers/SKILL.md":"# sentence-transformers\n\nPath: `15-rag/sentence-transformers/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/SKILL.md)","16-prompt-engineering/dspy/SKILL.md":"# dspy\n\nPath: `16-prompt-engineering/dspy/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/SKILL.md)","16-prompt-engineering/guidance/SKILL.md":"# guidance\n\nPath: `16-prompt-engineering/guidance/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/SKILL.md)","16-prompt-engineering/instructor/SKILL.md":"# instructor\n\nPath: `16-prompt-engineering/instructor/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/SKILL.md)","16-prompt-engineering/outlines/SKILL.md":"# outlines\n\nPath: `16-prompt-engineering/outlines/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/SKILL.md)","17-observability/langsmith/SKILL.md":"# langsmith\n\nPath: `17-observability/langsmith/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/SKILL.md)","17-observability/phoenix/SKILL.md":"# phoenix\n\nPath: `17-observability/phoenix/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/SKILL.md)","18-multimodal/audiocraft/SKILL.md":"# audiocraft\n\nPath: `18-multimodal/audiocraft/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/SKILL.md)","18-multimodal/blip-2/SKILL.md":"# blip-2\n\nPath: `18-multimodal/blip-2/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/SKILL.md)","18-multimodal/clip/SKILL.md":"# clip\n\nPath: `18-multimodal/clip/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/SKILL.md)","18-multimodal/cosmos-policy/SKILL.md":"# cosmos-policy\n\nPath: `18-multimodal/cosmos-policy/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/SKILL.md)","18-multimodal/llava/SKILL.md":"# llava\n\nPath: `18-multimodal/llava/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/SKILL.md)","18-multimodal/openpi/SKILL.md":"# openpi\n\nPath: `18-multimodal/openpi/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/SKILL.md)","18-multimodal/openvla-oft/SKILL.md":"# openvla-oft\n\nPath: `18-multimodal/openvla-oft/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/SKILL.md)","18-multimodal/segment-anything/SKILL.md":"# segment-anything\n\nPath: `18-multimodal/segment-anything/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/SKILL.md)","18-multimodal/stable-diffusion/SKILL.md":"# stable-diffusion\n\nPath: `18-multimodal/stable-diffusion/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/SKILL.md)","18-multimodal/whisper/SKILL.md":"# whisper\n\nPath: `18-multimodal/whisper/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/SKILL.md)","19-emerging-techniques/knowledge-distillation/SKILL.md":"# knowledge-distillation\n\nPath: `19-emerging-techniques/knowledge-distillation/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/SKILL.md)","19-emerging-techniques/long-context/SKILL.md":"# long-context\n\nPath: `19-emerging-techniques/long-context/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/SKILL.md)","19-emerging-techniques/model-merging/SKILL.md":"# model-merging\n\nPath: `19-emerging-techniques/model-merging/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/SKILL.md)","19-emerging-techniques/model-pruning/SKILL.md":"# model-pruning\n\nPath: `19-emerging-techniques/model-pruning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/SKILL.md)","19-emerging-techniques/moe-training/SKILL.md":"# moe-training\n\nPath: `19-emerging-techniques/moe-training/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/SKILL.md)","19-emerging-techniques/speculative-decoding/SKILL.md":"# speculative-decoding\n\nPath: `19-emerging-techniques/speculative-decoding/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/SKILL.md)","20-ml-paper-writing/systems-paper-writing/SKILL.md":"# systems-paper-writing\n\nPath: `20-ml-paper-writing/systems-paper-writing/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/SKILL.md)","20-ml-paper-writing/presenting-conference-talks/SKILL.md":"# presenting-conference-talks\n\nPath: `20-ml-paper-writing/presenting-conference-talks/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/SKILL.md)","20-ml-paper-writing/ml-paper-writing/SKILL.md":"# ml-paper-writing\n\nPath: `20-ml-paper-writing/ml-paper-writing/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/SKILL.md)","20-ml-paper-writing/academic-plotting/SKILL.md":"# academic-plotting\n\nPath: `20-ml-paper-writing/academic-plotting/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/SKILL.md)","21-research-ideation/creative-thinking-for-research/SKILL.md":"# creative-thinking-for-research\n\nPath: `21-research-ideation/creative-thinking-for-research/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/SKILL.md)","21-research-ideation/brainstorming-research-ideas/SKILL.md":"# brainstorming-research-ideas\n\nPath: `21-research-ideation/brainstorming-research-ideas/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/SKILL.md)","22-agent-native-research-artifact/rigor-reviewer/SKILL.md":"# rigor-reviewer\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/SKILL.md)","22-agent-native-research-artifact/research-manager/SKILL.md":"# research-manager\n\nPath: `22-agent-native-research-artifact/research-manager/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/SKILL.md)","22-agent-native-research-artifact/compiler/SKILL.md":"# compiler\n\nPath: `22-agent-native-research-artifact/compiler/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/SKILL.md)","CLAUDE.md":"# Claude Agent Guidelines & System Prompt\n\nPath: `CLAUDE.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/CLAUDE.md)",".claude-plugin/marketplace.json":"# Claude Plugin Marketplace Catalog\n\nPath: `.claude-plugin/marketplace.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/.claude-plugin/marketplace.json)","0-autoresearch-skill/README.md":"# readme.md\n\nPath: `0-autoresearch-skill/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/README.md)","0-autoresearch-skill/references/agent-continuity.md":"# References Skill\n\nPath: `0-autoresearch-skill/references/agent-continuity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/agent-continuity.md)","0-autoresearch-skill/references/progress-reporting.md":"# References Skill\n\nPath: `0-autoresearch-skill/references/progress-reporting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/progress-reporting.md)","0-autoresearch-skill/references/skill-routing.md":"# References Skill\n\nPath: `0-autoresearch-skill/references/skill-routing.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/skill-routing.md)","packages/ai-research-skills/package.json":"# Ai-research-skills Skill\n\nPath: `packages/ai-research-skills/package.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package.json)","packages/ai-research-skills/package-lock.json":"# Ai-research-skills Skill\n\nPath: `packages/ai-research-skills/package-lock.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package-lock.json)","packages/ai-research-skills/README.md":"# ai-research-skills Documentation\n\nPath: `packages/ai-research-skills/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/README.md)","01-model-architecture/litgpt/README.md":"# readme.md\n\nPath: `01-model-architecture/litgpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/README.md)","01-model-architecture/mamba/README.md":"# readme.md\n\nPath: `01-model-architecture/mamba/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/README.md)","01-model-architecture/nanogpt/README.md":"# readme.md\n\nPath: `01-model-architecture/nanogpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/README.md)","01-model-architecture/rwkv/README.md":"# readme.md\n\nPath: `01-model-architecture/rwkv/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/README.md)","01-model-architecture/torchtitan/README.md":"# readme.md\n\nPath: `01-model-architecture/torchtitan/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/README.md)","02-tokenization/huggingface-tokenizers/README.md":"# readme.md\n\nPath: `02-tokenization/huggingface-tokenizers/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/README.md)","02-tokenization/sentencepiece/README.md":"# readme.md\n\nPath: `02-tokenization/sentencepiece/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/README.md)","03-fine-tuning/axolotl/README.md":"# readme.md\n\nPath: `03-fine-tuning/axolotl/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/README.md)","03-fine-tuning/llama-factory/README.md":"# readme.md\n\nPath: `03-fine-tuning/llama-factory/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/README.md)","03-fine-tuning/peft/README.md":"# readme.md\n\nPath: `03-fine-tuning/peft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/README.md)","03-fine-tuning/unsloth/README.md":"# readme.md\n\nPath: `03-fine-tuning/unsloth/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/README.md)","04-mechanistic-interpretability/nnsight/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/nnsight/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/README.md)","04-mechanistic-interpretability/pyvene/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/pyvene/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/README.md)","04-mechanistic-interpretability/saelens/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/saelens/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/README.md)","04-mechanistic-interpretability/transformer-lens/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/transformer-lens/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/README.md)","05-data-processing/nemo-curator/README.md":"# readme.md\n\nPath: `05-data-processing/nemo-curator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/README.md)","05-data-processing/ray-data/README.md":"# readme.md\n\nPath: `05-data-processing/ray-data/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/README.md)","06-post-training/grpo-rl-training/README.md":"# readme.md\n\nPath: `06-post-training/grpo-rl-training/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/README.md)","06-post-training/miles/README.md":"# readme.md\n\nPath: `06-post-training/miles/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/README.md)","06-post-training/openrlhf/README.md":"# readme.md\n\nPath: `06-post-training/openrlhf/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/README.md)","06-post-training/simpo/README.md":"# readme.md\n\nPath: `06-post-training/simpo/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/README.md)","06-post-training/slime/README.md":"# readme.md\n\nPath: `06-post-training/slime/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/README.md)","06-post-training/torchforge/README.md":"# readme.md\n\nPath: `06-post-training/torchforge/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/README.md)","06-post-training/trl-fine-tuning/README.md":"# readme.md\n\nPath: `06-post-training/trl-fine-tuning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/README.md)","06-post-training/verl/README.md":"# readme.md\n\nPath: `06-post-training/verl/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/README.md)","07-safety-alignment/constitutional-ai/README.md":"# readme.md\n\nPath: `07-safety-alignment/constitutional-ai/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/README.md)","07-safety-alignment/llamaguard/README.md":"# readme.md\n\nPath: `07-safety-alignment/llamaguard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/README.md)","07-safety-alignment/nemo-guardrails/README.md":"# readme.md\n\nPath: `07-safety-alignment/nemo-guardrails/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/README.md)","07-safety-alignment/prompt-guard/README.md":"# readme.md\n\nPath: `07-safety-alignment/prompt-guard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/README.md)","08-distributed-training/accelerate/README.md":"# readme.md\n\nPath: `08-distributed-training/accelerate/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/README.md)","08-distributed-training/deepspeed/README.md":"# readme.md\n\nPath: `08-distributed-training/deepspeed/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/README.md)","08-distributed-training/megatron-core/README.md":"# readme.md\n\nPath: `08-distributed-training/megatron-core/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/README.md)","08-distributed-training/pytorch-fsdp2/README.md":"# readme.md\n\nPath: `08-distributed-training/pytorch-fsdp2/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/README.md)","08-distributed-training/pytorch-lightning/README.md":"# readme.md\n\nPath: `08-distributed-training/pytorch-lightning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/README.md)","08-distributed-training/ray-train/README.md":"# readme.md\n\nPath: `08-distributed-training/ray-train/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/README.md)","09-infrastructure/lambda-labs/README.md":"# readme.md\n\nPath: `09-infrastructure/lambda-labs/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/README.md)","09-infrastructure/modal/README.md":"# readme.md\n\nPath: `09-infrastructure/modal/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/README.md)","09-infrastructure/skypilot/README.md":"# readme.md\n\nPath: `09-infrastructure/skypilot/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/README.md)","10-optimization/awq/README.md":"# readme.md\n\nPath: `10-optimization/awq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/README.md)","10-optimization/bitsandbytes/README.md":"# readme.md\n\nPath: `10-optimization/bitsandbytes/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/README.md)","10-optimization/flash-attention/README.md":"# readme.md\n\nPath: `10-optimization/flash-attention/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/README.md)","10-optimization/gguf/README.md":"# readme.md\n\nPath: `10-optimization/gguf/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/README.md)","10-optimization/gptq/README.md":"# readme.md\n\nPath: `10-optimization/gptq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/README.md)","10-optimization/hqq/README.md":"# readme.md\n\nPath: `10-optimization/hqq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/README.md)","10-optimization/ml-training-recipes/README.md":"# readme.md\n\nPath: `10-optimization/ml-training-recipes/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/README.md)","11-evaluation/bigcode-evaluation-harness/README.md":"# readme.md\n\nPath: `11-evaluation/bigcode-evaluation-harness/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/README.md)","11-evaluation/lm-evaluation-harness/README.md":"# readme.md\n\nPath: `11-evaluation/lm-evaluation-harness/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/README.md)","11-evaluation/nemo-evaluator/README.md":"# readme.md\n\nPath: `11-evaluation/nemo-evaluator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/README.md)","12-inference-serving/llama-cpp/README.md":"# readme.md\n\nPath: `12-inference-serving/llama-cpp/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/README.md)","12-inference-serving/sglang/README.md":"# readme.md\n\nPath: `12-inference-serving/sglang/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/README.md)","12-inference-serving/tensorrt-llm/README.md":"# readme.md\n\nPath: `12-inference-serving/tensorrt-llm/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/README.md)","12-inference-serving/vllm/README.md":"# readme.md\n\nPath: `12-inference-serving/vllm/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/README.md)","13-mlops/mlflow/README.md":"# readme.md\n\nPath: `13-mlops/mlflow/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/README.md)","13-mlops/tensorboard/README.md":"# readme.md\n\nPath: `13-mlops/tensorboard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/README.md)","13-mlops/weights-and-biases/README.md":"# readme.md\n\nPath: `13-mlops/weights-and-biases/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/README.md)","14-agents/autogpt/README.md":"# readme.md\n\nPath: `14-agents/autogpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/README.md)","14-agents/crewai/README.md":"# readme.md\n\nPath: `14-agents/crewai/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/README.md)","14-agents/langchain/README.md":"# readme.md\n\nPath: `14-agents/langchain/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/README.md)","14-agents/llamaindex/README.md":"# readme.md\n\nPath: `14-agents/llamaindex/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/README.md)","15-rag/chroma/README.md":"# readme.md\n\nPath: `15-rag/chroma/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/README.md)","15-rag/faiss/README.md":"# readme.md\n\nPath: `15-rag/faiss/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/README.md)","15-rag/pinecone/README.md":"# readme.md\n\nPath: `15-rag/pinecone/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/README.md)","15-rag/qdrant/README.md":"# readme.md\n\nPath: `15-rag/qdrant/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/README.md)","15-rag/sentence-transformers/README.md":"# readme.md\n\nPath: `15-rag/sentence-transformers/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/README.md)","16-prompt-engineering/dspy/README.md":"# readme.md\n\nPath: `16-prompt-engineering/dspy/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/README.md)","16-prompt-engineering/guidance/README.md":"# readme.md\n\nPath: `16-prompt-engineering/guidance/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/README.md)","16-prompt-engineering/instructor/README.md":"# readme.md\n\nPath: `16-prompt-engineering/instructor/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/README.md)","16-prompt-engineering/outlines/README.md":"# readme.md\n\nPath: `16-prompt-engineering/outlines/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/README.md)","17-observability/langsmith/README.md":"# readme.md\n\nPath: `17-observability/langsmith/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/README.md)","17-observability/phoenix/README.md":"# readme.md\n\nPath: `17-observability/phoenix/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/README.md)","18-multimodal/audiocraft/README.md":"# readme.md\n\nPath: `18-multimodal/audiocraft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/README.md)","18-multimodal/blip-2/README.md":"# readme.md\n\nPath: `18-multimodal/blip-2/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/README.md)","18-multimodal/clip/README.md":"# readme.md\n\nPath: `18-multimodal/clip/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/README.md)","18-multimodal/cosmos-policy/README.md":"# readme.md\n\nPath: `18-multimodal/cosmos-policy/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/README.md)","18-multimodal/llava/README.md":"# readme.md\n\nPath: `18-multimodal/llava/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/README.md)","18-multimodal/openpi/README.md":"# readme.md\n\nPath: `18-multimodal/openpi/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/README.md)","18-multimodal/openvla-oft/README.md":"# readme.md\n\nPath: `18-multimodal/openvla-oft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/README.md)","18-multimodal/segment-anything/README.md":"# readme.md\n\nPath: `18-multimodal/segment-anything/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/README.md)","18-multimodal/stable-diffusion/README.md":"# readme.md\n\nPath: `18-multimodal/stable-diffusion/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/README.md)","18-multimodal/whisper/README.md":"# readme.md\n\nPath: `18-multimodal/whisper/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/README.md)","19-emerging-techniques/knowledge-distillation/README.md":"# readme.md\n\nPath: `19-emerging-techniques/knowledge-distillation/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/README.md)","19-emerging-techniques/long-context/README.md":"# readme.md\n\nPath: `19-emerging-techniques/long-context/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/README.md)","19-emerging-techniques/model-merging/README.md":"# readme.md\n\nPath: `19-emerging-techniques/model-merging/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/README.md)","19-emerging-techniques/model-pruning/README.md":"# readme.md\n\nPath: `19-emerging-techniques/model-pruning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/README.md)","19-emerging-techniques/moe-training/README.md":"# readme.md\n\nPath: `19-emerging-techniques/moe-training/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/README.md)","19-emerging-techniques/speculative-decoding/README.md":"# readme.md\n\nPath: `19-emerging-techniques/speculative-decoding/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/README.md)","20-ml-paper-writing/ml-paper-writing/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/ml-paper-writing/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/README.md)","20-ml-paper-writing/academic-plotting/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/academic-plotting/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/README.md)","20-ml-paper-writing/systems-paper-writing/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/systems-paper-writing/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/README.md)","20-ml-paper-writing/presenting-conference-talks/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/presenting-conference-talks/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/README.md)","21-research-ideation/brainstorming-research-ideas/README.md":"# readme.md\n\nPath: `21-research-ideation/brainstorming-research-ideas/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/README.md)","21-research-ideation/creative-thinking-for-research/README.md":"# readme.md\n\nPath: `21-research-ideation/creative-thinking-for-research/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/README.md)","22-agent-native-research-artifact/compiler/README.md":"# readme.md\n\nPath: `22-agent-native-research-artifact/compiler/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/README.md)","22-agent-native-research-artifact/research-manager/README.md":"# readme.md\n\nPath: `22-agent-native-research-artifact/research-manager/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/README.md)","22-agent-native-research-artifact/rigor-reviewer/README.md":"# readme.md\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/README.md)","01-model-architecture/litgpt/references/custom-models.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/custom-models.md)","01-model-architecture/litgpt/references/distributed-training.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/distributed-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/distributed-training.md)","01-model-architecture/litgpt/references/supported-models.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/supported-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/supported-models.md)","01-model-architecture/litgpt/references/training-recipes.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/training-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/training-recipes.md)","01-model-architecture/mamba/references/architecture-details.md":"# References Skill\n\nPath: `01-model-architecture/mamba/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/architecture-details.md)","01-model-architecture/mamba/references/benchmarks.md":"# References Skill\n\nPath: `01-model-architecture/mamba/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/benchmarks.md)","01-model-architecture/mamba/references/training-guide.md":"# References Skill\n\nPath: `01-model-architecture/mamba/references/training-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/training-guide.md)","01-model-architecture/nanogpt/references/architecture.md":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/architecture.md)","01-model-architecture/nanogpt/references/data.md":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/data.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/data.md)","01-model-architecture/nanogpt/references/training.md":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/training.md)","01-model-architecture/rwkv/references/architecture-details.md":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/architecture-details.md)","01-model-architecture/rwkv/references/rwkv7.md":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/rwkv7.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/rwkv7.md)","01-model-architecture/rwkv/references/state-management.md":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/state-management.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/state-management.md)","01-model-architecture/torchtitan/references/checkpoint.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/checkpoint.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/checkpoint.md)","01-model-architecture/torchtitan/references/custom-models.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/custom-models.md)","01-model-architecture/torchtitan/references/float8.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/float8.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/float8.md)","01-model-architecture/torchtitan/references/fsdp.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/fsdp.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/fsdp.md)","02-tokenization/huggingface-tokenizers/references/algorithms.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/algorithms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/algorithms.md)","02-tokenization/huggingface-tokenizers/references/integration.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/integration.md)","02-tokenization/huggingface-tokenizers/references/pipeline.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/pipeline.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/pipeline.md)","02-tokenization/huggingface-tokenizers/references/training.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/training.md)","02-tokenization/sentencepiece/references/algorithms.md":"# References Skill\n\nPath: `02-tokenization/sentencepiece/references/algorithms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/algorithms.md)","02-tokenization/sentencepiece/references/training.md":"# References Skill\n\nPath: `02-tokenization/sentencepiece/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/training.md)","03-fine-tuning/unsloth/references/index.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/index.md)","03-fine-tuning/unsloth/references/llms.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms.md)","03-fine-tuning/unsloth/references/llms-full.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms-full.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-full.md)","03-fine-tuning/unsloth/references/llms-txt.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms-txt.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-txt.md)","03-fine-tuning/peft/references/advanced-usage.md":"# References Skill\n\nPath: `03-fine-tuning/peft/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/advanced-usage.md)","03-fine-tuning/peft/references/troubleshooting.md":"# References Skill\n\nPath: `03-fine-tuning/peft/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/troubleshooting.md)","03-fine-tuning/llama-factory/references/_images.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/_images.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/_images.md)","03-fine-tuning/llama-factory/references/advanced.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/advanced.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/advanced.md)","03-fine-tuning/llama-factory/references/getting_started.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/getting_started.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/getting_started.md)","03-fine-tuning/llama-factory/references/index.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/index.md)","03-fine-tuning/llama-factory/references/other.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/other.md)","03-fine-tuning/axolotl/references/api.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/api.md)","03-fine-tuning/axolotl/references/dataset-formats.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/dataset-formats.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/dataset-formats.md)","03-fine-tuning/axolotl/references/index.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/index.md)","03-fine-tuning/axolotl/references/other.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/other.md)","04-mechanistic-interpretability/transformer-lens/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/api.md)","04-mechanistic-interpretability/transformer-lens/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/README.md)","04-mechanistic-interpretability/transformer-lens/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/tutorials.md)","04-mechanistic-interpretability/saelens/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/api.md)","04-mechanistic-interpretability/saelens/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/README.md)","04-mechanistic-interpretability/saelens/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/tutorials.md)","04-mechanistic-interpretability/pyvene/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/api.md)","04-mechanistic-interpretability/pyvene/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/README.md)","04-mechanistic-interpretability/pyvene/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/tutorials.md)","04-mechanistic-interpretability/nnsight/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/api.md)","04-mechanistic-interpretability/nnsight/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/README.md)","04-mechanistic-interpretability/nnsight/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/tutorials.md)","05-data-processing/nemo-curator/references/deduplication.md":"# References Skill\n\nPath: `05-data-processing/nemo-curator/references/deduplication.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/deduplication.md)","05-data-processing/nemo-curator/references/filtering.md":"# References Skill\n\nPath: `05-data-processing/nemo-curator/references/filtering.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/filtering.md)","05-data-processing/ray-data/references/integration.md":"# References Skill\n\nPath: `05-data-processing/ray-data/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/integration.md)","05-data-processing/ray-data/references/transformations.md":"# References Skill\n\nPath: `05-data-processing/ray-data/references/transformations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/transformations.md)","06-post-training/verl/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/verl/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/api-reference.md)","06-post-training/verl/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/verl/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/troubleshooting.md)","06-post-training/trl-fine-tuning/references/dpo-variants.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/dpo-variants.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/dpo-variants.md)","06-post-training/trl-fine-tuning/references/online-rl.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/online-rl.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/online-rl.md)","06-post-training/trl-fine-tuning/references/reward-modeling.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/reward-modeling.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/reward-modeling.md)","06-post-training/trl-fine-tuning/references/sft-training.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/sft-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/sft-training.md)","06-post-training/torchforge/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/torchforge/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/api-reference.md)","06-post-training/torchforge/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/torchforge/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/troubleshooting.md)","06-post-training/slime/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/slime/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/api-reference.md)","06-post-training/slime/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/slime/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/troubleshooting.md)","06-post-training/simpo/references/datasets.md":"# References Skill\n\nPath: `06-post-training/simpo/references/datasets.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/datasets.md)","06-post-training/simpo/references/hyperparameters.md":"# References Skill\n\nPath: `06-post-training/simpo/references/hyperparameters.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/hyperparameters.md)","06-post-training/simpo/references/loss-functions.md":"# References Skill\n\nPath: `06-post-training/simpo/references/loss-functions.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/loss-functions.md)","06-post-training/openrlhf/references/algorithm-comparison.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/algorithm-comparison.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/algorithm-comparison.md)","06-post-training/openrlhf/references/custom-rewards.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/custom-rewards.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/custom-rewards.md)","06-post-training/openrlhf/references/hybrid-engine.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/hybrid-engine.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/hybrid-engine.md)","06-post-training/openrlhf/references/multi-node-training.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/multi-node-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/multi-node-training.md)","06-post-training/miles/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/miles/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/api-reference.md)","06-post-training/miles/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/miles/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/troubleshooting.md)","08-distributed-training/ray-train/references/multi-node.md":"# References Skill\n\nPath: `08-distributed-training/ray-train/references/multi-node.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/references/multi-node.md)","08-distributed-training/pytorch-lightning/references/callbacks.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/callbacks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/callbacks.md)","08-distributed-training/pytorch-lightning/references/distributed.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/distributed.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/distributed.md)","08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md)","08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md)","08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md)","08-distributed-training/megatron-core/references/benchmarks.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/benchmarks.md)","08-distributed-training/megatron-core/references/parallelism-guide.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/parallelism-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/parallelism-guide.md)","08-distributed-training/megatron-core/references/production-examples.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/production-examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/production-examples.md)","08-distributed-training/megatron-core/references/training-recipes.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/training-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/training-recipes.md)","08-distributed-training/deepspeed/references/08.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/08.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/08.md)","08-distributed-training/deepspeed/references/09.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/09.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/09.md)","08-distributed-training/deepspeed/references/2020.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/2020.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2020.md)","08-distributed-training/deepspeed/references/2023.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/2023.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2023.md)","08-distributed-training/deepspeed/references/assets.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/assets.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/assets.md)","08-distributed-training/deepspeed/references/index.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/index.md)","08-distributed-training/deepspeed/references/mii.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/mii.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/mii.md)","08-distributed-training/deepspeed/references/other.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/other.md)","08-distributed-training/deepspeed/references/tutorials.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/tutorials.md)","08-distributed-training/accelerate/references/custom-plugins.md":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/custom-plugins.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/custom-plugins.md)","08-distributed-training/accelerate/references/megatron-integration.md":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/megatron-integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/megatron-integration.md)","08-distributed-training/accelerate/references/performance.md":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/performance.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/performance.md)","09-infrastructure/lambda-labs/references/advanced-usage.md":"# References Skill\n\nPath: `09-infrastructure/lambda-labs/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/advanced-usage.md)","09-infrastructure/lambda-labs/references/troubleshooting.md":"# References Skill\n\nPath: `09-infrastructure/lambda-labs/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/troubleshooting.md)","09-infrastructure/modal/references/advanced-usage.md":"# References Skill\n\nPath: `09-infrastructure/modal/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/advanced-usage.md)","09-infrastructure/modal/references/troubleshooting.md":"# References Skill\n\nPath: `09-infrastructure/modal/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/troubleshooting.md)","09-infrastructure/skypilot/references/advanced-usage.md":"# References Skill\n\nPath: `09-infrastructure/skypilot/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/advanced-usage.md)","09-infrastructure/skypilot/references/troubleshooting.md":"# References Skill\n\nPath: `09-infrastructure/skypilot/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/troubleshooting.md)","10-optimization/awq/references/advanced-usage.md":"# References Skill\n\nPath: `10-optimization/awq/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/advanced-usage.md)","10-optimization/awq/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/awq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/troubleshooting.md)","10-optimization/bitsandbytes/references/memory-optimization.md":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/memory-optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/memory-optimization.md)","10-optimization/bitsandbytes/references/qlora-training.md":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/qlora-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/qlora-training.md)","10-optimization/bitsandbytes/references/quantization-formats.md":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/quantization-formats.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/quantization-formats.md)","10-optimization/flash-attention/references/benchmarks.md":"# References Skill\n\nPath: `10-optimization/flash-attention/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/benchmarks.md)","10-optimization/flash-attention/references/transformers-integration.md":"# References Skill\n\nPath: `10-optimization/flash-attention/references/transformers-integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/transformers-integration.md)","10-optimization/gguf/references/advanced-usage.md":"# References Skill\n\nPath: `10-optimization/gguf/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/advanced-usage.md)","10-optimization/gguf/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/gguf/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/troubleshooting.md)","10-optimization/gptq/references/calibration.md":"# References Skill\n\nPath: `10-optimization/gptq/references/calibration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/calibration.md)","10-optimization/gptq/references/integration.md":"# References Skill\n\nPath: `10-optimization/gptq/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/integration.md)","10-optimization/gptq/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/gptq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/troubleshooting.md)","10-optimization/hqq/references/advanced-usage.md":"# References Skill\n\nPath: `10-optimization/hqq/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/advanced-usage.md)","10-optimization/hqq/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/hqq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/troubleshooting.md)","10-optimization/ml-training-recipes/references/architecture.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/architecture.md)","10-optimization/ml-training-recipes/references/biomedical.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/biomedical.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/biomedical.md)","10-optimization/ml-training-recipes/references/domain-specific.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/domain-specific.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/domain-specific.md)","10-optimization/ml-training-recipes/references/experiment-loop.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/experiment-loop.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/experiment-loop.md)","10-optimization/ml-training-recipes/references/optimizers.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/optimizers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/optimizers.md)","10-optimization/ml-training-recipes/references/scaling-and-selection.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/scaling-and-selection.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/scaling-and-selection.md)","11-evaluation/bigcode-evaluation-harness/references/benchmarks.md":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/benchmarks.md)","11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md)","11-evaluation/bigcode-evaluation-harness/references/issues.md":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/issues.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/issues.md)","11-evaluation/lm-evaluation-harness/references/api-evaluation.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/api-evaluation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/api-evaluation.md)","11-evaluation/lm-evaluation-harness/references/benchmark-guide.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/benchmark-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/benchmark-guide.md)","11-evaluation/lm-evaluation-harness/references/custom-tasks.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/custom-tasks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/custom-tasks.md)","11-evaluation/lm-evaluation-harness/references/distributed-eval.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/distributed-eval.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/distributed-eval.md)","11-evaluation/nemo-evaluator/references/adapter-system.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/adapter-system.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/adapter-system.md)","11-evaluation/nemo-evaluator/references/configuration.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/configuration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/configuration.md)","11-evaluation/nemo-evaluator/references/custom-benchmarks.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/custom-benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/custom-benchmarks.md)","11-evaluation/nemo-evaluator/references/execution-backends.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/execution-backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/execution-backends.md)","12-inference-serving/llama-cpp/references/optimization.md":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/optimization.md)","12-inference-serving/llama-cpp/references/quantization.md":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/quantization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/quantization.md)","12-inference-serving/llama-cpp/references/server.md":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/server.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/server.md)","12-inference-serving/sglang/references/deployment.md":"# References Skill\n\nPath: `12-inference-serving/sglang/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/deployment.md)","12-inference-serving/sglang/references/radix-attention.md":"# References Skill\n\nPath: `12-inference-serving/sglang/references/radix-attention.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/radix-attention.md)","12-inference-serving/sglang/references/structured-generation.md":"# References Skill\n\nPath: `12-inference-serving/sglang/references/structured-generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/structured-generation.md)","12-inference-serving/tensorrt-llm/references/multi-gpu.md":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/multi-gpu.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/multi-gpu.md)","12-inference-serving/tensorrt-llm/references/optimization.md":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/optimization.md)","12-inference-serving/tensorrt-llm/references/serving.md":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/serving.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/serving.md)","12-inference-serving/vllm/references/optimization.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/optimization.md)","12-inference-serving/vllm/references/quantization.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/quantization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/quantization.md)","12-inference-serving/vllm/references/server-deployment.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/server-deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/server-deployment.md)","12-inference-serving/vllm/references/troubleshooting.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/troubleshooting.md)","13-mlops/mlflow/references/deployment.md":"# References Skill\n\nPath: `13-mlops/mlflow/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/deployment.md)","13-mlops/mlflow/references/model-registry.md":"# References Skill\n\nPath: `13-mlops/mlflow/references/model-registry.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/model-registry.md)","13-mlops/mlflow/references/tracking.md":"# References Skill\n\nPath: `13-mlops/mlflow/references/tracking.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/tracking.md)","13-mlops/swanlab/references/integrations.md":"# References Skill\n\nPath: `13-mlops/swanlab/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/integrations.md)","13-mlops/swanlab/references/visualization.md":"# References Skill\n\nPath: `13-mlops/swanlab/references/visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/visualization.md)","13-mlops/tensorboard/references/integrations.md":"# References Skill\n\nPath: `13-mlops/tensorboard/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/integrations.md)","13-mlops/tensorboard/references/profiling.md":"# References Skill\n\nPath: `13-mlops/tensorboard/references/profiling.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/profiling.md)","13-mlops/tensorboard/references/visualization.md":"# References Skill\n\nPath: `13-mlops/tensorboard/references/visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/visualization.md)","13-mlops/weights-and-biases/references/artifacts.md":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/artifacts.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/artifacts.md)","13-mlops/weights-and-biases/references/integrations.md":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/integrations.md)","13-mlops/weights-and-biases/references/sweeps.md":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/sweeps.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/sweeps.md)","14-agents/a-evolve/references/api.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/api.md)","14-agents/a-evolve/references/architecture.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/architecture.md)","14-agents/a-evolve/references/design-patterns.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/design-patterns.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/design-patterns.md)","14-agents/a-evolve/references/examples.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/examples.md)","14-agents/a-evolve/references/issues.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/issues.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/issues.md)","14-agents/a-evolve/references/README.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/README.md)","14-agents/a-evolve/references/releases.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/releases.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/releases.md)","14-agents/a-evolve/references/tutorials.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/tutorials.md)","14-agents/autogpt/references/advanced-usage.md":"# References Skill\n\nPath: `14-agents/autogpt/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/advanced-usage.md)","14-agents/autogpt/references/troubleshooting.md":"# References Skill\n\nPath: `14-agents/autogpt/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/troubleshooting.md)","14-agents/crewai/references/flows.md":"# References Skill\n\nPath: `14-agents/crewai/references/flows.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/flows.md)","14-agents/crewai/references/tools.md":"# References Skill\n\nPath: `14-agents/crewai/references/tools.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/tools.md)","14-agents/crewai/references/troubleshooting.md":"# References Skill\n\nPath: `14-agents/crewai/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/troubleshooting.md)","14-agents/langchain/references/agents.md":"# References Skill\n\nPath: `14-agents/langchain/references/agents.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/agents.md)","14-agents/langchain/references/integration.md":"# References Skill\n\nPath: `14-agents/langchain/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/integration.md)","14-agents/langchain/references/rag.md":"# References Skill\n\nPath: `14-agents/langchain/references/rag.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/rag.md)","14-agents/llamaindex/references/agents.md":"# References Skill\n\nPath: `14-agents/llamaindex/references/agents.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/agents.md)","14-agents/llamaindex/references/data_connectors.md":"# References Skill\n\nPath: `14-agents/llamaindex/references/data_connectors.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/data_connectors.md)","14-agents/llamaindex/references/query_engines.md":"# References Skill\n\nPath: `14-agents/llamaindex/references/query_engines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/query_engines.md)","15-rag/chroma/references/integration.md":"# References Skill\n\nPath: `15-rag/chroma/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/references/integration.md)","15-rag/faiss/references/index_types.md":"# References Skill\n\nPath: `15-rag/faiss/references/index_types.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/references/index_types.md)","15-rag/pinecone/references/deployment.md":"# References Skill\n\nPath: `15-rag/pinecone/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/references/deployment.md)","15-rag/qdrant/references/advanced-usage.md":"# References Skill\n\nPath: `15-rag/qdrant/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/advanced-usage.md)","15-rag/qdrant/references/troubleshooting.md":"# References Skill\n\nPath: `15-rag/qdrant/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/troubleshooting.md)","15-rag/sentence-transformers/references/models.md":"# References Skill\n\nPath: `15-rag/sentence-transformers/references/models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/references/models.md)","16-prompt-engineering/dspy/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/examples.md)","16-prompt-engineering/dspy/references/modules.md":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/modules.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/modules.md)","16-prompt-engineering/dspy/references/optimizers.md":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/optimizers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/optimizers.md)","16-prompt-engineering/guidance/references/backends.md":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/backends.md)","16-prompt-engineering/guidance/references/constraints.md":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/constraints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/constraints.md)","16-prompt-engineering/guidance/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/examples.md)","16-prompt-engineering/instructor/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/examples.md)","16-prompt-engineering/instructor/references/providers.md":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/providers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/providers.md)","16-prompt-engineering/instructor/references/validation.md":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/validation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/validation.md)","16-prompt-engineering/outlines/references/backends.md":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/backends.md)","16-prompt-engineering/outlines/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/examples.md)","16-prompt-engineering/outlines/references/json_generation.md":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/json_generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/json_generation.md)","17-observability/langsmith/references/advanced-usage.md":"# References Skill\n\nPath: `17-observability/langsmith/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/advanced-usage.md)","17-observability/langsmith/references/troubleshooting.md":"# References Skill\n\nPath: `17-observability/langsmith/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/troubleshooting.md)","17-observability/phoenix/references/advanced-usage.md":"# References Skill\n\nPath: `17-observability/phoenix/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/advanced-usage.md)","17-observability/phoenix/references/troubleshooting.md":"# References Skill\n\nPath: `17-observability/phoenix/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/troubleshooting.md)","18-multimodal/audiocraft/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/audiocraft/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/advanced-usage.md)","18-multimodal/audiocraft/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/audiocraft/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/troubleshooting.md)","18-multimodal/blip-2/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/blip-2/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/advanced-usage.md)","18-multimodal/blip-2/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/blip-2/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/troubleshooting.md)","18-multimodal/clip/references/applications.md":"# References Skill\n\nPath: `18-multimodal/clip/references/applications.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/references/applications.md)","18-multimodal/cosmos-policy/references/libero-commands.md":"# References Skill\n\nPath: `18-multimodal/cosmos-policy/references/libero-commands.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/libero-commands.md)","18-multimodal/cosmos-policy/references/robocasa-commands.md":"# References Skill\n\nPath: `18-multimodal/cosmos-policy/references/robocasa-commands.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/robocasa-commands.md)","18-multimodal/llava/references/training.md":"# References Skill\n\nPath: `18-multimodal/llava/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/references/training.md)","18-multimodal/openpi/references/checkpoints-and-env-map.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/checkpoints-and-env-map.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/checkpoints-and-env-map.md)","18-multimodal/openpi/references/config-recipes.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/config-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/config-recipes.md)","18-multimodal/openpi/references/pytorch-gotchas.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/pytorch-gotchas.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/pytorch-gotchas.md)","18-multimodal/openpi/references/remote-client-pattern.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/remote-client-pattern.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/remote-client-pattern.md)","18-multimodal/openpi/references/training-debugging.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/training-debugging.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/training-debugging.md)","18-multimodal/openvla-oft/references/aloha-workflow.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/aloha-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/aloha-workflow.md)","18-multimodal/openvla-oft/references/config-troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/config-troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/config-troubleshooting.md)","18-multimodal/openvla-oft/references/libero-workflow.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/libero-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/libero-workflow.md)","18-multimodal/openvla-oft/references/paper-and-checkpoints.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/paper-and-checkpoints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/paper-and-checkpoints.md)","18-multimodal/segment-anything/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/segment-anything/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/advanced-usage.md)","18-multimodal/segment-anything/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/segment-anything/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/troubleshooting.md)","18-multimodal/stable-diffusion/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/stable-diffusion/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/advanced-usage.md)","18-multimodal/stable-diffusion/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/stable-diffusion/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/troubleshooting.md)","18-multimodal/whisper/references/languages.md":"# References Skill\n\nPath: `18-multimodal/whisper/references/languages.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/references/languages.md)","19-emerging-techniques/knowledge-distillation/references/minillm.md":"# References Skill\n\nPath: `19-emerging-techniques/knowledge-distillation/references/minillm.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/references/minillm.md)","19-emerging-techniques/long-context/references/extension_methods.md":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/extension_methods.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/extension_methods.md)","19-emerging-techniques/long-context/references/fine_tuning.md":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/fine_tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/fine_tuning.md)","19-emerging-techniques/long-context/references/rope.md":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/rope.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/rope.md)","19-emerging-techniques/model-merging/references/coefficient-tuning.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/coefficient-tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/coefficient-tuning.md)","19-emerging-techniques/model-merging/references/evaluation.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/evaluation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/evaluation.md)","19-emerging-techniques/model-merging/references/examples.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/examples.md)","19-emerging-techniques/model-merging/references/methods.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/methods.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/methods.md)","19-emerging-techniques/model-pruning/references/wanda.md":"# References Skill\n\nPath: `19-emerging-techniques/model-pruning/references/wanda.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/references/wanda.md)","19-emerging-techniques/moe-training/references/architectures.md":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/architectures.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/architectures.md)","19-emerging-techniques/moe-training/references/inference.md":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/inference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/inference.md)","19-emerging-techniques/moe-training/references/training.md":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/training.md)","19-emerging-techniques/speculative-decoding/references/lookahead.md":"# References Skill\n\nPath: `19-emerging-techniques/speculative-decoding/references/lookahead.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/lookahead.md)","19-emerging-techniques/speculative-decoding/references/medusa.md":"# References Skill\n\nPath: `19-emerging-techniques/speculative-decoding/references/medusa.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/medusa.md)","20-ml-paper-writing/systems-paper-writing/references/checklist.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/checklist.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/checklist.md)","20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md)","20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md)","20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md)","20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md)","20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md":"# References Skill\n\nPath: `20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md)","20-ml-paper-writing/ml-paper-writing/references/checklists.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/checklists.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/checklists.md)","20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md)","20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md)","20-ml-paper-writing/ml-paper-writing/references/sources.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/sources.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/sources.md)","20-ml-paper-writing/ml-paper-writing/references/writing-guide.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/writing-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/writing-guide.md)","20-ml-paper-writing/academic-plotting/references/data-visualization.md":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/data-visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/data-visualization.md)","20-ml-paper-writing/academic-plotting/references/diagram-generation.md":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/diagram-generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/diagram-generation.md)","20-ml-paper-writing/academic-plotting/references/style-guide.md":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/style-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/style-guide.md)","22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md)","22-agent-native-research-artifact/research-manager/references/event-taxonomy.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/event-taxonomy.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/event-taxonomy.md)","22-agent-native-research-artifact/research-manager/references/provenance-tags.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/provenance-tags.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/provenance-tags.md)","22-agent-native-research-artifact/research-manager/references/session-protocol.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/session-protocol.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/session-protocol.md)","22-agent-native-research-artifact/compiler/references/ara-schema.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/ara-schema.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/ara-schema.md)","22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md)","22-agent-native-research-artifact/compiler/references/validation-checklist.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/validation-checklist.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/validation-checklist.md)"},"files":{"01-model-architecture/mamba/SKILL.md":"---\nname: mamba-architecture\ndescription: State-space model with O(n) complexity vs Transformers' O(n²). 5× faster inference, million-token sequences, no KV cache. Selective SSM with hardware-aware design. Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head). Models 130M-2.8B on HuggingFace.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Mamba, State Space Models, SSM, Linear Complexity, Long Context, Efficient Inference, Hardware-Aware, Alternative To Transformers]\ndependencies: [mamba-ssm, torch, transformers, causal-conv1d]\n---\n\n# Mamba - Selective State Space Models\n\n## Quick start\n\nMamba is a state-space model architecture achieving O(n) linear complexity for sequence modeling.\n\n**Installation**:\n```bash\n# Install causal-conv1d (optional, for efficiency)\npip install causal-conv1d>=1.4.0\n\n# Install Mamba\npip install mamba-ssm\n# Or both together\npip install mamba-ssm[causal-conv1d]\n```\n\n**Prerequisites**: Linux, NVIDIA GPU, PyTorch 1.12+, CUDA 11.6+\n\n**Basic usage** (Mamba block):\n```python\nimport torch\nfrom mamba_ssm import Mamba\n\nbatch, length, dim = 2, 64, 16\nx = torch.randn(batch, length, dim).to(\"cuda\")\n\nmodel = Mamba(\n d_model=dim, # Model dimension\n d_state=16, # SSM state dimension\n d_conv=4, # Conv1d kernel size\n expand=2 # Expansion factor\n).to(\"cuda\")\n\ny = model(x) # O(n) complexity!\nassert y.shape == x.shape\n```\n\n## Common workflows\n\n### Workflow 1: Language model with Mamba-2\n\n**Complete LM with generation**:\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nfrom mamba_ssm.models.config_mamba import MambaConfig\nimport torch\n\n# Configure Mamba-2 LM\nconfig = MambaConfig(\n d_model=1024, # Hidden dimension\n n_layer=24, # Number of layers\n vocab_size=50277, # Vocabulary size\n ssm_cfg=dict(\n layer=\"Mamba2\", # Use Mamba-2\n d_state=128, # Larger state for Mamba-2\n headdim=64, # Head dimension\n ngroups=1 # Number of groups\n )\n)\n\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\n\n# Generate text\ninput_ids = torch.randint(0, 1000, (1, 20), device=\"cuda\", dtype=torch.long)\noutput = model.generate(\n input_ids=input_ids,\n max_length=100,\n temperature=0.7,\n top_p=0.9\n)\n```\n\n### Workflow 2: Use pretrained Mamba models\n\n**Load from HuggingFace**:\n```python\nfrom transformers import AutoTokenizer\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\n\n# Load pretrained model\nmodel_name = \"state-spaces/mamba-2.8b\"\ntokenizer = AutoTokenizer.from_pretrained(\"EleutherAI/gpt-neox-20b\") # Use compatible tokenizer\nmodel = MambaLMHeadModel.from_pretrained(model_name, device=\"cuda\", dtype=torch.float16)\n\n# Generate\nprompt = \"The future of AI is\"\ninput_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(\"cuda\")\noutput_ids = model.generate(\n input_ids=input_ids,\n max_length=200,\n temperature=0.7,\n top_p=0.9,\n repetition_penalty=1.2\n)\ngenerated_text = tokenizer.decode(output_ids[0])\nprint(generated_text)\n```\n\n**Available models**:\n- `state-spaces/mamba-130m`\n- `state-spaces/mamba-370m`\n- `state-spaces/mamba-790m`\n- `state-spaces/mamba-1.4b`\n- `state-spaces/mamba-2.8b`\n\n### Workflow 3: Mamba-1 vs Mamba-2\n\n**Mamba-1** (smaller state):\n```python\nfrom mamba_ssm import Mamba\n\nmodel = Mamba(\n d_model=256,\n d_state=16, # Smaller state dimension\n d_conv=4,\n expand=2\n).to(\"cuda\")\n```\n\n**Mamba-2** (multi-head, larger state):\n```python\nfrom mamba_ssm import Mamba2\n\nmodel = Mamba2(\n d_model=256,\n d_state=128, # Larger state dimension\n d_conv=4,\n expand=2,\n headdim=64, # Head dimension for multi-head\n ngroups=1 # Parallel groups\n).to(\"cuda\")\n```\n\n**Key differences**:\n- **State size**: Mamba-1 (d_state=16) vs Mamba-2 (d_state=128)\n- **Architecture**: Mamba-2 has multi-head structure\n- **Normalization**: Mamba-2 uses RMSNorm\n- **Distributed**: Mamba-2 supports tensor parallelism\n\n### Workflow 4: Benchmark vs Transformers\n\n**Generation speed comparison**:\n```bash\n# Benchmark Mamba\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"state-spaces/mamba-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n\n# Benchmark Transformer\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"EleutherAI/pythia-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n```\n\n**Expected results**:\n- **Mamba**: 5× faster inference\n- **Memory**: No KV cache needed\n- **Scaling**: Linear with sequence length\n\n## When to use vs alternatives\n\n**Use Mamba when**:\n- Need long sequences (100K+ tokens)\n- Want faster inference than Transformers\n- Memory-constrained (no KV cache)\n- Building streaming applications\n- Linear scaling important\n\n**Advantages**:\n- **O(n) complexity**: Linear vs quadratic\n- **5× faster inference**: No attention overhead\n- **No KV cache**: Lower memory usage\n- **Million-token sequences**: Hardware-efficient\n- **Streaming**: Constant memory per token\n\n**Use alternatives instead**:\n- **Transformers**: Need best-in-class performance, have compute\n- **RWKV**: Want RNN+Transformer hybrid\n- **RetNet**: Need retention-based architecture\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or use gradient checkpointing:\n```python\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\nmodel.gradient_checkpointing_enable() # Enable checkpointing\n```\n\n**Issue: Slow installation**\n\nInstall binary wheels (not source):\n```bash\npip install mamba-ssm --no-build-isolation\n```\n\n**Issue: Missing causal-conv1d**\n\nInstall separately:\n```bash\npip install causal-conv1d>=1.4.0\n```\n\n**Issue: Model not loading from HuggingFace**\n\nUse `MambaLMHeadModel.from_pretrained` (not `AutoModel`):\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nmodel = MambaLMHeadModel.from_pretrained(\"state-spaces/mamba-2.8b\")\n```\n\n## Advanced topics\n\n**Selective SSM**: See [references/selective-ssm.md](references/selective-ssm.md) for mathematical formulation, state-space equations, and how selectivity enables O(n) complexity.\n\n**Mamba-2 architecture**: See [references/mamba2-details.md](references/mamba2-details.md) for multi-head structure, tensor parallelism, and distributed training setup.\n\n**Performance optimization**: See [references/performance.md](references/performance.md) for hardware-aware design, CUDA kernels, and memory efficiency techniques.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA with CUDA 11.6+\n- **VRAM**:\n - 130M model: 2GB\n - 370M model: 4GB\n - 790M model: 8GB\n - 1.4B model: 14GB\n - 2.8B model: 28GB (FP16)\n- **Inference**: 5× faster than Transformers\n- **Memory**: No KV cache (lower than Transformers)\n\n**Performance** (vs Transformers):\n- **Speed**: 5× faster inference\n- **Memory**: 50% less (no KV cache)\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (Mamba-1): https://arxiv.org/abs/2312.00752 (Dec 2023)\n- Paper (Mamba-2): https://arxiv.org/abs/2405.21060 (May 2024)\n- GitHub: https://github.com/state-spaces/mamba ⭐ 13,000+\n- Models: https://huggingface.co/state-spaces\n- Docs: Repository README and wiki\n\n\n","01-model-architecture/rwkv/SKILL.md":"---\nname: rwkv-architecture\ndescription: RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [RWKV, Model Architecture, RNN, Transformer Hybrid, Linear Complexity, Infinite Context, Efficient Inference, Linux Foundation, Alternative Architecture]\ndependencies: [rwkv, torch, transformers]\n---\n\n# RWKV - Receptance Weighted Key Value\n\n## Quick start\n\nRWKV (RwaKuv) combines Transformer parallelization (training) with RNN efficiency (inference).\n\n**Installation**:\n```bash\n# Install PyTorch\npip install torch --upgrade --extra-index-url https://download.pytorch.org/whl/cu121\n\n# Install dependencies\npip install pytorch-lightning==1.9.5 deepspeed wandb ninja --upgrade\n\n# Install RWKV\npip install rwkv\n```\n\n**Basic usage** (GPT mode + RNN mode):\n```python\nimport os\nfrom rwkv.model import RWKV\n\nos.environ[\"RWKV_JIT_ON\"] = '1'\nos.environ[\"RWKV_CUDA_ON\"] = '1' # Use CUDA kernel for speed\n\n# Load model\nmodel = RWKV(\n model='/path/to/RWKV-4-Pile-1B5-20220903-8040',\n strategy='cuda fp16'\n)\n\n# GPT mode (parallel processing)\nout, state = model.forward([187, 510, 1563, 310, 247], None)\nprint(out.detach().cpu().numpy()) # Logits\n\n# RNN mode (sequential processing, same result)\nout, state = model.forward([187, 510], None) # First 2 tokens\nout, state = model.forward([1563], state) # Next token\nout, state = model.forward([310, 247], state) # Last tokens\nprint(out.detach().cpu().numpy()) # Same logits as above!\n```\n\n## Common workflows\n\n### Workflow 1: Text generation (streaming)\n\n**Efficient token-by-token generation**:\n```python\nfrom rwkv.model import RWKV\nfrom rwkv.utils import PIPELINE\n\nmodel = RWKV(model='RWKV-4-Pile-14B-20230313-ctx8192-test1050', strategy='cuda fp16')\npipeline = PIPELINE(model, \"20B_tokenizer.json\")\n\n# Initial prompt\nprompt = \"The future of AI is\"\nstate = None\n\n# Generate token by token\nfor token in prompt:\n out, state = pipeline.model.forward(pipeline.encode(token), state)\n\n# Continue generation\nfor _ in range(100):\n out, state = pipeline.model.forward(None, state)\n token = pipeline.sample_logits(out)\n print(pipeline.decode(token), end='', flush=True)\n```\n\n**Key advantage**: Constant memory per token (no growing KV cache)\n\n### Workflow 2: Long context processing (infinite context)\n\n**Process million-token sequences**:\n```python\nmodel = RWKV(model='RWKV-4-Pile-14B', strategy='cuda fp16')\n\n# Process very long document\nstate = None\nlong_document = load_document() # e.g., 1M tokens\n\n# Stream through entire document\nfor chunk in chunks(long_document, chunk_size=1024):\n out, state = model.forward(chunk, state)\n\n# State now contains information from entire 1M token document\n# Memory usage: O(1) (constant, not O(n)!)\n```\n\n### Workflow 3: Fine-tuning RWKV\n\n**Standard fine-tuning workflow**:\n```python\n# Training script\nimport pytorch_lightning as pl\nfrom rwkv.model import RWKV\nfrom rwkv.trainer import RWKVTrainer\n\n# Configure model\nconfig = {\n 'n_layer': 24,\n 'n_embd': 1024,\n 'vocab_size': 50277,\n 'ctx_len': 1024\n}\n\n# Setup trainer\ntrainer = pl.Trainer(\n accelerator='gpu',\n devices=8,\n precision='bf16',\n strategy='deepspeed_stage_2',\n max_epochs=1\n)\n\n# Train\nmodel = RWKV(config)\ntrainer.fit(model, train_dataloader)\n```\n\n### Workflow 4: RWKV vs Transformer comparison\n\n**Memory comparison** (1M token sequence):\n```python\n# Transformer (GPT)\n# Memory: O(n²) for attention\n# KV cache: 1M × hidden_dim × n_layers × 2 (keys + values)\n# Example: 1M × 4096 × 24 × 2 = ~400GB (impractical!)\n\n# RWKV\n# Memory: O(1) per token\n# State: hidden_dim × n_layers = 4096 × 24 = ~400KB\n# 1,000,000× more efficient!\n```\n\n**Speed comparison** (inference):\n```python\n# Transformer: O(n) per token (quadratic overall)\n# First token: 1 computation\n# Second token: 2 computations\n# ...\n# 1000th token: 1000 computations\n\n# RWKV: O(1) per token (linear overall)\n# Every token: 1 computation\n# 1000th token: 1 computation (same as first!)\n```\n\n## When to use vs alternatives\n\n**Use RWKV when**:\n- Need very long context (100K+ tokens)\n- Want constant memory usage\n- Building streaming applications\n- Need RNN efficiency with Transformer performance\n- Memory-constrained deployment\n\n**Key advantages**:\n- **Linear time**: O(n) vs O(n²) for Transformers\n- **No KV cache**: Constant memory per token\n- **Infinite context**: No fixed window limit\n- **Parallelizable training**: Like GPT\n- **Sequential inference**: Like RNN\n\n**Use alternatives instead**:\n- **Transformers**: Need absolute best performance, have compute\n- **Mamba**: Want state-space models\n- **RetNet**: Need retention mechanism\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: Out of memory during training**\n\nUse gradient checkpointing and DeepSpeed:\n```python\ntrainer = pl.Trainer(\n strategy='deepspeed_stage_3', # Full ZeRO-3\n precision='bf16'\n)\n```\n\n**Issue: Slow inference**\n\nEnable CUDA kernel:\n```python\nos.environ[\"RWKV_CUDA_ON\"] = '1'\n```\n\n**Issue: Model not loading**\n\nCheck model path and strategy:\n```python\nmodel = RWKV(\n model='/absolute/path/to/model.pth',\n strategy='cuda fp16' # Or 'cpu fp32' for CPU\n)\n```\n\n**Issue: State management in RNN mode**\n\nAlways pass state between forward calls:\n```python\n# WRONG: State lost\nout1, _ = model.forward(tokens1, None)\nout2, _ = model.forward(tokens2, None) # No context from tokens1!\n\n# CORRECT: State preserved\nout1, state = model.forward(tokens1, None)\nout2, state = model.forward(tokens2, state) # Has context from tokens1\n```\n\n## Advanced topics\n\n**Time-mixing and channel-mixing**: See [references/architecture-details.md](references/architecture-details.md) for WKV operation, time-decay mechanism, and receptance gates.\n\n**State management**: See [references/state-management.md](references/state-management.md) for att_x_prev, att_kv, ffn_x_prev states, and numerical stability considerations.\n\n**RWKV-7 improvements**: See [references/rwkv7.md](references/rwkv7.md) for latest architectural improvements (March 2025) and multimodal capabilities.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.6+) or CPU\n- **VRAM** (FP16):\n - 169M model: 1GB\n - 430M model: 2GB\n - 1.5B model: 4GB\n - 3B model: 8GB\n - 7B model: 16GB\n - 14B model: 32GB\n- **Inference**: O(1) memory per token\n- **Training**: Parallelizable like GPT\n\n**Performance** (vs Transformers):\n- **Speed**: Similar training, faster inference\n- **Memory**: 1000× less for long sequences\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (RWKV): https://arxiv.org/abs/2305.13048 (May 2023)\n- Paper (RWKV-7): https://arxiv.org/abs/2503.14456 (March 2025)\n- GitHub: https://github.com/BlinkDL/RWKV-LM ⭐ 12,000+\n- Docs: https://wiki.rwkv.com/\n- Models: https://huggingface.co/BlinkDL\n- Linux Foundation AI: Official project\n- Production: Microsoft Windows, Office integration, NeMo support\n\n\n","01-model-architecture/nanogpt/SKILL.md":"---\nname: nanogpt\ndescription: Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, NanoGPT, GPT-2, Educational, Andrej Karpathy, Transformer, Minimalist, From Scratch, Training]\ndependencies: [torch, transformers, datasets, tiktoken, wandb]\n---\n\n# nanoGPT - Minimalist GPT Training\n\n## Quick start\n\nnanoGPT is a simplified GPT implementation designed for learning and experimentation.\n\n**Installation**:\n```bash\npip install torch numpy transformers datasets tiktoken wandb tqdm\n```\n\n**Train on Shakespeare** (CPU-friendly):\n```bash\n# Prepare data\npython data/shakespeare_char/prepare.py\n\n# Train (5 minutes on CPU)\npython train.py config/train_shakespeare_char.py\n\n# Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Output**:\n```\nROMEO:\nWhat say'st thou? Shall I speak, and be a man?\n\nJULIET:\nI am afeard, and yet I'll speak; for thou art\nOne that hath been a man, and yet I know not\nWhat thou art.\n```\n\n## Common workflows\n\n### Workflow 1: Character-level Shakespeare\n\n**Complete training pipeline**:\n```bash\n# Step 1: Prepare data (creates train.bin, val.bin)\npython data/shakespeare_char/prepare.py\n\n# Step 2: Train small model\npython train.py config/train_shakespeare_char.py\n\n# Step 3: Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Config** (`config/train_shakespeare_char.py`):\n```python\n# Model config\nn_layer = 6 # 6 transformer layers\nn_head = 6 # 6 attention heads\nn_embd = 384 # 384-dim embeddings\nblock_size = 256 # 256 char context\n\n# Training config\nbatch_size = 64\nlearning_rate = 1e-3\nmax_iters = 5000\neval_interval = 500\n\n# Hardware\ndevice = 'cpu' # Or 'cuda'\ncompile = False # Set True for PyTorch 2.0\n```\n\n**Training time**: ~5 minutes (CPU), ~1 minute (GPU)\n\n### Workflow 2: Reproduce GPT-2 (124M)\n\n**Multi-GPU training on OpenWebText**:\n```bash\n# Step 1: Prepare OpenWebText (takes ~1 hour)\npython data/openwebtext/prepare.py\n\n# Step 2: Train GPT-2 124M with DDP (8 GPUs)\ntorchrun --standalone --nproc_per_node=8 \\\n train.py config/train_gpt2.py\n\n# Step 3: Sample from trained model\npython sample.py --out_dir=out\n```\n\n**Config** (`config/train_gpt2.py`):\n```python\n# GPT-2 (124M) architecture\nn_layer = 12\nn_head = 12\nn_embd = 768\nblock_size = 1024\ndropout = 0.0\n\n# Training\nbatch_size = 12\ngradient_accumulation_steps = 5 * 8 # Total batch ~0.5M tokens\nlearning_rate = 6e-4\nmax_iters = 600000\nlr_decay_iters = 600000\n\n# System\ncompile = True # PyTorch 2.0\n```\n\n**Training time**: ~4 days (8× A100)\n\n### Workflow 3: Fine-tune pretrained GPT-2\n\n**Start from OpenAI checkpoint**:\n```python\n# In train.py or config\ninit_from = 'gpt2' # Options: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n\n# Model loads OpenAI weights automatically\npython train.py config/finetune_shakespeare.py\n```\n\n**Example config** (`config/finetune_shakespeare.py`):\n```python\n# Start from GPT-2\ninit_from = 'gpt2'\n\n# Dataset\ndataset = 'shakespeare_char'\nbatch_size = 1\nblock_size = 1024\n\n# Fine-tuning\nlearning_rate = 3e-5 # Lower LR for fine-tuning\nmax_iters = 2000\nwarmup_iters = 100\n\n# Regularization\nweight_decay = 1e-1\n```\n\n### Workflow 4: Custom dataset\n\n**Train on your own text**:\n```python\n# data/custom/prepare.py\nimport numpy as np\n\n# Load your data\nwith open('my_data.txt', 'r') as f:\n text = f.read()\n\n# Create character mappings\nchars = sorted(list(set(text)))\nstoi = {ch: i for i, ch in enumerate(chars)}\nitos = {i: ch for i, ch in enumerate(chars)}\n\n# Tokenize\ndata = np.array([stoi[ch] for ch in text], dtype=np.uint16)\n\n# Split train/val\nn = len(data)\ntrain_data = data[:int(n*0.9)]\nval_data = data[int(n*0.9):]\n\n# Save\ntrain_data.tofile('data/custom/train.bin')\nval_data.tofile('data/custom/val.bin')\n```\n\n**Train**:\n```bash\npython data/custom/prepare.py\npython train.py --dataset=custom\n```\n\n## When to use vs alternatives\n\n**Use nanoGPT when**:\n- Learning how GPT works\n- Experimenting with transformer variants\n- Teaching/education purposes\n- Quick prototyping\n- Limited compute (can run on CPU)\n\n**Simplicity advantages**:\n- **~300 lines**: Entire model in `model.py`\n- **~300 lines**: Training loop in `train.py`\n- **Hackable**: Easy to modify\n- **No abstractions**: Pure PyTorch\n\n**Use alternatives instead**:\n- **HuggingFace Transformers**: Production use, many models\n- **Megatron-LM**: Large-scale distributed training\n- **LitGPT**: More architectures, production-ready\n- **PyTorch Lightning**: Need high-level framework\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or context length:\n```python\nbatch_size = 1 # Reduce from 12\nblock_size = 512 # Reduce from 1024\ngradient_accumulation_steps = 40 # Increase to maintain effective batch\n```\n\n**Issue: Training too slow**\n\nEnable compilation (PyTorch 2.0+):\n```python\ncompile = True # 2× speedup\n```\n\nUse mixed precision:\n```python\ndtype = 'bfloat16' # Or 'float16'\n```\n\n**Issue: Poor generation quality**\n\nTrain longer:\n```python\nmax_iters = 10000 # Increase from 5000\n```\n\nLower temperature:\n```python\n# In sample.py\ntemperature = 0.7 # Lower from 1.0\ntop_k = 200 # Add top-k sampling\n```\n\n**Issue: Can't load GPT-2 weights**\n\nInstall transformers:\n```bash\npip install transformers\n```\n\nCheck model name:\n```python\ninit_from = 'gpt2' # Valid: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n```\n\n## Advanced topics\n\n**Model architecture**: See [references/architecture.md](references/architecture.md) for GPT block structure, multi-head attention, and MLP layers explained simply.\n\n**Training loop**: See [references/training.md](references/training.md) for learning rate schedule, gradient accumulation, and distributed data parallel setup.\n\n**Data preparation**: See [references/data.md](references/data.md) for tokenization strategies (character-level vs BPE) and binary format details.\n\n## Hardware requirements\n\n- **Shakespeare (char-level)**:\n - CPU: 5 minutes\n - GPU (T4): 1 minute\n - VRAM: <1GB\n\n- **GPT-2 (124M)**:\n - 1× A100: ~1 week\n - 8× A100: ~4 days\n - VRAM: ~16GB per GPU\n\n- **GPT-2 Medium (350M)**:\n - 8× A100: ~2 weeks\n - VRAM: ~40GB per GPU\n\n**Performance**:\n- With `compile=True`: 2× speedup\n- With `dtype=bfloat16`: 50% memory reduction\n\n## Resources\n\n- GitHub: https://github.com/karpathy/nanoGPT ⭐ 48,000+\n- Video: \"Let's build GPT\" by Andrej Karpathy\n- Paper: \"Attention is All You Need\" (Vaswani et al.)\n- OpenWebText: https://huggingface.co/datasets/Skylion007/openwebtext\n- Educational: Best for understanding transformers from scratch\n\n\n","01-model-architecture/torchtitan/SKILL.md":"---\nname: distributed-llm-pretraining-torchtitan\ndescription: Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Distributed Training, TorchTitan, FSDP2, Tensor Parallel, Pipeline Parallel, Context Parallel, Float8, Llama, Pretraining]\ndependencies: [torch>=2.6.0, torchtitan>=0.2.0, torchao>=0.5.0]\n---\n\n# TorchTitan - PyTorch Native Distributed LLM Pretraining\n\n## Quick start\n\nTorchTitan is PyTorch's official platform for large-scale LLM pretraining with composable 4D parallelism (FSDP2, TP, PP, CP), achieving 65%+ speedups over baselines on H100 GPUs.\n\n**Installation**:\n```bash\n# From PyPI (stable)\npip install torchtitan\n\n# From source (latest features, requires PyTorch nightly)\ngit clone https://github.com/pytorch/torchtitan\ncd torchtitan\npip install -r requirements.txt\n```\n\n**Download tokenizer**:\n```bash\n# Get HF token from https://huggingface.co/settings/tokens\npython scripts/download_hf_assets.py --repo_id meta-llama/Llama-3.1-8B --assets tokenizer --hf_token=...\n```\n\n**Start training on 8 GPUs**:\n```bash\nCONFIG_FILE=\"./torchtitan/models/llama3/train_configs/llama3_8b.toml\" ./run_train.sh\n```\n\n## Common workflows\n\n### Workflow 1: Pretrain Llama 3.1 8B on single node\n\nCopy this checklist:\n\n```\nSingle Node Pretraining:\n- [ ] Step 1: Download tokenizer\n- [ ] Step 2: Configure training\n- [ ] Step 3: Launch training\n- [ ] Step 4: Monitor and checkpoint\n```\n\n**Step 1: Download tokenizer**\n\n```bash\npython scripts/download_hf_assets.py \\\n --repo_id meta-llama/Llama-3.1-8B \\\n --assets tokenizer \\\n --hf_token=YOUR_HF_TOKEN\n```\n\n**Step 2: Configure training**\n\nEdit or create a TOML config file:\n\n```toml\n# llama3_8b_custom.toml\n[job]\ndump_folder = \"./outputs\"\ndescription = \"Llama 3.1 8B training\"\n\n[model]\nname = \"llama3\"\nflavor = \"8B\"\nhf_assets_path = \"./assets/hf/Llama-3.1-8B\"\n\n[optimizer]\nname = \"AdamW\"\nlr = 3e-4\n\n[lr_scheduler]\nwarmup_steps = 200\n\n[training]\nlocal_batch_size = 2\nseq_len = 8192\nmax_norm = 1.0\nsteps = 1000\ndataset = \"c4\"\n\n[parallelism]\ndata_parallel_shard_degree = -1 # Use all GPUs for FSDP\n\n[activation_checkpoint]\nmode = \"selective\"\nselective_ac_option = \"op\"\n\n[checkpoint]\nenable = true\nfolder = \"checkpoint\"\ninterval = 500\n```\n\n**Step 3: Launch training**\n\n```bash\n# 8 GPUs on single node\nCONFIG_FILE=\"./llama3_8b_custom.toml\" ./run_train.sh\n\n# Or explicitly with torchrun\ntorchrun --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_8b_custom.toml\n```\n\n**Step 4: Monitor and checkpoint**\n\nTensorBoard logs are saved to `./outputs/tb/`:\n```bash\ntensorboard --logdir ./outputs/tb\n```\n\n### Workflow 2: Multi-node training with SLURM\n\n```\nMulti-Node Training:\n- [ ] Step 1: Configure parallelism for scale\n- [ ] Step 2: Set up SLURM script\n- [ ] Step 3: Submit job\n- [ ] Step 4: Resume from checkpoint\n```\n\n**Step 1: Configure parallelism for scale**\n\nFor 70B model on 256 GPUs (32 nodes):\n```toml\n[parallelism]\ndata_parallel_shard_degree = 32 # FSDP across 32 ranks\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 1 # No PP for 70B\ncontext_parallel_degree = 1 # Increase for long sequences\n```\n\n**Step 2: Set up SLURM script**\n\n```bash\n#!/bin/bash\n#SBATCH --job-name=llama70b\n#SBATCH --nodes=32\n#SBATCH --ntasks-per-node=8\n#SBATCH --gpus-per-node=8\n\nsrun torchrun \\\n --nnodes=32 \\\n --nproc_per_node=8 \\\n --rdzv_backend=c10d \\\n --rdzv_endpoint=$MASTER_ADDR:$MASTER_PORT \\\n -m torchtitan.train \\\n --job.config_file ./llama3_70b.toml\n```\n\n**Step 3: Submit job**\n\n```bash\nsbatch multinode_trainer.slurm\n```\n\n**Step 4: Resume from checkpoint**\n\nTraining auto-resumes if checkpoint exists in configured folder.\n\n### Workflow 3: Enable Float8 training for H100s\n\nFloat8 provides 30-50% speedup on H100 GPUs.\n\n```\nFloat8 Training:\n- [ ] Step 1: Install torchao\n- [ ] Step 2: Configure Float8\n- [ ] Step 3: Launch with compile\n```\n\n**Step 1: Install torchao**\n\n```bash\nUSE_CPP=0 pip install git+https://github.com/pytorch/ao.git\n```\n\n**Step 2: Configure Float8**\n\nAdd to your TOML config:\n```toml\n[model]\nconverters = [\"quantize.linear.float8\"]\n\n[quantize.linear.float8]\nenable_fsdp_float8_all_gather = true\nprecompute_float8_dynamic_scale_for_fsdp = true\nfilter_fqns = [\"output\"] # Exclude output layer\n\n[compile]\nenable = true\ncomponents = [\"model\", \"loss\"]\n```\n\n**Step 3: Launch with compile**\n\n```bash\nCONFIG_FILE=\"./llama3_8b.toml\" ./run_train.sh \\\n --model.converters=\"quantize.linear.float8\" \\\n --quantize.linear.float8.enable_fsdp_float8_all_gather \\\n --compile.enable\n```\n\n### Workflow 4: 4D parallelism for 405B models\n\n```\n4D Parallelism (FSDP + TP + PP + CP):\n- [ ] Step 1: Create seed checkpoint\n- [ ] Step 2: Configure 4D parallelism\n- [ ] Step 3: Launch on 512 GPUs\n```\n\n**Step 1: Create seed checkpoint**\n\nRequired for consistent initialization across PP stages:\n```bash\nNGPU=1 CONFIG_FILE=./llama3_405b.toml ./run_train.sh \\\n --checkpoint.enable \\\n --checkpoint.create_seed_checkpoint \\\n --parallelism.data_parallel_shard_degree 1 \\\n --parallelism.tensor_parallel_degree 1 \\\n --parallelism.pipeline_parallel_degree 1\n```\n\n**Step 2: Configure 4D parallelism**\n\n```toml\n[parallelism]\ndata_parallel_shard_degree = 8 # FSDP\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 8 # PP across nodes\ncontext_parallel_degree = 1 # CP for long sequences\n\n[training]\nlocal_batch_size = 32\nseq_len = 8192\n```\n\n**Step 3: Launch on 512 GPUs**\n\n```bash\n# 64 nodes x 8 GPUs = 512 GPUs\nsrun torchrun --nnodes=64 --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_405b.toml\n```\n\n## When to use vs alternatives\n\n**Use TorchTitan when:**\n- Pretraining LLMs from scratch (8B to 405B+)\n- Need PyTorch-native solution without third-party dependencies\n- Require composable 4D parallelism (FSDP2, TP, PP, CP)\n- Training on H100s with Float8 support\n- Want interoperable checkpoints with torchtune/HuggingFace\n\n**Use alternatives instead:**\n- **Megatron-LM**: Maximum performance for NVIDIA-only deployments\n- **DeepSpeed**: Broader ZeRO optimization ecosystem, inference support\n- **Axolotl/TRL**: Fine-tuning rather than pretraining\n- **LitGPT**: Educational, smaller-scale training\n\n## Common issues\n\n**Issue: Out of memory on large models**\n\nEnable activation checkpointing and reduce batch size:\n```toml\n[activation_checkpoint]\nmode = \"full\" # Instead of \"selective\"\n\n[training]\nlocal_batch_size = 1\n```\n\nOr use gradient accumulation:\n```toml\n[training]\nlocal_batch_size = 1\nglobal_batch_size = 32 # Accumulates gradients\n```\n\n**Issue: TP causes high memory with async collectives**\n\nSet environment variable:\n```bash\nexport TORCH_NCCL_AVOID_RECORD_STREAMS=1\n```\n\n**Issue: Float8 training not faster**\n\nFloat8 only benefits large GEMMs. Filter small layers:\n```toml\n[quantize.linear.float8]\nfilter_fqns = [\"attention.wk\", \"attention.wv\", \"output\", \"auto_filter_small_kn\"]\n```\n\n**Issue: Checkpoint loading fails after parallelism change**\n\nUse DCP's resharding capability:\n```bash\n# Convert sharded checkpoint to single file\npython -m torch.distributed.checkpoint.format_utils \\\n dcp_to_torch checkpoint/step-1000 checkpoint.pt\n```\n\n**Issue: Pipeline parallelism initialization**\n\nCreate seed checkpoint first (see Workflow 4, Step 1).\n\n## Supported models\n\n| Model | Sizes | Status |\n|-------|-------|--------|\n| Llama 3.1 | 8B, 70B, 405B | Production |\n| Llama 4 | Various | Experimental |\n| DeepSeek V3 | 16B, 236B, 671B (MoE) | Experimental |\n| GPT-OSS | 20B, 120B (MoE) | Experimental |\n| Qwen 3 | Various | Experimental |\n| Flux | Diffusion | Experimental |\n\n## Performance benchmarks (H100)\n\n| Model | GPUs | Parallelism | TPS/GPU | Techniques |\n|-------|------|-------------|---------|------------|\n| Llama 8B | 8 | FSDP | 5,762 | Baseline |\n| Llama 8B | 8 | FSDP+compile+FP8 | 8,532 | +48% |\n| Llama 70B | 256 | FSDP+TP+AsyncTP | 876 | 2D parallel |\n| Llama 405B | 512 | FSDP+TP+PP | 128 | 3D parallel |\n\n## Advanced topics\n\n**FSDP2 configuration**: See [references/fsdp.md](references/fsdp.md) for detailed FSDP2 vs FSDP1 comparison and ZeRO equivalents.\n\n**Float8 training**: See [references/float8.md](references/float8.md) for tensorwise vs rowwise scaling recipes.\n\n**Checkpointing**: See [references/checkpoint.md](references/checkpoint.md) for HuggingFace conversion and async checkpointing.\n\n**Adding custom models**: See [references/custom-models.md](references/custom-models.md) for TrainSpec protocol.\n\n## Resources\n\n- GitHub: https://github.com/pytorch/torchtitan\n- Paper: https://arxiv.org/abs/2410.06511\n- ICLR 2025: https://iclr.cc/virtual/2025/poster/29620\n- PyTorch Forum: https://discuss.pytorch.org/c/distributed/torchtitan/44\n\n","01-model-architecture/litgpt/SKILL.md":"---\nname: implementing-llms-litgpt\ndescription: Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, LitGPT, Lightning AI, LLM Implementation, LoRA, QLoRA, Fine-Tuning, Llama, Gemma, Phi, Mistral, Educational]\ndependencies: [litgpt, torch, transformers]\n---\n\n# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft/phi-2\")\n\n# Generate text\nresult = llm.generate(\n \"What is the capital of France?\",\n max_new_tokens=50,\n temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama/Meta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft/phi-2\n\n# Download Gemma 2B\nlitgpt download google/gemma-2b\n```\n\nModels are saved to `checkpoints/` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n {\n \"instruction\": \"What is the capital of France?\",\n \"input\": \"\",\n \"output\": \"The capital of France is Paris.\"\n },\n {\n \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n \"input\": \"\",\n \"output\": \"Hola, ¿cómo estás?\"\n }\n]\n```\n\nSave as `data/my_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n meta-llama/Meta-Llama-3-8B \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --train.max_steps 1000 \\\n --train.learning_rate 2e-5 \\\n --train.micro_batch_size 1 \\\n --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --lora_alpha 32 \\\n --lora_dropout 0.05 \\\n --train.max_steps 1000 \\\n --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out/finetune/` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out/finetune/logs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out/finetune/lightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality/size tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\ # LoRA rank (8-64, higher=more capacity)\n --lora_alpha 32 \\ # LoRA scaling (typically 2×r)\n --lora_dropout 0.05 \\ # Prevent overfitting\n --lora_query true \\ # Apply LoRA to query projection\n --lora_key false \\ # Usually not needed\n --lora_value true \\ # Apply LoRA to value projection\n --lora_projection true \\ # Apply LoRA to output projection\n --lora_mlp false \\ # Usually not needed\n --lora_head false # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --train.epochs 3 \\\n --train.learning_rate 1e-4 \\\n --train.micro_batch_size 4 \\\n --train.global_batch_size 32 \\\n --out_dir out/phi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n out/phi2-lora/final \\\n --out_dir out/phi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out/phi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts/prepare_dataset.py \\\n --source_path data/my_corpus.txt \\\n --checkpoint_dir checkpoints/tokenizer \\\n --destination_path data/pretrain \\\n --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config/pythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n --config config/pythia-160m.yaml \\\n --data.data_dir data/pretrain \\\n --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir data/pretrain \\\n --devices 8 \\\n --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir /shared/data/pretrain \\\n --devices 8 \\\n --num_nodes 8 \\\n --train.global_batch_size 512 \\\n --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out/phi2-lora/final\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --dtype bfloat16 \\\n --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --quantize bnb.nf4-dq # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts/convert_lit_checkpoint.py \\\n --checkpoint_path out/phi2-lora/final \\\n --output_path models/phi2.gguf \\\n --model_name microsoft/phi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out/phi2-lora/final\")\n\n@app.post(\"/generate\")\ndef generate(prompt: str, max_tokens: int = 100):\n result = llm.generate(\n prompt,\n max_new_tokens=max_tokens,\n temperature=0.7\n )\n return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl/TRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n ... \\\n --train.gradient_accumulation_iters 4 # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30/40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32 # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama/Meta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints/\n# Should see: meta-llama/Meta-Llama-3-8B/\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8 # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\ # Disable this\n--lora_mlp false # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references/supported-models.md](references/supported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references/training-recipes.md](references/training-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references/distributed-training.md](references/distributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references/custom-models.md](references/custom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n - Inference (Phi-2): 6GB\n - LoRA fine-tuning (7B): 16GB\n - Full fine-tuning (7B): 40GB+\n - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https://github.com/Lightning-AI/litgpt\n- Docs: https://lightning.ai/docs/litgpt\n- Tutorials: https://lightning.ai/docs/litgpt/tutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n","02-tokenization/huggingface-tokenizers/SKILL.md":"---\nname: huggingface-tokenizers\ndescription: Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Tokenization, HuggingFace, BPE, WordPiece, Unigram, Fast Tokenization, Rust, Custom Tokenizer, Alignment Tracking, Production]\ndependencies: [tokenizers, transformers, datasets]\n---\n\n# HuggingFace Tokenizers - Fast Tokenization for NLP\n\nFast, production-ready tokenizers with Rust performance and Python ease-of-use.\n\n## When to use HuggingFace Tokenizers\n\n**Use HuggingFace Tokenizers when:**\n- Need extremely fast tokenization (<20s per GB of text)\n- Training custom tokenizers from scratch\n- Want alignment tracking (token → original text position)\n- Building production NLP pipelines\n- Need to tokenize large corpora efficiently\n\n**Performance**:\n- **Speed**: <20 seconds to tokenize 1GB on CPU\n- **Implementation**: Rust core with Python/Node.js bindings\n- **Efficiency**: 10-100× faster than pure Python implementations\n\n**Use alternatives instead**:\n- **SentencePiece**: Language-independent, used by T5/ALBERT\n- **tiktoken**: OpenAI's BPE tokenizer for GPT models\n- **transformers AutoTokenizer**: Loading pretrained only (uses this library internally)\n\n## Quick start\n\n### Installation\n\n```bash\n# Install tokenizers\npip install tokenizers\n\n# With transformers integration\npip install tokenizers transformers\n```\n\n### Load pretrained tokenizer\n\n```python\nfrom tokenizers import Tokenizer\n\n# Load from HuggingFace Hub\ntokenizer = Tokenizer.from_pretrained(\"bert-base-uncased\")\n\n# Encode text\noutput = tokenizer.encode(\"Hello, how are you?\")\nprint(output.tokens) # ['hello', ',', 'how', 'are', 'you', '?']\nprint(output.ids) # [7592, 1010, 2129, 2024, 2017, 1029]\n\n# Decode back\ntext = tokenizer.decode(output.ids)\nprint(text) # \"hello, how are you?\"\n```\n\n### Train custom BPE tokenizer\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import BPE\nfrom tokenizers.trainers import BpeTrainer\nfrom tokenizers.pre_tokenizers import Whitespace\n\n# Initialize tokenizer with BPE model\ntokenizer = Tokenizer(BPE(unk_token=\"[UNK]\"))\ntokenizer.pre_tokenizer = Whitespace()\n\n# Configure trainer\ntrainer = BpeTrainer(\n vocab_size=30000,\n special_tokens=[\"[UNK]\", \"[CLS]\", \"[SEP]\", \"[PAD]\", \"[MASK]\"],\n min_frequency=2\n)\n\n# Train on files\nfiles = [\"train.txt\", \"validation.txt\"]\ntokenizer.train(files, trainer)\n\n# Save\ntokenizer.save(\"my-tokenizer.json\")\n```\n\n**Training time**: ~1-2 minutes for 100MB corpus, ~10-20 minutes for 1GB\n\n### Batch encoding with padding\n\n```python\n# Enable padding\ntokenizer.enable_padding(pad_id=3, pad_token=\"[PAD]\")\n\n# Encode batch\ntexts = [\"Hello world\", \"This is a longer sentence\"]\nencodings = tokenizer.encode_batch(texts)\n\nfor encoding in encodings:\n print(encoding.ids)\n# [101, 7592, 2088, 102, 3, 3, 3]\n# [101, 2023, 2003, 1037, 2936, 6251, 102]\n```\n\n## Tokenization algorithms\n\n### BPE (Byte-Pair Encoding)\n\n**How it works**:\n1. Start with character-level vocabulary\n2. Find most frequent character pair\n3. Merge into new token, add to vocabulary\n4. Repeat until vocabulary size reached\n\n**Used by**: GPT-2, GPT-3, RoBERTa, BART, DeBERTa\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import BPE\nfrom tokenizers.trainers import BpeTrainer\nfrom tokenizers.pre_tokenizers import ByteLevel\n\ntokenizer = Tokenizer(BPE(unk_token=\"<|endoftext|>\"))\ntokenizer.pre_tokenizer = ByteLevel()\n\ntrainer = BpeTrainer(\n vocab_size=50257,\n special_tokens=[\"<|endoftext|>\"],\n min_frequency=2\n)\n\ntokenizer.train(files=[\"data.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Handles OOV words well (breaks into subwords)\n- Flexible vocabulary size\n- Good for morphologically rich languages\n\n**Trade-offs**:\n- Tokenization depends on merge order\n- May split common words unexpectedly\n\n### WordPiece\n\n**How it works**:\n1. Start with character vocabulary\n2. Score merge pairs: `frequency(pair) / (frequency(first) × frequency(second))`\n3. Merge highest scoring pair\n4. Repeat until vocabulary size reached\n\n**Used by**: BERT, DistilBERT, MobileBERT\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import WordPiece\nfrom tokenizers.trainers import WordPieceTrainer\nfrom tokenizers.pre_tokenizers import Whitespace\nfrom tokenizers.normalizers import BertNormalizer\n\ntokenizer = Tokenizer(WordPiece(unk_token=\"[UNK]\"))\ntokenizer.normalizer = BertNormalizer(lowercase=True)\ntokenizer.pre_tokenizer = Whitespace()\n\ntrainer = WordPieceTrainer(\n vocab_size=30522,\n special_tokens=[\"[UNK]\", \"[CLS]\", \"[SEP]\", \"[PAD]\", \"[MASK]\"],\n continuing_subword_prefix=\"##\"\n)\n\ntokenizer.train(files=[\"corpus.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Prioritizes meaningful merges (high score = semantically related)\n- Used successfully in BERT (state-of-the-art results)\n\n**Trade-offs**:\n- Unknown words become `[UNK]` if no subword match\n- Saves vocabulary, not merge rules (larger files)\n\n### Unigram\n\n**How it works**:\n1. Start with large vocabulary (all substrings)\n2. Compute loss for corpus with current vocabulary\n3. Remove tokens with minimal impact on loss\n4. Repeat until vocabulary size reached\n\n**Used by**: ALBERT, T5, mBART, XLNet (via SentencePiece)\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import Unigram\nfrom tokenizers.trainers import UnigramTrainer\n\ntokenizer = Tokenizer(Unigram())\n\ntrainer = UnigramTrainer(\n vocab_size=8000,\n special_tokens=[\"\", \"\", \"\"],\n unk_token=\"\"\n)\n\ntokenizer.train(files=[\"data.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Probabilistic (finds most likely tokenization)\n- Works well for languages without word boundaries\n- Handles diverse linguistic contexts\n\n**Trade-offs**:\n- Computationally expensive to train\n- More hyperparameters to tune\n\n## Tokenization pipeline\n\nComplete pipeline: **Normalization → Pre-tokenization → Model → Post-processing**\n\n### Normalization\n\nClean and standardize text:\n\n```python\nfrom tokenizers.normalizers import NFD, StripAccents, Lowercase, Sequence\n\ntokenizer.normalizer = Sequence([\n NFD(), # Unicode normalization (decompose)\n Lowercase(), # Convert to lowercase\n StripAccents() # Remove accents\n])\n\n# Input: \"Héllo WORLD\"\n# After normalization: \"hello world\"\n```\n\n**Common normalizers**:\n- `NFD`, `NFC`, `NFKD`, `NFKC` - Unicode normalization forms\n- `Lowercase()` - Convert to lowercase\n- `StripAccents()` - Remove accents (é → e)\n- `Strip()` - Remove whitespace\n- `Replace(pattern, content)` - Regex replacement\n\n### Pre-tokenization\n\nSplit text into word-like units:\n\n```python\nfrom tokenizers.pre_tokenizers import Whitespace, Punctuation, Sequence, ByteLevel\n\n# Split on whitespace and punctuation\ntokenizer.pre_tokenizer = Sequence([\n Whitespace(),\n Punctuation()\n])\n\n# Input: \"Hello, world!\"\n# After pre-tokenization: [\"Hello\", \",\", \"world\", \"!\"]\n```\n\n**Common pre-tokenizers**:\n- `Whitespace()` - Split on spaces, tabs, newlines\n- `ByteLevel()` - GPT-2 style byte-level splitting\n- `Punctuation()` - Isolate punctuation\n- `Digits(individual_digits=True)` - Split digits individually\n- `Metaspace()` - Replace spaces with ▁ (SentencePiece style)\n\n### Post-processing\n\nAdd special tokens for model input:\n\n```python\nfrom tokenizers.processors import TemplateProcessing\n\n# BERT-style: [CLS] sentence [SEP]\ntokenizer.post_processor = TemplateProcessing(\n single=\"[CLS] $A [SEP]\",\n pair=\"[CLS] $A [SEP] $B [SEP]\",\n special_tokens=[\n (\"[CLS]\", 1),\n (\"[SEP]\", 2),\n ],\n)\n```\n\n**Common patterns**:\n```python\n# GPT-2: sentence <|endoftext|>\nTemplateProcessing(\n single=\"$A <|endoftext|>\",\n special_tokens=[(\"<|endoftext|>\", 50256)]\n)\n\n# RoBERTa: sentence \nTemplateProcessing(\n single=\" $A \",\n pair=\" $A $B \",\n special_tokens=[(\"\", 0), (\"\", 2)]\n)\n```\n\n## Alignment tracking\n\nTrack token positions in original text:\n\n```python\noutput = tokenizer.encode(\"Hello, world!\")\n\n# Get token offsets\nfor token, offset in zip(output.tokens, output.offsets):\n start, end = offset\n print(f\"{token:10} → [{start:2}, {end:2}): {text[start:end]!r}\")\n\n# Output:\n# hello → [ 0, 5): 'Hello'\n# , → [ 5, 6): ','\n# world → [ 7, 12): 'world'\n# ! → [12, 13): '!'\n```\n\n**Use cases**:\n- Named entity recognition (map predictions back to text)\n- Question answering (extract answer spans)\n- Token classification (align labels to original positions)\n\n## Integration with transformers\n\n### Load with AutoTokenizer\n\n```python\nfrom transformers import AutoTokenizer\n\n# AutoTokenizer automatically uses fast tokenizers\ntokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n\n# Check if using fast tokenizer\nprint(tokenizer.is_fast) # True\n\n# Access underlying tokenizers.Tokenizer\nfast_tokenizer = tokenizer.backend_tokenizer\nprint(type(fast_tokenizer)) # \n```\n\n### Convert custom tokenizer to transformers\n\n```python\nfrom tokenizers import Tokenizer\nfrom transformers import PreTrainedTokenizerFast\n\n# Train custom tokenizer\ntokenizer = Tokenizer(BPE())\n# ... train tokenizer ...\ntokenizer.save(\"my-tokenizer.json\")\n\n# Wrap for transformers\ntransformers_tokenizer = PreTrainedTokenizerFast(\n tokenizer_file=\"my-tokenizer.json\",\n unk_token=\"[UNK]\",\n pad_token=\"[PAD]\",\n cls_token=\"[CLS]\",\n sep_token=\"[SEP]\",\n mask_token=\"[MASK]\"\n)\n\n# Use like any transformers tokenizer\noutputs = transformers_tokenizer(\n \"Hello world\",\n padding=True,\n truncation=True,\n max_length=512,\n return_tensors=\"pt\"\n)\n```\n\n## Common patterns\n\n### Train from iterator (large datasets)\n\n```python\nfrom datasets import load_dataset\n\n# Load dataset\ndataset = load_dataset(\"wikitext\", \"wikitext-103-raw-v1\", split=\"train\")\n\n# Create batch iterator\ndef batch_iterator(batch_size=1000):\n for i in range(0, len(dataset), batch_size):\n yield dataset[i:i + batch_size][\"text\"]\n\n# Train tokenizer\ntokenizer.train_from_iterator(\n batch_iterator(),\n trainer=trainer,\n length=len(dataset) # For progress bar\n)\n```\n\n**Performance**: Processes 1GB in ~10-20 minutes\n\n### Enable truncation and padding\n\n```python\n# Enable truncation\ntokenizer.enable_truncation(max_length=512)\n\n# Enable padding\ntokenizer.enable_padding(\n pad_id=tokenizer.token_to_id(\"[PAD]\"),\n pad_token=\"[PAD]\",\n length=512 # Fixed length, or None for batch max\n)\n\n# Encode with both\noutput = tokenizer.encode(\"This is a long sentence that will be truncated...\")\nprint(len(output.ids)) # 512\n```\n\n### Multi-processing\n\n```python\nfrom tokenizers import Tokenizer\nfrom multiprocessing import Pool\n\n# Load tokenizer\ntokenizer = Tokenizer.from_file(\"tokenizer.json\")\n\ndef encode_batch(texts):\n return tokenizer.encode_batch(texts)\n\n# Process large corpus in parallel\nwith Pool(8) as pool:\n # Split corpus into chunks\n chunk_size = 1000\n chunks = [corpus[i:i+chunk_size] for i in range(0, len(corpus), chunk_size)]\n\n # Encode in parallel\n results = pool.map(encode_batch, chunks)\n```\n\n**Speedup**: 5-8× with 8 cores\n\n## Performance benchmarks\n\n### Training speed\n\n| Corpus Size | BPE (30k vocab) | WordPiece (30k) | Unigram (8k) |\n|-------------|-----------------|-----------------|--------------|\n| 10 MB | 15 sec | 18 sec | 25 sec |\n| 100 MB | 1.5 min | 2 min | 4 min |\n| 1 GB | 15 min | 20 min | 40 min |\n\n**Hardware**: 16-core CPU, tested on English Wikipedia\n\n### Tokenization speed\n\n| Implementation | 1 GB corpus | Throughput |\n|----------------|-------------|---------------|\n| Pure Python | ~20 minutes | ~50 MB/min |\n| HF Tokenizers | ~15 seconds | ~4 GB/min |\n| **Speedup** | **80×** | **80×** |\n\n**Test**: English text, average sentence length 20 words\n\n### Memory usage\n\n| Task | Memory |\n|-------------------------|---------|\n| Load tokenizer | ~10 MB |\n| Train BPE (30k vocab) | ~200 MB |\n| Encode 1M sentences | ~500 MB |\n\n## Supported models\n\nPre-trained tokenizers available via `from_pretrained()`:\n\n**BERT family**:\n- `bert-base-uncased`, `bert-large-cased`\n- `distilbert-base-uncased`\n- `roberta-base`, `roberta-large`\n\n**GPT family**:\n- `gpt2`, `gpt2-medium`, `gpt2-large`\n- `distilgpt2`\n\n**T5 family**:\n- `t5-small`, `t5-base`, `t5-large`\n- `google/flan-t5-xxl`\n\n**Other**:\n- `facebook/bart-base`, `facebook/mbart-large-cc25`\n- `albert-base-v2`, `albert-xlarge-v2`\n- `xlm-roberta-base`, `xlm-roberta-large`\n\nBrowse all: https://huggingface.co/models?library=tokenizers\n\n## References\n\n- **[Training Guide](references/training.md)** - Train custom tokenizers, configure trainers, handle large datasets\n- **[Algorithms Deep Dive](references/algorithms.md)** - BPE, WordPiece, Unigram explained in detail\n- **[Pipeline Components](references/pipeline.md)** - Normalizers, pre-tokenizers, post-processors, decoders\n- **[Transformers Integration](references/integration.md)** - AutoTokenizer, PreTrainedTokenizerFast, special tokens\n\n## Resources\n\n- **Docs**: https://huggingface.co/docs/tokenizers\n- **GitHub**: https://github.com/huggingface/tokenizers ⭐ 9,000+\n- **Version**: 0.20.0+\n- **Course**: https://huggingface.co/learn/nlp-course/chapter6/1\n- **Paper**: BPE (Sennrich et al., 2016), WordPiece (Schuster & Nakajima, 2012)\n\n\n","0-autoresearch-skill/SKILL.md":"---\nname: autoresearch\ndescription: Orchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for execution, supports continuous agent operation via Claude Code /loop and OpenClaw heartbeat, and produces research presentations and papers. Use when starting a research project, running autonomous experiments, or managing a multi-hypothesis research effort.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Autonomous Research, Two-Loop Architecture, Experiment Orchestration, Research Synthesis, Project Management]\n---\n\n# Autoresearch\n\nAutonomous research orchestration for AI coding agents. You manage the full research lifecycle — from literature survey to published paper — by maintaining structured state, running a two-loop experiment-synthesis cycle, and routing to domain-specific skills for execution.\n\nYou are a research project manager, not a domain expert. You orchestrate; the domain skills execute.\n\n**This runs fully autonomously.** Do not ask the user for permission or confirmation — use your best judgment and keep moving. Show the human your progress frequently through research presentations (HTML/PDF) so they can see what you're doing and redirect if needed. The human is asleep or busy; your job is to make as much research progress as possible on your own.\n\n## Getting Started\n\nUsers arrive in different states. Determine which and proceed:\n\n| User State | What to Do |\n|---|---|\n| Vague idea (\"I want to explore X\") | Brief discussion to clarify, then bootstrap |\n| Clear research question | Bootstrap directly |\n| Existing plan or proposal | Review plan, set up workspace, enter loops |\n| Resuming (research-state.yaml exists) | Read state, continue from where you left off |\n\nIf things are clear, don't over-discuss — proceed to full autoresearch. Most users want you to just start researching.\n\n**Step 0 — before anything else**: Set up the agent continuity loop. See [Agent Continuity](#agent-continuity-mandatory--set-up-first). This is MANDATORY. Without it, the research stops after one cycle.\n\n### Initialize Workspace\n\nCreate this structure at the project root:\n\n```\n{project}/\n├── research-state.yaml # Central state tracking\n├── research-log.md # Decision timeline\n├── findings.md # Evolving narrative synthesis\n├── literature/ # Papers, survey notes\n├── src/ # Reusable code (utils, plotting, shared modules)\n├── data/ # Raw result data (CSVs, JSONs, checkpoints)\n├── experiments/ # Per-hypothesis work\n│ └── {hypothesis-slug}/\n│ ├── protocol.md # What, why, and prediction\n│ ├── code/ # Experiment-specific code\n│ ├── results/ # Raw outputs, metrics, logs\n│ └── analysis.md # What we learned\n├── to_human/ # Progress presentations and reports for human review\n└── paper/ # Final paper (via ml-paper-writing)\n```\n\n- **`src/`**: When you write useful code (plotting functions, data loaders, evaluation helpers), move it here so it can be reused across experiments. Don't duplicate code in every experiment directory.\n- **`data/`**: Save raw result data (metric CSVs, training logs, small outputs) here in a structured way. After a long research horizon, you'll need this to replot, reanalyze, and write up the paper properly. Name files descriptively (e.g., `trajectory_H1_runs001-010.csv`). Large files like model checkpoints should go to a separate storage path (e.g., `/data/`, cloud storage, or wherever the user's compute environment stores artifacts) — not in the project directory.\n\nInitialize `research-state.yaml`, `research-log.md`, and `findings.md` from [templates/](templates/). Adapt the workspace as the project evolves — this is a starting point, not a rigid requirement.\n\n## The Two-Loop Architecture\n\nThis is the core engine. Everything else supports it.\n\n```\nBOOTSTRAP (once, lightweight)\n Scope question → search literature → form initial hypotheses\n\nINNER LOOP (fast, autonomous, repeating)\n Pick hypothesis → experiment → measure → record → learn → next\n Goal: run constrained experiments with clear measurable outcomes\n\nOUTER LOOP (periodic, reflective)\n Review results → find patterns → update findings.md →\n new hypotheses → decide direction\n Goal: synthesize understanding, find the story — this is where novelty comes from\n\nFINALIZE (when concluding)\n Write paper via ml-paper-writing → final presentation → archive\n```\n\nThe inner loop runs tight experiment cycles with clear measurable outcomes. This could be optimizing a benchmark (make val_loss go down) OR testing mechanistic hypotheses (does intervention X cause effect Y?). The outer loop steps back to ask: what do these results *mean*? What patterns emerge? What's the story? Research is open-ended — the two loops let you both optimize and discover.\n\nThere is no rigid boundary between the two loops — you decide when enough inner loop results have accumulated to warrant reflection. Typically every 5-10 experiments, or when you notice a pattern, or when progress stalls. The agent's judgment drives the rhythm.\n\n### Research is Non-Linear\n\nThe two-loop structure is a rhythm, not a railroad. At any point during research you can and should:\n\n- **Return to literature** when results surprise you, assumptions break, or you need context for a new direction — always save what you find to `literature/`\n- **Brainstorm new ideas** using `21-research-ideation/` skills when you're stuck or when results open unexpected questions\n- **Pivot the question entirely** if experiments reveal the original question was wrong or less interesting than what you found\n\nThis is normal. Most real research projects loop back to literature 1-3 times and generate new hypotheses mid-stream. Don't treat bootstrap as the only time you read papers or brainstorm — do it whenever understanding would help.\n\n## Bootstrap: Literature and Hypotheses\n\nBefore entering the loops, understand the landscape. Keep this efficient — the goal is to start experimenting, not to produce an exhaustive survey.\n\n1. **Search literature** for the research question. Use multiple sources — never stop at one:\n - **Exa MCP** (`web_search_exa`) if available — best for broad discovery and finding relevant papers quickly\n - **Semantic Scholar** (`pip install semanticscholar`) — best for ML/AI papers, citation graphs, and specific paper lookup. See `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code examples\n - **arXiv** (`pip install arxiv`) — best for recent preprints and open-access papers\n - **CrossRef** — best for DOI lookup and BibTeX retrieval\n - Keep searching until you have good coverage. If one source comes up empty, try another with different keywords\n\n **Save everything to `literature/`**: For every paper you find, save a summary to `literature/` — title, authors, year, key findings, relevance to your question, and the URL/DOI. Create one file per paper and a running `literature/survey.md` with all summaries. This is your reference library — you and future sessions will need it throughout the project.\n\n2. **Identify gaps** from the literature\n - What's been tried? What hasn't? Where do existing methods break?\n - What do Discussion sections flag as future work?\n\n3. **Form initial hypotheses** — invoke `21-research-ideation/` skills\n - `brainstorming-research-ideas` for structured diverge-converge workflow\n - `creative-thinking-for-research` for deeper cognitive frameworks\n - Each hypothesis must be testable with a clear prediction\n\n4. **Define the evaluation**\n - Set the proxy metric and baseline before running experiments\n - The metric should be computable quickly (minutes, not hours)\n - Lock evaluation criteria upfront to prevent unconscious metric gaming\n\n5. **Record** in research-state.yaml, log the bootstrap in research-log.md\n\n## The Inner Loop\n\nRapid iteration with clear measurable outcomes. Two flavors:\n\n- **Optimization**: make a metric go up/down (val_loss, accuracy, throughput). Think Karpathy's autoresearch.\n- **Discovery**: test mechanistic hypotheses about why something works. The metric is a measurement (does grokking happen faster? does entropy increase before forgetting?), not just a target to optimize.\n\n```\n1. Pick the highest-priority untested hypothesis\n2. Write a protocol: what change, what prediction, why\n Lock it: commit to git BEFORE running (research(protocol): {hypothesis})\n This creates temporal proof your plan existed before results\n3. Run the experiment (invoke the relevant domain skill)\n4. Sanity check before trusting results:\n - Did training converge? No NaN/Inf?\n - Does baseline reproduce expected performance?\n - Data loading correct? (spot-check a few samples)\n5. Measure the proxy metric\n6. Record in experiments/{hypothesis-slug}/\n Label clearly: CONFIRMATORY (in your protocol) vs EXPLORATORY (discovered during execution)\n7. If positive: keep, note WHY it worked\n8. If negative: this is progress — note what it rules out and what it suggests\n9. Update research-state.yaml\n10. If stuck: search literature or invoke ideation skills — don't just keep trying random things\n```\n\n**Never stop.** Even if something fails, find a path forward. Debug, adjust, simplify, or pivot — but keep the research moving. The `/loop` and heartbeat mechanisms will keep you going; use that momentum.\n\n### Route to Domain Skills\n\nWhen you need domain-specific execution, search the skills library:\n\n| Research Activity | Look In |\n|---|---|\n| Data preparation | `05-data-processing/` |\n| Model training / fine-tuning | `01-model-architecture/`, `03-fine-tuning/`, `06-post-training/` |\n| Distributed training | `08-distributed-training/` |\n| Optimization (quantization, attention) | `10-optimization/` |\n| Evaluation / benchmarks | `11-evaluation/` |\n| Inference / serving | `12-inference-serving/` |\n| Interpretability analysis | `04-mechanistic-interpretability/` |\n| Experiment tracking (W&B, MLflow) | `13-mlops/` |\n| Cloud compute | `09-infrastructure/` |\n\nRead the relevant SKILL.md before starting — it has workflows, common issues, and code examples. See [references/skill-routing.md](references/skill-routing.md) for a complete guide.\n\n### Track the Experiment Trajectory\n\nMaintain a running record of measurable outcomes across experiments:\n\n```json\n{\n \"experiment_id\": \"run_014\",\n \"hypothesis\": \"H3\",\n \"metric_value\": 0.847,\n \"baseline\": 0.812,\n \"delta\": \"+0.035\",\n \"wall_time_min\": 23,\n \"change_summary\": \"Added cosine annealing warmup schedule\"\n}\n```\n\nThis trajectory produces the optimization plot (like Karpathy's progress chart) — include it in progress reports. Humans love seeing the upward curve.\n\n## The Outer Loop\n\nStep back from individual experiments. Synthesize.\n\n```\n1. Review all results since last reflection\n2. Cluster by type: what kinds of changes worked? Which didn't?\n3. Ask WHY — identify the mechanism behind successes and failures\n4. Update findings.md with current understanding\n5. Search literature if results were surprising or assumptions need revisiting\n6. Generate new hypotheses if warranted (invoke 21-research-ideation/ skills)\n7. Decide direction (see criteria below)\n8. Update research-state.yaml with new direction\n9. Log the reflection in research-log.md\n10. If there's something meaningful, generate a progress presentation\n```\n\n### Deciding Direction\n\nDon't just pick randomly — use these criteria:\n\n**DEEPEN** — a supported result raises follow-up questions\n- Does the effect hold under different conditions? What's the mechanism?\n- Action: generate sub-hypotheses (H1.1, H1.2) → back to inner loop\n\n**BROADEN** — current results are solid, but adjacent questions are untested\n- New questions emerged. The current contribution is clear but more is possible.\n- Action: generate new root hypotheses → back to inner loop\n\n**PIVOT** — results invalidate key assumptions or something more interesting appeared\n- A core assumption was wrong, or an unexpected finding is more promising than the original question.\n- Action: return to literature with new questions → re-bootstrap\n\n**CONCLUDE** — sufficient evidence for a contribution\n- At least one hypothesis is strongly supported (or a coherent set of negative results)\n- Key ablations completed, error analysis done\n- findings.md reads like a paper backbone — a human could write the abstract from it\n- No critical open questions that would change the story\n\nNote: coherent negative results are a valid contribution. \"X does NOT work because Y\" is publishable if the reasoning is rigorous.\n\n### findings.md Is Your Project Memory\n\nThis file serves two purposes: it's the research narrative for humans AND your accumulated knowledge base as an agent. Read it at the start of every session, /loop tick, or heartbeat to remember what you've learned.\n\nAfter every outer loop, update it to answer:\n\n- What do we know so far? (Current Understanding)\n- What patterns explain our results? (Patterns and Insights)\n- What specific things did we learn not to repeat? (Lessons and Constraints)\n- What remains open? (Open Questions)\n\nThe \"Lessons and Constraints\" section is especially important — it captures specific actionable learnings like \"weight decay > 0.1 diverges at this scale\" or \"baseline only reproduces with batch_size=64.\" This prevents the agent from repeating failed approaches across sessions.\n\n**Quality test**: After 30 inner loop experiments, a human should be able to read findings.md and write a paper abstract from it. If they can't, the outer loop isn't synthesizing — it's just logging.\n\n## Agent Continuity (MANDATORY — Set Up First)\n\n**Before doing anything else**, set up the wall-clock loop. This is what keeps the research running continuously. Without it, the agent stops after one cycle.\n\n### Claude Code\n\n**First action — run this immediately:**\n\n```\n/loop 20m Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a report in to_human/ and opening the HTML/PDF. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\n```\n\nThis fires every 20 minutes regardless of what's happening. It's a rhythm that keeps you working — not a research phase boundary. If your previous work isn't done, just continue it.\n\n### OpenClaw\n\n**First action — set up a 20-minute cron job in the current session. You MUST do this before any research work.**\n\nUse the `cron.add` tool to create a recurring job bound to this chat session:\n\n```json\n{\n \"name\": \"autoresearch-loop\",\n \"schedule\": { \"kind\": \"every\", \"everyMs\": 1200000 },\n \"sessionTarget\": \"current\",\n \"payload\": {\n \"kind\": \"agentTurn\",\n \"message\": \"Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a PDF report in to_human/ and sending it to the user via Telegram, WhatsApp, or Slack. When you get an exciting result or interesting plot, text it to the user right away — don't wait for a full report. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\"\n }\n}\n```\n\nKey details:\n- `sessionTarget: \"current\"` binds the cron to this chat session so it maintains conversation context across ticks\n- `everyMs: 1200000` = 20 minutes\n- After creation, verify with `cron.list` that the job exists and is enabled\n- If the cron fires while you're mid-experiment, just continue — the tick is a nudge, not a restart\n\n### What the Loop Does\n\nThe `/loop` and cron job are purely **wall-clock rhythm**. They are completely separate from your research loops (inner/outer). On each tick:\n\n1. Read `research-state.yaml` and `findings.md` — remember where you are\n2. Check if anything is broken (failed experiments, stalled training, errors)\n3. If on track → keep working on whatever you were doing\n4. If stuck or something's wrong → step back, diagnose, fix, then continue\n5. Never idle. Always be making progress.\n\n## Progress Reporting\n\nWhen you have something meaningful to share, create a research presentation — not just a status dashboard, but a compelling story.\n\n**When to report** (your judgment):\n- After an outer loop that found a significant pattern\n- When the optimization trajectory shows clear progress (include the plot!)\n- After a pivot in direction\n- Before requesting human input on a decision\n- When concluding\n\n**What to include** (adapt to what's compelling):\n- The research question and why it matters\n- Key results with visualizations (plots, metric tables)\n- The optimization trajectory chart (metric over experiments)\n- What was tried and why (selective, not exhaustive)\n- Current understanding (the findings narrative)\n- What's planned next\n\nFor Claude Code: generate HTML and `open` it. If HTML fails to open or render, convert to PDF as fallback (use `weasyprint`, `playwright pdf`, or `wkhtmltopdf`). For OpenClaw: generate PDF directly.\n\nSee [references/progress-reporting.md](references/progress-reporting.md) for template scaffolding and the optimization plot approach. Use the template as a starting point — be creative with what you show.\n\n## Git Protocol\n\nCommit at natural research milestones:\n\n| When | Message Pattern |\n|---|---|\n| Workspace initialized | `research(init): {project} — {question}` |\n| Experiment protocol locked | `research(protocol): {hypothesis}` |\n| Significant results | `research(results): {hypothesis} — {outcome}` |\n| Outer loop direction change | `research(reflect): {direction} — {reason}` |\n| Paper draft complete | `research(paper): {title}` |\n\n**Hard rule**: Protocol commits MUST precede result commits. Never combine them. The git history is your lightweight pre-registration — it proves what you planned before you saw results. Don't commit after every experiment — commit when there's meaningful progress.\n\n## Concluding: Paper Writing\n\nWhen the outer loop decides to CONCLUDE:\n\n1. Ensure findings.md has a clear, well-supported narrative\n2. Study 2-3 top related papers to learn their format, style, and section structure\n3. Invoke the `20-ml-paper-writing` skill — it has LaTeX templates for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, and systems venues\n4. Feed it the accumulated literature, experimental results, and findings\n5. Follow its citation verification workflow — never hallucinate references\n6. Generate a final comprehensive research presentation\n\nProceed autonomously through the writing process. If the ml-paper-writing skill suggests human collaboration points, adapt and keep going — produce the best draft you can. The human will review and provide feedback.\n\n## Research Discipline\n\nPrinciples to enforce continuously — not tied to any specific phase:\n\n- **Lock before you run**: Commit your experiment protocol to git before executing. This proves your plan existed before you saw results. Never combine protocol + results in one commit.\n- **Confirmatory vs exploratory**: Results matching your locked protocol are confirmatory. Everything else is exploratory — interesting but requiring more skepticism.\n- **Negative results are progress**: A refuted hypothesis tells you something. Log what it rules out and what it suggests. Don't treat it as failure.\n- **Sanity check before analysis**: Verify training converged, baselines reproduce, and data is correct before trusting your primary metric.\n- **Return to literature when confused**: Don't guess — search. If results surprise you or assumptions break, go find papers. Use Exa MCP for discovery, Semantic Scholar for specific ML/AI paper lookup, arXiv for preprints.\n- **Never stop**: Don't wait for human approval on routine decisions. If a skill or tool suggests collaboration, adapt and keep going. Find the best path forward autonomously. The human will see your progress reports and can redirect if needed.\n- **Use whatever compute is available**: Adapt to the user's environment — local GPU, cluster job submission, cloud instances, or just CPU. If no GPU is available, use CPU and adjust experiment scale accordingly. Don't block on compute availability.\n\n## Quality Standards\n\n**Good agent behavior:**\n- Hypotheses have mechanistic reasoning (\"X because Y, predicting Z\"), not just \"try X\"\n- findings.md builds a coherent narrative, not a flat list of results\n- Negative results are recorded with what they rule out\n- The agent updates its model when experiments contradict expectations\n- Progress reports tell a research story with compelling visualizations\n\n**Bad agent behavior:**\n- Pure hyperparameter sweeps without interpretation\n- findings.md is just experiment logs copy-pasted\n- Agent never revisits its assumptions after failures\n- Optimizing metrics without understanding why changes work\n\n## When to Use vs Alternatives\n\n**Use autoresearch when:**\n- You have a research question explorable through experiments\n- There's a measurable proxy metric for inner loop optimization\n- The real contribution requires synthesis beyond the metric\n- You want continuous autonomous research operation\n\n**Use individual domain skills instead when:**\n- You have a specific one-off task (train a model, run eval, write a paper)\n- No iterative experimentation needed\n\n## Common Issues\n\n**Inner loop stalls (no metric improvement)**\nRun an outer loop. Is the metric the right one? Is the search space exhausted? Consider broadening or pivoting. Search literature for new approaches.\n\n**Stuck and not making progress**\nDon't keep trying random changes. Step back: search literature for related work, invoke `21-research-ideation/` brainstorming skills, or run an outer loop reflection. Being stuck means you need new information or a new perspective, not more experiments.\n\n**Results contradict baseline expectations**\nInvestigate, don't ignore. Return to literature — your protocol might have an error, the published baseline may be wrong, or conditions differ. Update findings.md with what you learn.\n\n**Agent loses context between ticks**\nEnsure research-state.yaml and findings.md are updated after every action. These files are your memory across sessions.\n\n**Can't find relevant papers**\nTry multiple approaches in order: Exa MCP for broad search, Semantic Scholar for specific ML/AI paper lookup (`pip install semanticscholar`), arXiv for preprints (`pip install arxiv`). Check `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code. Note: Google Scholar has no official API — use Semantic Scholar instead for programmatic search.\n\n**No GPU available**\nUse CPU and scale experiments down. Many research tasks (analysis, interpretability, small model training) run fine on CPU. Adjust experiment design to fit available compute rather than blocking.\n\n**Experiments take longer than /loop interval**\nNormal. On the next tick, check if it finished. If not, keep waiting or do something else useful (update notes, search papers). Adjust interval if needed.\n\n**Not sure when to conclude**\nThree questions: Do you have a strongly supported finding? Can you explain WHY it works? Would findings.md make a convincing paper abstract? If yes to all: conclude.\n\n## Advanced Topics\n\n- **Detailed agent continuity**: [references/agent-continuity.md](references/agent-continuity.md)\n- **Progress presentation templates**: [references/progress-reporting.md](references/progress-reporting.md)\n- **Complete skill routing**: [references/skill-routing.md](references/skill-routing.md)\n","03-fine-tuning/unsloth/SKILL.md":"---\nname: unsloth\ndescription: Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, Unsloth, Fast Training, LoRA, QLoRA, Memory-Efficient, Optimization, Llama, Mistral, Gemma, Qwen]\ndependencies: [unsloth, torch, transformers, trl, datasets, peft]\n---\n\n# Unsloth Skill\n\nComprehensive assistance with unsloth development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with unsloth\n- Asking about unsloth features or APIs\n- Implementing unsloth solutions\n- Debugging unsloth code\n- Learning unsloth best practices\n\n## Quick Reference\n\n### Common Patterns\n\n*Quick reference patterns will be added as you use the skill.*\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **llms-txt.md** - Llms-Txt documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n\n\n\n","03-fine-tuning/llama-factory/SKILL.md":"---\nname: llama-factory\ndescription: Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, LLaMA Factory, LLM, WebUI, No-Code, QLoRA, LoRA, Multimodal, HuggingFace, Llama, Qwen, Gemma]\ndependencies: [llmtuner, torch, transformers, datasets, peft, accelerate, gradio]\n---\n\n# Llama-Factory Skill\n\nComprehensive assistance with llama-factory development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with llama-factory\n- Asking about llama-factory features or APIs\n- Implementing llama-factory solutions\n- Debugging llama-factory code\n- Learning llama-factory best practices\n\n## Quick Reference\n\n### Common Patterns\n\n*Quick reference patterns will be added as you use the skill.*\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **_images.md** - Images documentation\n- **advanced.md** - Advanced documentation\n- **getting_started.md** - Getting Started documentation\n- **other.md** - Other documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n","02-tokenization/sentencepiece/SKILL.md":"---\nname: sentencepiece\ndescription: Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Tokenization, SentencePiece, Language-Independent, BPE, Unigram, Multilingual, CJK Languages, Unicode, Deterministic, Google]\ndependencies: [sentencepiece, transformers]\n---\n\n# SentencePiece - Language-Independent Tokenization\n\nUnsupervised tokenizer that works on raw text without language-specific preprocessing.\n\n## When to use SentencePiece\n\n**Use SentencePiece when:**\n- Building multilingual models (no language-specific rules)\n- Working with CJK languages (Chinese, Japanese, Korean)\n- Need reproducible tokenization (deterministic vocabulary)\n- Want to train on raw text (no pre-tokenization needed)\n- Require lightweight deployment (6MB memory, 50k sentences/sec)\n\n**Performance**:\n- **Speed**: 50,000 sentences/sec\n- **Memory**: ~6MB for loaded model\n- **Languages**: All (language-independent)\n\n**Use alternatives instead**:\n- **HuggingFace Tokenizers**: Faster training, more flexibility\n- **tiktoken**: OpenAI models (GPT-3.5/4)\n- **BERT WordPiece**: English-centric tasks\n\n## Quick start\n\n### Installation\n\n```bash\n# Python\npip install sentencepiece\n\n# C++ (requires CMake)\ngit clone https://github.com/google/sentencepiece.git\ncd sentencepiece\nmkdir build && cd build\ncmake .. && make -j $(nproc)\nsudo make install\n```\n\n### Train model\n\n```bash\n# Command-line (BPE with 8000 vocab)\nspm_train --input=data.txt --model_prefix=m --vocab_size=8000 --model_type=bpe\n\n# Python API\nimport sentencepiece as spm\n\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='m',\n vocab_size=8000,\n model_type='bpe'\n)\n```\n\n**Training time**: ~1-2 minutes for 100MB corpus\n\n### Encode and decode\n\n```python\nimport sentencepiece as spm\n\n# Load model\nsp = spm.SentencePieceProcessor(model_file='m.model')\n\n# Encode to pieces\npieces = sp.encode('This is a test', out_type=str)\nprint(pieces) # ['▁This', '▁is', '▁a', '▁test']\n\n# Encode to IDs\nids = sp.encode('This is a test', out_type=int)\nprint(ids) # [284, 47, 11, 1243]\n\n# Decode\ntext = sp.decode(ids)\nprint(text) # \"This is a test\"\n```\n\n## Language-independent design\n\n### Whitespace as symbol (▁)\n\n```python\ntext = \"Hello world\"\npieces = sp.encode(text, out_type=str)\nprint(pieces) # ['▁Hello', '▁world']\n\n# Decode preserves spaces\ndecoded = sp.decode_pieces(pieces)\nprint(decoded) # \"Hello world\"\n```\n\n**Key principle**: Treat text as raw Unicode, whitespace = ▁ (meta symbol)\n\n## Tokenization algorithms\n\n### BPE (Byte-Pair Encoding)\n\n```python\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='bpe_model',\n vocab_size=16000,\n model_type='bpe'\n)\n```\n\n**Used by**: mBART\n\n### Unigram (default)\n\n```python\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='unigram_model',\n vocab_size=8000,\n model_type='unigram'\n)\n```\n\n**Used by**: T5, ALBERT, XLNet\n\n## Training configuration\n\n### Essential parameters\n\n```python\nspm.SentencePieceTrainer.train(\n input='corpus.txt',\n model_prefix='m',\n vocab_size=32000,\n model_type='unigram',\n character_coverage=0.9995, # 1.0 for CJK\n user_defined_symbols=['[SEP]', '[CLS]'],\n unk_piece='',\n num_threads=16\n)\n```\n\n### Character coverage\n\n| Language Type | Coverage | Rationale |\n|---------------|----------|-----------|\n| English | 0.9995 | Most common chars |\n| CJK (Chinese) | 1.0 | All characters needed |\n| Multilingual | 0.9995 | Balance |\n\n## Encoding options\n\n### Subword regularization\n\n```python\n# Sample different tokenizations\nfor _ in range(3):\n pieces = sp.encode('tokenization', out_type=str, enable_sampling=True, alpha=0.1)\n print(pieces)\n\n# Output (different each time):\n# ['▁token', 'ization']\n# ['▁tok', 'en', 'ization']\n```\n\n**Use case**: Data augmentation for robustness.\n\n## Common patterns\n\n### T5-style training\n\n```python\nspm.SentencePieceTrainer.train(\n input='c4_corpus.txt',\n model_prefix='t5',\n vocab_size=32000,\n model_type='unigram',\n user_defined_symbols=[f'' for i in range(100)],\n unk_id=2,\n eos_id=1,\n pad_id=0\n)\n```\n\n### Integration with transformers\n\n```python\nfrom transformers import T5Tokenizer\n\n# T5 uses SentencePiece internally\ntokenizer = T5Tokenizer.from_pretrained('t5-base')\ninputs = tokenizer('translate English to French: Hello', return_tensors='pt')\n```\n\n## Performance benchmarks\n\n### Training speed\n\n| Corpus | BPE (16k) | Unigram (8k) |\n|--------|-----------|--------------|\n| 100 MB | 1-2 min | 3-4 min |\n| 1 GB | 10-15 min | 30-40 min |\n\n### Tokenization speed\n\n- **SentencePiece**: 50,000 sentences/sec\n- **HF Tokenizers**: 200,000 sentences/sec (4× faster)\n\n## Supported models\n\n**T5 family**: `t5-base`, `t5-large` (32k vocab, Unigram)\n**ALBERT**: `albert-base-v2` (30k vocab, Unigram)\n**XLNet**: `xlnet-base-cased` (32k vocab, Unigram)\n**mBART**: `facebook/mbart-large-50` (250k vocab, BPE)\n\n## References\n\n- **[Training Guide](references/training.md)** - Detailed options, corpus preparation\n- **[Algorithms](references/algorithms.md)** - BPE vs Unigram, subword regularization\n\n## Resources\n\n- **GitHub**: https://github.com/google/sentencepiece ⭐ 10,000+\n- **Paper**: https://arxiv.org/abs/1808.06226 (EMNLP 2018)\n- **Version**: 0.2.0+\n\n\n","03-fine-tuning/axolotl/SKILL.md":"---\nname: axolotl\ndescription: Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, Axolotl, LLM, LoRA, QLoRA, DPO, KTO, ORPO, GRPO, YAML, HuggingFace, DeepSpeed, Multimodal]\ndependencies: [axolotl, torch, transformers, datasets, peft, accelerate, deepspeed]\n---\n\n# Axolotl Skill\n\nComprehensive assistance with axolotl development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with axolotl\n- Asking about axolotl features or APIs\n- Implementing axolotl solutions\n- Debugging axolotl code\n- Learning axolotl best practices\n\n## Quick Reference\n\n### Common Patterns\n\n**Pattern 1:** To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example:\n\n```\n./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3\n```\n\n**Pattern 2:** Configure your model to use FSDP in the Axolotl yaml. For example:\n\n```\nfsdp_version: 2\nfsdp_config:\n offload_params: true\n state_dict_type: FULL_STATE_DICT\n auto_wrap_policy: TRANSFORMER_BASED_WRAP\n transformer_layer_cls_to_wrap: LlamaDecoderLayer\n reshard_after_forward: true\n```\n\n**Pattern 3:** The context_parallel_size should be a divisor of the total number of GPUs. For example:\n\n```\ncontext_parallel_size\n```\n\n**Pattern 4:** For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and context_parallel_size=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU micro_batch_size is 2, the global batch size decreases from 16 to 4\n\n```\ncontext_parallel_size=4\n```\n\n**Pattern 5:** Setting save_compressed: true in your configuration enables saving models in a compressed format, which: - Reduces disk space usage by approximately 40% - Maintains compatibility with vLLM for accelerated inference - Maintains compatibility with llmcompressor for further optimization (example: quantization)\n\n```\nsave_compressed: true\n```\n\n**Pattern 6:** Note It is not necessary to place your integration in the integrations folder. It can be in any location, so long as it’s installed in a package in your python env. See this repo for an example: https://github.com/axolotl-ai-cloud/diff-transformer\n\n```\nintegrations\n```\n\n**Pattern 7:** Handle both single-example and batched data. - single example: sample[‘input_ids’] is a list[int] - batched data: sample[‘input_ids’] is a list[list[int]]\n\n```\nutils.trainer.drop_long_seq(sample, sequence_len=2048, min_sequence_len=2)\n```\n\n### Example Code Patterns\n\n**Example 1** (python):\n```python\ncli.cloud.modal_.ModalCloud(config, app=None)\n```\n\n**Example 2** (python):\n```python\ncli.cloud.modal_.run_cmd(cmd, run_folder, volumes=None)\n```\n\n**Example 3** (python):\n```python\ncore.trainers.base.AxolotlTrainer(\n *_args,\n bench_data_collator=None,\n eval_data_collator=None,\n dataset_tags=None,\n **kwargs,\n)\n```\n\n**Example 4** (python):\n```python\ncore.trainers.base.AxolotlTrainer.log(logs, start_time=None)\n```\n\n**Example 5** (python):\n```python\nprompt_strategies.input_output.RawInputOutputPrompter()\n```\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **api.md** - Api documentation\n- **dataset-formats.md** - Dataset-Formats documentation\n- **other.md** - Other documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n","03-fine-tuning/peft/SKILL.md":"---\nname: peft-fine-tuning\ndescription: Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, PEFT, LoRA, QLoRA, Parameter-Efficient, Adapters, Low-Rank, Memory Optimization, Multi-Adapter]\ndependencies: [peft>=0.13.0, transformers>=4.45.0, torch>=2.0.0, bitsandbytes>=0.43.0]\n---\n\n# PEFT (Parameter-Efficient Fine-Tuning)\n\nFine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods.\n\n## When to use PEFT\n\n**Use PEFT/LoRA when:**\n- Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100)\n- Need to train <1% parameters (6MB adapters vs 14GB full model)\n- Want fast iteration with multiple task-specific adapters\n- Deploying multiple fine-tuned variants from one base model\n\n**Use QLoRA (PEFT + quantization) when:**\n- Fine-tuning 70B models on single 24GB GPU\n- Memory is the primary constraint\n- Can accept ~5% quality trade-off vs full fine-tuning\n\n**Use full fine-tuning instead when:**\n- Training small models (<1B parameters)\n- Need maximum quality and have compute budget\n- Significant domain shift requires updating all weights\n\n## Quick start\n\n### Installation\n\n```bash\n# Basic installation\npip install peft\n\n# With quantization support (recommended)\npip install peft bitsandbytes\n\n# Full stack\npip install peft transformers accelerate bitsandbytes datasets\n```\n\n### LoRA fine-tuning (standard)\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer\nfrom peft import get_peft_model, LoraConfig, TaskType\nfrom datasets import load_dataset\n\n# Load base model\nmodel_name = \"meta-llama/Llama-3.1-8B\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=\"auto\", device_map=\"auto\")\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# LoRA configuration\nlora_config = LoraConfig(\n task_type=TaskType.CAUSAL_LM,\n r=16, # Rank (8-64, higher = more capacity)\n lora_alpha=32, # Scaling factor (typically 2*r)\n lora_dropout=0.05, # Dropout for regularization\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\"], # Attention layers\n bias=\"none\" # Don't train biases\n)\n\n# Apply LoRA\nmodel = get_peft_model(model, lora_config)\nmodel.print_trainable_parameters()\n# Output: trainable params: 13,631,488 || all params: 8,043,307,008 || trainable%: 0.17%\n\n# Prepare dataset\ndataset = load_dataset(\"databricks/databricks-dolly-15k\", split=\"train\")\n\ndef tokenize(example):\n text = f\"### Instruction:\\n{example['instruction']}\\n\\n### Response:\\n{example['response']}\"\n return tokenizer(text, truncation=True, max_length=512, padding=\"max_length\")\n\ntokenized = dataset.map(tokenize, remove_columns=dataset.column_names)\n\n# Training\ntraining_args = TrainingArguments(\n output_dir=\"./lora-llama\",\n num_train_epochs=3,\n per_device_train_batch_size=4,\n gradient_accumulation_steps=4,\n learning_rate=2e-4,\n fp16=True,\n logging_steps=10,\n save_strategy=\"epoch\"\n)\n\ntrainer = Trainer(\n model=model,\n args=training_args,\n train_dataset=tokenized,\n data_collator=lambda data: {\"input_ids\": torch.stack([f[\"input_ids\"] for f in data]),\n \"attention_mask\": torch.stack([f[\"attention_mask\"] for f in data]),\n \"labels\": torch.stack([f[\"input_ids\"] for f in data])}\n)\n\ntrainer.train()\n\n# Save adapter only (6MB vs 16GB)\nmodel.save_pretrained(\"./lora-llama-adapter\")\n```\n\n### QLoRA fine-tuning (memory-efficient)\n\n```python\nfrom transformers import AutoModelForCausalLM, BitsAndBytesConfig\nfrom peft import get_peft_model, LoraConfig, prepare_model_for_kbit_training\n\n# 4-bit quantization config\nbnb_config = BitsAndBytesConfig(\n load_in_4bit=True,\n bnb_4bit_quant_type=\"nf4\", # NormalFloat4 (best for LLMs)\n bnb_4bit_compute_dtype=\"bfloat16\", # Compute in bf16\n bnb_4bit_use_double_quant=True # Nested quantization\n)\n\n# Load quantized model\nmodel = AutoModelForCausalLM.from_pretrained(\n \"meta-llama/Llama-3.1-70B\",\n quantization_config=bnb_config,\n device_map=\"auto\"\n)\n\n# Prepare for training (enables gradient checkpointing)\nmodel = prepare_model_for_kbit_training(model)\n\n# LoRA config for QLoRA\nlora_config = LoraConfig(\n r=64, # Higher rank for 70B\n lora_alpha=128,\n lora_dropout=0.1,\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"],\n bias=\"none\",\n task_type=\"CAUSAL_LM\"\n)\n\nmodel = get_peft_model(model, lora_config)\n# 70B model now fits on single 24GB GPU!\n```\n\n## LoRA parameter selection\n\n### Rank (r) - capacity vs efficiency\n\n| Rank | Trainable Params | Memory | Quality | Use Case |\n|------|-----------------|--------|---------|----------|\n| 4 | ~3M | Minimal | Lower | Simple tasks, prototyping |\n| **8** | ~7M | Low | Good | **Recommended starting point** |\n| **16** | ~14M | Medium | Better | **General fine-tuning** |\n| 32 | ~27M | Higher | High | Complex tasks |\n| 64 | ~54M | High | Highest | Domain adaptation, 70B models |\n\n### Alpha (lora_alpha) - scaling factor\n\n```python\n# Rule of thumb: alpha = 2 * rank\nLoraConfig(r=16, lora_alpha=32) # Standard\nLoraConfig(r=16, lora_alpha=16) # Conservative (lower learning rate effect)\nLoraConfig(r=16, lora_alpha=64) # Aggressive (higher learning rate effect)\n```\n\n### Target modules by architecture\n\n```python\n# Llama / Mistral / Qwen\ntarget_modules = [\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"]\n\n# GPT-2 / GPT-Neo\ntarget_modules = [\"c_attn\", \"c_proj\", \"c_fc\"]\n\n# Falcon\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# BLOOM\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# Auto-detect all linear layers\ntarget_modules = \"all-linear\" # PEFT 0.6.0+\n```\n\n## Loading and merging adapters\n\n### Load trained adapter\n\n```python\nfrom peft import PeftModel, AutoPeftModelForCausalLM\nfrom transformers import AutoModelForCausalLM\n\n# Option 1: Load with PeftModel\nbase_model = AutoModelForCausalLM.from_pretrained(\"meta-llama/Llama-3.1-8B\")\nmodel = PeftModel.from_pretrained(base_model, \"./lora-llama-adapter\")\n\n# Option 2: Load directly (recommended)\nmodel = AutoPeftModelForCausalLM.from_pretrained(\n \"./lora-llama-adapter\",\n device_map=\"auto\"\n)\n```\n\n### Merge adapter into base model\n\n```python\n# Merge for deployment (no adapter overhead)\nmerged_model = model.merge_and_unload()\n\n# Save merged model\nmerged_model.save_pretrained(\"./llama-merged\")\ntokenizer.save_pretrained(\"./llama-merged\")\n\n# Push to Hub\nmerged_model.push_to_hub(\"username/llama-finetuned\")\n```\n\n### Multi-adapter serving\n\n```python\nfrom peft import PeftModel\n\n# Load base with first adapter\nmodel = AutoPeftModelForCausalLM.from_pretrained(\"./adapter-task1\")\n\n# Load additional adapters\nmodel.load_adapter(\"./adapter-task2\", adapter_name=\"task2\")\nmodel.load_adapter(\"./adapter-task3\", adapter_name=\"task3\")\n\n# Switch between adapters at runtime\nmodel.set_adapter(\"task1\") # Use task1 adapter\noutput1 = model.generate(**inputs)\n\nmodel.set_adapter(\"task2\") # Switch to task2\noutput2 = model.generate(**inputs)\n\n# Disable adapters (use base model)\nwith model.disable_adapter():\n base_output = model.generate(**inputs)\n```\n\n## PEFT methods comparison\n\n| Method | Trainable % | Memory | Speed | Best For |\n|--------|------------|--------|-------|----------|\n| **LoRA** | 0.1-1% | Low | Fast | General fine-tuning |\n| **QLoRA** | 0.1-1% | Very Low | Medium | Memory-constrained |\n| AdaLoRA | 0.1-1% | Low | Medium | Automatic rank selection |\n| IA3 | 0.01% | Minimal | Fastest | Few-shot adaptation |\n| Prefix Tuning | 0.1% | Low | Medium | Generation control |\n| Prompt Tuning | 0.001% | Minimal | Fast | Simple task adaptation |\n| P-Tuning v2 | 0.1% | Low | Medium | NLU tasks |\n\n### IA3 (minimal parameters)\n\n```python\nfrom peft import IA3Config\n\nia3_config = IA3Config(\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"down_proj\"],\n feedforward_modules=[\"down_proj\"]\n)\nmodel = get_peft_model(model, ia3_config)\n# Trains only 0.01% of parameters!\n```\n\n### Prefix Tuning\n\n```python\nfrom peft import PrefixTuningConfig\n\nprefix_config = PrefixTuningConfig(\n task_type=\"CAUSAL_LM\",\n num_virtual_tokens=20, # Prepended tokens\n prefix_projection=True # Use MLP projection\n)\nmodel = get_peft_model(model, prefix_config)\n```\n\n## Integration patterns\n\n### With TRL (SFTTrainer)\n\n```python\nfrom trl import SFTTrainer, SFTConfig\nfrom peft import LoraConfig\n\nlora_config = LoraConfig(r=16, lora_alpha=32, target_modules=\"all-linear\")\n\ntrainer = SFTTrainer(\n model=model,\n args=SFTConfig(output_dir=\"./output\", max_seq_length=512),\n train_dataset=dataset,\n peft_config=lora_config, # Pass LoRA config directly\n)\ntrainer.train()\n```\n\n### With Axolotl (YAML config)\n\n```yaml\n# axolotl config.yaml\nadapter: lora\nlora_r: 16\nlora_alpha: 32\nlora_dropout: 0.05\nlora_target_modules:\n - q_proj\n - v_proj\n - k_proj\n - o_proj\nlora_target_linear: true # Target all linear layers\n```\n\n### With vLLM (inference)\n\n```python\nfrom vllm import LLM\nfrom vllm.lora.request import LoRARequest\n\n# Load base model with LoRA support\nllm = LLM(model=\"meta-llama/Llama-3.1-8B\", enable_lora=True)\n\n# Serve with adapter\noutputs = llm.generate(\n prompts,\n lora_request=LoRARequest(\"adapter1\", 1, \"./lora-adapter\")\n)\n```\n\n## Performance benchmarks\n\n### Memory usage (Llama 3.1 8B)\n\n| Method | GPU Memory | Trainable Params |\n|--------|-----------|------------------|\n| Full fine-tuning | 60+ GB | 8B (100%) |\n| LoRA r=16 | 18 GB | 14M (0.17%) |\n| QLoRA r=16 | 6 GB | 14M (0.17%) |\n| IA3 | 16 GB | 800K (0.01%) |\n\n### Training speed (A100 80GB)\n\n| Method | Tokens/sec | vs Full FT |\n|--------|-----------|------------|\n| Full FT | 2,500 | 1x |\n| LoRA | 3,200 | 1.3x |\n| QLoRA | 2,100 | 0.84x |\n\n### Quality (MMLU benchmark)\n\n| Model | Full FT | LoRA | QLoRA |\n|-------|---------|------|-------|\n| Llama 2-7B | 45.3 | 44.8 | 44.1 |\n| Llama 2-13B | 54.8 | 54.2 | 53.5 |\n\n## Common issues\n\n### CUDA OOM during training\n\n```python\n# Solution 1: Enable gradient checkpointing\nmodel.gradient_checkpointing_enable()\n\n# Solution 2: Reduce batch size + increase accumulation\nTrainingArguments(\n per_device_train_batch_size=1,\n gradient_accumulation_steps=16\n)\n\n# Solution 3: Use QLoRA\nfrom transformers import BitsAndBytesConfig\nbnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\")\n```\n\n### Adapter not applying\n\n```python\n# Verify adapter is active\nprint(model.active_adapters) # Should show adapter name\n\n# Check trainable parameters\nmodel.print_trainable_parameters()\n\n# Ensure model in training mode\nmodel.train()\n```\n\n### Quality degradation\n\n```python\n# Increase rank\nLoraConfig(r=32, lora_alpha=64)\n\n# Target more modules\ntarget_modules = \"all-linear\"\n\n# Use more training data and epochs\nTrainingArguments(num_train_epochs=5)\n\n# Lower learning rate\nTrainingArguments(learning_rate=1e-4)\n```\n\n## Best practices\n\n1. **Start with r=8-16**, increase if quality insufficient\n2. **Use alpha = 2 * rank** as starting point\n3. **Target attention + MLP layers** for best quality/efficiency\n4. **Enable gradient checkpointing** for memory savings\n5. **Save adapters frequently** (small files, easy rollback)\n6. **Evaluate on held-out data** before merging\n7. **Use QLoRA for 70B+ models** on consumer hardware\n\n## References\n\n- **[Advanced Usage](references/advanced-usage.md)** - DoRA, LoftQ, rank stabilization, custom modules\n- **[Troubleshooting](references/troubleshooting.md)** - Common errors, debugging, optimization\n\n## Resources\n\n- **GitHub**: https://github.com/huggingface/peft\n- **Docs**: https://huggingface.co/docs/peft\n- **LoRA Paper**: arXiv:2106.09685\n- **QLoRA Paper**: arXiv:2305.14314\n- **Models**: https://huggingface.co/models?library=peft\n","04-mechanistic-interpretability/transformer-lens/SKILL.md":"# transformer-lens\n\nPath: `04-mechanistic-interpretability/transformer-lens/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/SKILL.md)","04-mechanistic-interpretability/saelens/SKILL.md":"# saelens\n\nPath: `04-mechanistic-interpretability/saelens/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/SKILL.md)","04-mechanistic-interpretability/pyvene/SKILL.md":"# pyvene\n\nPath: `04-mechanistic-interpretability/pyvene/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/SKILL.md)","04-mechanistic-interpretability/nnsight/SKILL.md":"# nnsight\n\nPath: `04-mechanistic-interpretability/nnsight/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/SKILL.md)","05-data-processing/nemo-curator/SKILL.md":"# nemo-curator\n\nPath: `05-data-processing/nemo-curator/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/SKILL.md)","05-data-processing/ray-data/SKILL.md":"# ray-data\n\nPath: `05-data-processing/ray-data/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/SKILL.md)","06-post-training/verl/SKILL.md":"# verl\n\nPath: `06-post-training/verl/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/SKILL.md)","06-post-training/trl-fine-tuning/SKILL.md":"# trl-fine-tuning\n\nPath: `06-post-training/trl-fine-tuning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/SKILL.md)","06-post-training/torchforge/SKILL.md":"# torchforge\n\nPath: `06-post-training/torchforge/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/SKILL.md)","06-post-training/slime/SKILL.md":"# slime\n\nPath: `06-post-training/slime/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/SKILL.md)","06-post-training/simpo/SKILL.md":"# simpo\n\nPath: `06-post-training/simpo/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/SKILL.md)","06-post-training/openrlhf/SKILL.md":"# openrlhf\n\nPath: `06-post-training/openrlhf/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/SKILL.md)","06-post-training/miles/SKILL.md":"# miles\n\nPath: `06-post-training/miles/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/SKILL.md)","06-post-training/grpo-rl-training/SKILL.md":"# grpo-rl-training\n\nPath: `06-post-training/grpo-rl-training/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/SKILL.md)","07-safety-alignment/constitutional-ai/SKILL.md":"# constitutional-ai\n\nPath: `07-safety-alignment/constitutional-ai/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/SKILL.md)","07-safety-alignment/llamaguard/SKILL.md":"# llamaguard\n\nPath: `07-safety-alignment/llamaguard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/SKILL.md)","07-safety-alignment/nemo-guardrails/SKILL.md":"# nemo-guardrails\n\nPath: `07-safety-alignment/nemo-guardrails/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/SKILL.md)","07-safety-alignment/prompt-guard/SKILL.md":"# prompt-guard\n\nPath: `07-safety-alignment/prompt-guard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/SKILL.md)","08-distributed-training/ray-train/SKILL.md":"# ray-train\n\nPath: `08-distributed-training/ray-train/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/SKILL.md)","08-distributed-training/pytorch-lightning/SKILL.md":"# pytorch-lightning\n\nPath: `08-distributed-training/pytorch-lightning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/SKILL.md)","08-distributed-training/pytorch-fsdp2/SKILL.md":"# pytorch-fsdp2\n\nPath: `08-distributed-training/pytorch-fsdp2/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/SKILL.md)","08-distributed-training/megatron-core/SKILL.md":"# megatron-core\n\nPath: `08-distributed-training/megatron-core/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/SKILL.md)","08-distributed-training/deepspeed/SKILL.md":"# deepspeed\n\nPath: `08-distributed-training/deepspeed/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/SKILL.md)","08-distributed-training/accelerate/SKILL.md":"# accelerate\n\nPath: `08-distributed-training/accelerate/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/SKILL.md)","09-infrastructure/lambda-labs/SKILL.md":"# lambda-labs\n\nPath: `09-infrastructure/lambda-labs/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/SKILL.md)","09-infrastructure/modal/SKILL.md":"# modal\n\nPath: `09-infrastructure/modal/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/SKILL.md)","09-infrastructure/skypilot/SKILL.md":"# skypilot\n\nPath: `09-infrastructure/skypilot/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/SKILL.md)","10-optimization/awq/SKILL.md":"# awq\n\nPath: `10-optimization/awq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/SKILL.md)","10-optimization/bitsandbytes/SKILL.md":"# bitsandbytes\n\nPath: `10-optimization/bitsandbytes/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/SKILL.md)","10-optimization/flash-attention/SKILL.md":"# flash-attention\n\nPath: `10-optimization/flash-attention/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/SKILL.md)","10-optimization/gguf/SKILL.md":"# gguf\n\nPath: `10-optimization/gguf/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/SKILL.md)","10-optimization/gptq/SKILL.md":"# gptq\n\nPath: `10-optimization/gptq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/SKILL.md)","10-optimization/hqq/SKILL.md":"# hqq\n\nPath: `10-optimization/hqq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/SKILL.md)","10-optimization/ml-training-recipes/SKILL.md":"# ml-training-recipes\n\nPath: `10-optimization/ml-training-recipes/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/SKILL.md)","11-evaluation/bigcode-evaluation-harness/SKILL.md":"# bigcode-evaluation-harness\n\nPath: `11-evaluation/bigcode-evaluation-harness/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/SKILL.md)","11-evaluation/lm-evaluation-harness/SKILL.md":"# lm-evaluation-harness\n\nPath: `11-evaluation/lm-evaluation-harness/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/SKILL.md)","11-evaluation/nemo-evaluator/SKILL.md":"# nemo-evaluator\n\nPath: `11-evaluation/nemo-evaluator/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/SKILL.md)","12-inference-serving/llama-cpp/SKILL.md":"# llama-cpp\n\nPath: `12-inference-serving/llama-cpp/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/SKILL.md)","12-inference-serving/sglang/SKILL.md":"# sglang\n\nPath: `12-inference-serving/sglang/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/SKILL.md)","12-inference-serving/tensorrt-llm/SKILL.md":"# tensorrt-llm\n\nPath: `12-inference-serving/tensorrt-llm/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/SKILL.md)","12-inference-serving/vllm/SKILL.md":"# vllm\n\nPath: `12-inference-serving/vllm/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/SKILL.md)","13-mlops/mlflow/SKILL.md":"# mlflow\n\nPath: `13-mlops/mlflow/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/SKILL.md)","13-mlops/swanlab/SKILL.md":"# swanlab\n\nPath: `13-mlops/swanlab/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/SKILL.md)","13-mlops/tensorboard/SKILL.md":"# tensorboard\n\nPath: `13-mlops/tensorboard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/SKILL.md)","13-mlops/weights-and-biases/SKILL.md":"# weights-and-biases\n\nPath: `13-mlops/weights-and-biases/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/SKILL.md)","14-agents/a-evolve/SKILL.md":"# a-evolve\n\nPath: `14-agents/a-evolve/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/SKILL.md)","14-agents/autogpt/SKILL.md":"# autogpt\n\nPath: `14-agents/autogpt/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/SKILL.md)","14-agents/crewai/SKILL.md":"# crewai\n\nPath: `14-agents/crewai/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/SKILL.md)","14-agents/langchain/SKILL.md":"# langchain\n\nPath: `14-agents/langchain/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/SKILL.md)","14-agents/llamaindex/SKILL.md":"# llamaindex\n\nPath: `14-agents/llamaindex/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/SKILL.md)","15-rag/chroma/SKILL.md":"# chroma\n\nPath: `15-rag/chroma/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/SKILL.md)","15-rag/faiss/SKILL.md":"# faiss\n\nPath: `15-rag/faiss/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/SKILL.md)","15-rag/pinecone/SKILL.md":"# pinecone\n\nPath: `15-rag/pinecone/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/SKILL.md)","15-rag/qdrant/SKILL.md":"# qdrant\n\nPath: `15-rag/qdrant/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/SKILL.md)","15-rag/sentence-transformers/SKILL.md":"# sentence-transformers\n\nPath: `15-rag/sentence-transformers/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/SKILL.md)","16-prompt-engineering/dspy/SKILL.md":"# dspy\n\nPath: `16-prompt-engineering/dspy/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/SKILL.md)","16-prompt-engineering/guidance/SKILL.md":"# guidance\n\nPath: `16-prompt-engineering/guidance/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/SKILL.md)","16-prompt-engineering/instructor/SKILL.md":"# instructor\n\nPath: `16-prompt-engineering/instructor/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/SKILL.md)","16-prompt-engineering/outlines/SKILL.md":"# outlines\n\nPath: `16-prompt-engineering/outlines/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/SKILL.md)","17-observability/langsmith/SKILL.md":"# langsmith\n\nPath: `17-observability/langsmith/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/SKILL.md)","17-observability/phoenix/SKILL.md":"# phoenix\n\nPath: `17-observability/phoenix/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/SKILL.md)","18-multimodal/audiocraft/SKILL.md":"# audiocraft\n\nPath: `18-multimodal/audiocraft/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/SKILL.md)","18-multimodal/blip-2/SKILL.md":"# blip-2\n\nPath: `18-multimodal/blip-2/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/SKILL.md)","18-multimodal/clip/SKILL.md":"# clip\n\nPath: `18-multimodal/clip/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/SKILL.md)","18-multimodal/cosmos-policy/SKILL.md":"# cosmos-policy\n\nPath: `18-multimodal/cosmos-policy/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/SKILL.md)","18-multimodal/llava/SKILL.md":"# llava\n\nPath: `18-multimodal/llava/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/SKILL.md)","18-multimodal/openpi/SKILL.md":"# openpi\n\nPath: `18-multimodal/openpi/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/SKILL.md)","18-multimodal/openvla-oft/SKILL.md":"# openvla-oft\n\nPath: `18-multimodal/openvla-oft/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/SKILL.md)","18-multimodal/segment-anything/SKILL.md":"# segment-anything\n\nPath: `18-multimodal/segment-anything/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/SKILL.md)","18-multimodal/stable-diffusion/SKILL.md":"# stable-diffusion\n\nPath: `18-multimodal/stable-diffusion/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/SKILL.md)","18-multimodal/whisper/SKILL.md":"# whisper\n\nPath: `18-multimodal/whisper/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/SKILL.md)","19-emerging-techniques/knowledge-distillation/SKILL.md":"# knowledge-distillation\n\nPath: `19-emerging-techniques/knowledge-distillation/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/SKILL.md)","19-emerging-techniques/long-context/SKILL.md":"# long-context\n\nPath: `19-emerging-techniques/long-context/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/SKILL.md)","19-emerging-techniques/model-merging/SKILL.md":"# model-merging\n\nPath: `19-emerging-techniques/model-merging/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/SKILL.md)","19-emerging-techniques/model-pruning/SKILL.md":"# model-pruning\n\nPath: `19-emerging-techniques/model-pruning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/SKILL.md)","19-emerging-techniques/moe-training/SKILL.md":"# moe-training\n\nPath: `19-emerging-techniques/moe-training/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/SKILL.md)","19-emerging-techniques/speculative-decoding/SKILL.md":"# speculative-decoding\n\nPath: `19-emerging-techniques/speculative-decoding/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/SKILL.md)","20-ml-paper-writing/systems-paper-writing/SKILL.md":"# systems-paper-writing\n\nPath: `20-ml-paper-writing/systems-paper-writing/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/SKILL.md)","20-ml-paper-writing/presenting-conference-talks/SKILL.md":"# presenting-conference-talks\n\nPath: `20-ml-paper-writing/presenting-conference-talks/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/SKILL.md)","20-ml-paper-writing/ml-paper-writing/SKILL.md":"# ml-paper-writing\n\nPath: `20-ml-paper-writing/ml-paper-writing/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/SKILL.md)","20-ml-paper-writing/academic-plotting/SKILL.md":"# academic-plotting\n\nPath: `20-ml-paper-writing/academic-plotting/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/SKILL.md)","21-research-ideation/creative-thinking-for-research/SKILL.md":"# creative-thinking-for-research\n\nPath: `21-research-ideation/creative-thinking-for-research/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/SKILL.md)","21-research-ideation/brainstorming-research-ideas/SKILL.md":"# brainstorming-research-ideas\n\nPath: `21-research-ideation/brainstorming-research-ideas/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/SKILL.md)","22-agent-native-research-artifact/rigor-reviewer/SKILL.md":"# rigor-reviewer\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/SKILL.md)","22-agent-native-research-artifact/research-manager/SKILL.md":"# research-manager\n\nPath: `22-agent-native-research-artifact/research-manager/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/SKILL.md)","22-agent-native-research-artifact/compiler/SKILL.md":"# compiler\n\nPath: `22-agent-native-research-artifact/compiler/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/SKILL.md)","CLAUDE.md":"# Claude Agent Guidelines & System Prompt\n\nPath: `CLAUDE.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/CLAUDE.md)",".claude-plugin/marketplace.json":"# Claude Plugin Marketplace Catalog\n\nPath: `.claude-plugin/marketplace.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/.claude-plugin/marketplace.json)","0-autoresearch-skill/README.md":"# readme.md\n\nPath: `0-autoresearch-skill/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/README.md)","0-autoresearch-skill/references/agent-continuity.md":"# References Skill\n\nPath: `0-autoresearch-skill/references/agent-continuity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/agent-continuity.md)","0-autoresearch-skill/references/progress-reporting.md":"# References Skill\n\nPath: `0-autoresearch-skill/references/progress-reporting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/progress-reporting.md)","0-autoresearch-skill/references/skill-routing.md":"# References Skill\n\nPath: `0-autoresearch-skill/references/skill-routing.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/skill-routing.md)","packages/ai-research-skills/package.json":"# Ai-research-skills Skill\n\nPath: `packages/ai-research-skills/package.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package.json)","packages/ai-research-skills/package-lock.json":"# Ai-research-skills Skill\n\nPath: `packages/ai-research-skills/package-lock.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package-lock.json)","packages/ai-research-skills/README.md":"# ai-research-skills Documentation\n\nPath: `packages/ai-research-skills/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/README.md)","01-model-architecture/litgpt/README.md":"# readme.md\n\nPath: `01-model-architecture/litgpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/README.md)","01-model-architecture/mamba/README.md":"# readme.md\n\nPath: `01-model-architecture/mamba/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/README.md)","01-model-architecture/nanogpt/README.md":"# readme.md\n\nPath: `01-model-architecture/nanogpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/README.md)","01-model-architecture/rwkv/README.md":"# readme.md\n\nPath: `01-model-architecture/rwkv/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/README.md)","01-model-architecture/torchtitan/README.md":"# readme.md\n\nPath: `01-model-architecture/torchtitan/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/README.md)","02-tokenization/huggingface-tokenizers/README.md":"# readme.md\n\nPath: `02-tokenization/huggingface-tokenizers/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/README.md)","02-tokenization/sentencepiece/README.md":"# readme.md\n\nPath: `02-tokenization/sentencepiece/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/README.md)","03-fine-tuning/axolotl/README.md":"# readme.md\n\nPath: `03-fine-tuning/axolotl/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/README.md)","03-fine-tuning/llama-factory/README.md":"# readme.md\n\nPath: `03-fine-tuning/llama-factory/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/README.md)","03-fine-tuning/peft/README.md":"# readme.md\n\nPath: `03-fine-tuning/peft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/README.md)","03-fine-tuning/unsloth/README.md":"# readme.md\n\nPath: `03-fine-tuning/unsloth/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/README.md)","04-mechanistic-interpretability/nnsight/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/nnsight/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/README.md)","04-mechanistic-interpretability/pyvene/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/pyvene/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/README.md)","04-mechanistic-interpretability/saelens/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/saelens/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/README.md)","04-mechanistic-interpretability/transformer-lens/README.md":"# readme.md\n\nPath: `04-mechanistic-interpretability/transformer-lens/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/README.md)","05-data-processing/nemo-curator/README.md":"# readme.md\n\nPath: `05-data-processing/nemo-curator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/README.md)","05-data-processing/ray-data/README.md":"# readme.md\n\nPath: `05-data-processing/ray-data/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/README.md)","06-post-training/grpo-rl-training/README.md":"# readme.md\n\nPath: `06-post-training/grpo-rl-training/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/README.md)","06-post-training/miles/README.md":"# readme.md\n\nPath: `06-post-training/miles/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/README.md)","06-post-training/openrlhf/README.md":"# readme.md\n\nPath: `06-post-training/openrlhf/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/README.md)","06-post-training/simpo/README.md":"# readme.md\n\nPath: `06-post-training/simpo/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/README.md)","06-post-training/slime/README.md":"# readme.md\n\nPath: `06-post-training/slime/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/README.md)","06-post-training/torchforge/README.md":"# readme.md\n\nPath: `06-post-training/torchforge/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/README.md)","06-post-training/trl-fine-tuning/README.md":"# readme.md\n\nPath: `06-post-training/trl-fine-tuning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/README.md)","06-post-training/verl/README.md":"# readme.md\n\nPath: `06-post-training/verl/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/README.md)","07-safety-alignment/constitutional-ai/README.md":"# readme.md\n\nPath: `07-safety-alignment/constitutional-ai/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/README.md)","07-safety-alignment/llamaguard/README.md":"# readme.md\n\nPath: `07-safety-alignment/llamaguard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/README.md)","07-safety-alignment/nemo-guardrails/README.md":"# readme.md\n\nPath: `07-safety-alignment/nemo-guardrails/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/README.md)","07-safety-alignment/prompt-guard/README.md":"# readme.md\n\nPath: `07-safety-alignment/prompt-guard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/README.md)","08-distributed-training/accelerate/README.md":"# readme.md\n\nPath: `08-distributed-training/accelerate/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/README.md)","08-distributed-training/deepspeed/README.md":"# readme.md\n\nPath: `08-distributed-training/deepspeed/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/README.md)","08-distributed-training/megatron-core/README.md":"# readme.md\n\nPath: `08-distributed-training/megatron-core/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/README.md)","08-distributed-training/pytorch-fsdp2/README.md":"# readme.md\n\nPath: `08-distributed-training/pytorch-fsdp2/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/README.md)","08-distributed-training/pytorch-lightning/README.md":"# readme.md\n\nPath: `08-distributed-training/pytorch-lightning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/README.md)","08-distributed-training/ray-train/README.md":"# readme.md\n\nPath: `08-distributed-training/ray-train/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/README.md)","09-infrastructure/lambda-labs/README.md":"# readme.md\n\nPath: `09-infrastructure/lambda-labs/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/README.md)","09-infrastructure/modal/README.md":"# readme.md\n\nPath: `09-infrastructure/modal/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/README.md)","09-infrastructure/skypilot/README.md":"# readme.md\n\nPath: `09-infrastructure/skypilot/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/README.md)","10-optimization/awq/README.md":"# readme.md\n\nPath: `10-optimization/awq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/README.md)","10-optimization/bitsandbytes/README.md":"# readme.md\n\nPath: `10-optimization/bitsandbytes/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/README.md)","10-optimization/flash-attention/README.md":"# readme.md\n\nPath: `10-optimization/flash-attention/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/README.md)","10-optimization/gguf/README.md":"# readme.md\n\nPath: `10-optimization/gguf/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/README.md)","10-optimization/gptq/README.md":"# readme.md\n\nPath: `10-optimization/gptq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/README.md)","10-optimization/hqq/README.md":"# readme.md\n\nPath: `10-optimization/hqq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/README.md)","10-optimization/ml-training-recipes/README.md":"# readme.md\n\nPath: `10-optimization/ml-training-recipes/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/README.md)","11-evaluation/bigcode-evaluation-harness/README.md":"# readme.md\n\nPath: `11-evaluation/bigcode-evaluation-harness/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/README.md)","11-evaluation/lm-evaluation-harness/README.md":"# readme.md\n\nPath: `11-evaluation/lm-evaluation-harness/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/README.md)","11-evaluation/nemo-evaluator/README.md":"# readme.md\n\nPath: `11-evaluation/nemo-evaluator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/README.md)","12-inference-serving/llama-cpp/README.md":"# readme.md\n\nPath: `12-inference-serving/llama-cpp/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/README.md)","12-inference-serving/sglang/README.md":"# readme.md\n\nPath: `12-inference-serving/sglang/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/README.md)","12-inference-serving/tensorrt-llm/README.md":"# readme.md\n\nPath: `12-inference-serving/tensorrt-llm/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/README.md)","12-inference-serving/vllm/README.md":"# readme.md\n\nPath: `12-inference-serving/vllm/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/README.md)","13-mlops/mlflow/README.md":"# readme.md\n\nPath: `13-mlops/mlflow/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/README.md)","13-mlops/tensorboard/README.md":"# readme.md\n\nPath: `13-mlops/tensorboard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/README.md)","13-mlops/weights-and-biases/README.md":"# readme.md\n\nPath: `13-mlops/weights-and-biases/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/README.md)","14-agents/autogpt/README.md":"# readme.md\n\nPath: `14-agents/autogpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/README.md)","14-agents/crewai/README.md":"# readme.md\n\nPath: `14-agents/crewai/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/README.md)","14-agents/langchain/README.md":"# readme.md\n\nPath: `14-agents/langchain/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/README.md)","14-agents/llamaindex/README.md":"# readme.md\n\nPath: `14-agents/llamaindex/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/README.md)","15-rag/chroma/README.md":"# readme.md\n\nPath: `15-rag/chroma/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/README.md)","15-rag/faiss/README.md":"# readme.md\n\nPath: `15-rag/faiss/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/README.md)","15-rag/pinecone/README.md":"# readme.md\n\nPath: `15-rag/pinecone/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/README.md)","15-rag/qdrant/README.md":"# readme.md\n\nPath: `15-rag/qdrant/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/README.md)","15-rag/sentence-transformers/README.md":"# readme.md\n\nPath: `15-rag/sentence-transformers/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/README.md)","16-prompt-engineering/dspy/README.md":"# readme.md\n\nPath: `16-prompt-engineering/dspy/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/README.md)","16-prompt-engineering/guidance/README.md":"# readme.md\n\nPath: `16-prompt-engineering/guidance/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/README.md)","16-prompt-engineering/instructor/README.md":"# readme.md\n\nPath: `16-prompt-engineering/instructor/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/README.md)","16-prompt-engineering/outlines/README.md":"# readme.md\n\nPath: `16-prompt-engineering/outlines/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/README.md)","17-observability/langsmith/README.md":"# readme.md\n\nPath: `17-observability/langsmith/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/README.md)","17-observability/phoenix/README.md":"# readme.md\n\nPath: `17-observability/phoenix/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/README.md)","18-multimodal/audiocraft/README.md":"# readme.md\n\nPath: `18-multimodal/audiocraft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/README.md)","18-multimodal/blip-2/README.md":"# readme.md\n\nPath: `18-multimodal/blip-2/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/README.md)","18-multimodal/clip/README.md":"# readme.md\n\nPath: `18-multimodal/clip/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/README.md)","18-multimodal/cosmos-policy/README.md":"# readme.md\n\nPath: `18-multimodal/cosmos-policy/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/README.md)","18-multimodal/llava/README.md":"# readme.md\n\nPath: `18-multimodal/llava/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/README.md)","18-multimodal/openpi/README.md":"# readme.md\n\nPath: `18-multimodal/openpi/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/README.md)","18-multimodal/openvla-oft/README.md":"# readme.md\n\nPath: `18-multimodal/openvla-oft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/README.md)","18-multimodal/segment-anything/README.md":"# readme.md\n\nPath: `18-multimodal/segment-anything/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/README.md)","18-multimodal/stable-diffusion/README.md":"# readme.md\n\nPath: `18-multimodal/stable-diffusion/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/README.md)","18-multimodal/whisper/README.md":"# readme.md\n\nPath: `18-multimodal/whisper/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/README.md)","19-emerging-techniques/knowledge-distillation/README.md":"# readme.md\n\nPath: `19-emerging-techniques/knowledge-distillation/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/README.md)","19-emerging-techniques/long-context/README.md":"# readme.md\n\nPath: `19-emerging-techniques/long-context/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/README.md)","19-emerging-techniques/model-merging/README.md":"# readme.md\n\nPath: `19-emerging-techniques/model-merging/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/README.md)","19-emerging-techniques/model-pruning/README.md":"# readme.md\n\nPath: `19-emerging-techniques/model-pruning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/README.md)","19-emerging-techniques/moe-training/README.md":"# readme.md\n\nPath: `19-emerging-techniques/moe-training/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/README.md)","19-emerging-techniques/speculative-decoding/README.md":"# readme.md\n\nPath: `19-emerging-techniques/speculative-decoding/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/README.md)","20-ml-paper-writing/ml-paper-writing/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/ml-paper-writing/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/README.md)","20-ml-paper-writing/academic-plotting/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/academic-plotting/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/README.md)","20-ml-paper-writing/systems-paper-writing/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/systems-paper-writing/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/README.md)","20-ml-paper-writing/presenting-conference-talks/README.md":"# readme.md\n\nPath: `20-ml-paper-writing/presenting-conference-talks/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/README.md)","21-research-ideation/brainstorming-research-ideas/README.md":"# readme.md\n\nPath: `21-research-ideation/brainstorming-research-ideas/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/README.md)","21-research-ideation/creative-thinking-for-research/README.md":"# readme.md\n\nPath: `21-research-ideation/creative-thinking-for-research/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/README.md)","22-agent-native-research-artifact/compiler/README.md":"# readme.md\n\nPath: `22-agent-native-research-artifact/compiler/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/README.md)","22-agent-native-research-artifact/research-manager/README.md":"# readme.md\n\nPath: `22-agent-native-research-artifact/research-manager/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/README.md)","22-agent-native-research-artifact/rigor-reviewer/README.md":"# readme.md\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/README.md)","01-model-architecture/litgpt/references/custom-models.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/custom-models.md)","01-model-architecture/litgpt/references/distributed-training.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/distributed-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/distributed-training.md)","01-model-architecture/litgpt/references/supported-models.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/supported-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/supported-models.md)","01-model-architecture/litgpt/references/training-recipes.md":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/training-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/training-recipes.md)","01-model-architecture/mamba/references/architecture-details.md":"# References Skill\n\nPath: `01-model-architecture/mamba/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/architecture-details.md)","01-model-architecture/mamba/references/benchmarks.md":"# References Skill\n\nPath: `01-model-architecture/mamba/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/benchmarks.md)","01-model-architecture/mamba/references/training-guide.md":"# References Skill\n\nPath: `01-model-architecture/mamba/references/training-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/training-guide.md)","01-model-architecture/nanogpt/references/architecture.md":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/architecture.md)","01-model-architecture/nanogpt/references/data.md":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/data.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/data.md)","01-model-architecture/nanogpt/references/training.md":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/training.md)","01-model-architecture/rwkv/references/architecture-details.md":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/architecture-details.md)","01-model-architecture/rwkv/references/rwkv7.md":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/rwkv7.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/rwkv7.md)","01-model-architecture/rwkv/references/state-management.md":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/state-management.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/state-management.md)","01-model-architecture/torchtitan/references/checkpoint.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/checkpoint.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/checkpoint.md)","01-model-architecture/torchtitan/references/custom-models.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/custom-models.md)","01-model-architecture/torchtitan/references/float8.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/float8.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/float8.md)","01-model-architecture/torchtitan/references/fsdp.md":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/fsdp.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/fsdp.md)","02-tokenization/huggingface-tokenizers/references/algorithms.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/algorithms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/algorithms.md)","02-tokenization/huggingface-tokenizers/references/integration.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/integration.md)","02-tokenization/huggingface-tokenizers/references/pipeline.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/pipeline.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/pipeline.md)","02-tokenization/huggingface-tokenizers/references/training.md":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/training.md)","02-tokenization/sentencepiece/references/algorithms.md":"# References Skill\n\nPath: `02-tokenization/sentencepiece/references/algorithms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/algorithms.md)","02-tokenization/sentencepiece/references/training.md":"# References Skill\n\nPath: `02-tokenization/sentencepiece/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/training.md)","03-fine-tuning/unsloth/references/index.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/index.md)","03-fine-tuning/unsloth/references/llms.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms.md)","03-fine-tuning/unsloth/references/llms-full.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms-full.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-full.md)","03-fine-tuning/unsloth/references/llms-txt.md":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms-txt.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-txt.md)","03-fine-tuning/peft/references/advanced-usage.md":"# References Skill\n\nPath: `03-fine-tuning/peft/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/advanced-usage.md)","03-fine-tuning/peft/references/troubleshooting.md":"# References Skill\n\nPath: `03-fine-tuning/peft/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/troubleshooting.md)","03-fine-tuning/llama-factory/references/_images.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/_images.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/_images.md)","03-fine-tuning/llama-factory/references/advanced.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/advanced.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/advanced.md)","03-fine-tuning/llama-factory/references/getting_started.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/getting_started.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/getting_started.md)","03-fine-tuning/llama-factory/references/index.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/index.md)","03-fine-tuning/llama-factory/references/other.md":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/other.md)","03-fine-tuning/axolotl/references/api.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/api.md)","03-fine-tuning/axolotl/references/dataset-formats.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/dataset-formats.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/dataset-formats.md)","03-fine-tuning/axolotl/references/index.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/index.md)","03-fine-tuning/axolotl/references/other.md":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/other.md)","04-mechanistic-interpretability/transformer-lens/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/api.md)","04-mechanistic-interpretability/transformer-lens/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/README.md)","04-mechanistic-interpretability/transformer-lens/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/tutorials.md)","04-mechanistic-interpretability/saelens/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/api.md)","04-mechanistic-interpretability/saelens/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/README.md)","04-mechanistic-interpretability/saelens/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/tutorials.md)","04-mechanistic-interpretability/pyvene/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/api.md)","04-mechanistic-interpretability/pyvene/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/README.md)","04-mechanistic-interpretability/pyvene/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/tutorials.md)","04-mechanistic-interpretability/nnsight/references/api.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/api.md)","04-mechanistic-interpretability/nnsight/references/README.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/README.md)","04-mechanistic-interpretability/nnsight/references/tutorials.md":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/tutorials.md)","05-data-processing/nemo-curator/references/deduplication.md":"# References Skill\n\nPath: `05-data-processing/nemo-curator/references/deduplication.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/deduplication.md)","05-data-processing/nemo-curator/references/filtering.md":"# References Skill\n\nPath: `05-data-processing/nemo-curator/references/filtering.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/filtering.md)","05-data-processing/ray-data/references/integration.md":"# References Skill\n\nPath: `05-data-processing/ray-data/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/integration.md)","05-data-processing/ray-data/references/transformations.md":"# References Skill\n\nPath: `05-data-processing/ray-data/references/transformations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/transformations.md)","06-post-training/verl/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/verl/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/api-reference.md)","06-post-training/verl/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/verl/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/troubleshooting.md)","06-post-training/trl-fine-tuning/references/dpo-variants.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/dpo-variants.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/dpo-variants.md)","06-post-training/trl-fine-tuning/references/online-rl.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/online-rl.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/online-rl.md)","06-post-training/trl-fine-tuning/references/reward-modeling.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/reward-modeling.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/reward-modeling.md)","06-post-training/trl-fine-tuning/references/sft-training.md":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/sft-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/sft-training.md)","06-post-training/torchforge/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/torchforge/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/api-reference.md)","06-post-training/torchforge/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/torchforge/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/troubleshooting.md)","06-post-training/slime/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/slime/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/api-reference.md)","06-post-training/slime/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/slime/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/troubleshooting.md)","06-post-training/simpo/references/datasets.md":"# References Skill\n\nPath: `06-post-training/simpo/references/datasets.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/datasets.md)","06-post-training/simpo/references/hyperparameters.md":"# References Skill\n\nPath: `06-post-training/simpo/references/hyperparameters.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/hyperparameters.md)","06-post-training/simpo/references/loss-functions.md":"# References Skill\n\nPath: `06-post-training/simpo/references/loss-functions.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/loss-functions.md)","06-post-training/openrlhf/references/algorithm-comparison.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/algorithm-comparison.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/algorithm-comparison.md)","06-post-training/openrlhf/references/custom-rewards.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/custom-rewards.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/custom-rewards.md)","06-post-training/openrlhf/references/hybrid-engine.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/hybrid-engine.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/hybrid-engine.md)","06-post-training/openrlhf/references/multi-node-training.md":"# References Skill\n\nPath: `06-post-training/openrlhf/references/multi-node-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/multi-node-training.md)","06-post-training/miles/references/api-reference.md":"# References Skill\n\nPath: `06-post-training/miles/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/api-reference.md)","06-post-training/miles/references/troubleshooting.md":"# References Skill\n\nPath: `06-post-training/miles/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/troubleshooting.md)","08-distributed-training/ray-train/references/multi-node.md":"# References Skill\n\nPath: `08-distributed-training/ray-train/references/multi-node.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/references/multi-node.md)","08-distributed-training/pytorch-lightning/references/callbacks.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/callbacks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/callbacks.md)","08-distributed-training/pytorch-lightning/references/distributed.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/distributed.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/distributed.md)","08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md)","08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md)","08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md)","08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md)","08-distributed-training/megatron-core/references/benchmarks.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/benchmarks.md)","08-distributed-training/megatron-core/references/parallelism-guide.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/parallelism-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/parallelism-guide.md)","08-distributed-training/megatron-core/references/production-examples.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/production-examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/production-examples.md)","08-distributed-training/megatron-core/references/training-recipes.md":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/training-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/training-recipes.md)","08-distributed-training/deepspeed/references/08.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/08.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/08.md)","08-distributed-training/deepspeed/references/09.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/09.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/09.md)","08-distributed-training/deepspeed/references/2020.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/2020.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2020.md)","08-distributed-training/deepspeed/references/2023.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/2023.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2023.md)","08-distributed-training/deepspeed/references/assets.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/assets.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/assets.md)","08-distributed-training/deepspeed/references/index.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/index.md)","08-distributed-training/deepspeed/references/mii.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/mii.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/mii.md)","08-distributed-training/deepspeed/references/other.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/other.md)","08-distributed-training/deepspeed/references/tutorials.md":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/tutorials.md)","08-distributed-training/accelerate/references/custom-plugins.md":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/custom-plugins.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/custom-plugins.md)","08-distributed-training/accelerate/references/megatron-integration.md":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/megatron-integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/megatron-integration.md)","08-distributed-training/accelerate/references/performance.md":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/performance.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/performance.md)","09-infrastructure/lambda-labs/references/advanced-usage.md":"# References Skill\n\nPath: `09-infrastructure/lambda-labs/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/advanced-usage.md)","09-infrastructure/lambda-labs/references/troubleshooting.md":"# References Skill\n\nPath: `09-infrastructure/lambda-labs/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/troubleshooting.md)","09-infrastructure/modal/references/advanced-usage.md":"# References Skill\n\nPath: `09-infrastructure/modal/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/advanced-usage.md)","09-infrastructure/modal/references/troubleshooting.md":"# References Skill\n\nPath: `09-infrastructure/modal/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/troubleshooting.md)","09-infrastructure/skypilot/references/advanced-usage.md":"# References Skill\n\nPath: `09-infrastructure/skypilot/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/advanced-usage.md)","09-infrastructure/skypilot/references/troubleshooting.md":"# References Skill\n\nPath: `09-infrastructure/skypilot/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/troubleshooting.md)","10-optimization/awq/references/advanced-usage.md":"# References Skill\n\nPath: `10-optimization/awq/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/advanced-usage.md)","10-optimization/awq/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/awq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/troubleshooting.md)","10-optimization/bitsandbytes/references/memory-optimization.md":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/memory-optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/memory-optimization.md)","10-optimization/bitsandbytes/references/qlora-training.md":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/qlora-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/qlora-training.md)","10-optimization/bitsandbytes/references/quantization-formats.md":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/quantization-formats.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/quantization-formats.md)","10-optimization/flash-attention/references/benchmarks.md":"# References Skill\n\nPath: `10-optimization/flash-attention/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/benchmarks.md)","10-optimization/flash-attention/references/transformers-integration.md":"# References Skill\n\nPath: `10-optimization/flash-attention/references/transformers-integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/transformers-integration.md)","10-optimization/gguf/references/advanced-usage.md":"# References Skill\n\nPath: `10-optimization/gguf/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/advanced-usage.md)","10-optimization/gguf/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/gguf/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/troubleshooting.md)","10-optimization/gptq/references/calibration.md":"# References Skill\n\nPath: `10-optimization/gptq/references/calibration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/calibration.md)","10-optimization/gptq/references/integration.md":"# References Skill\n\nPath: `10-optimization/gptq/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/integration.md)","10-optimization/gptq/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/gptq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/troubleshooting.md)","10-optimization/hqq/references/advanced-usage.md":"# References Skill\n\nPath: `10-optimization/hqq/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/advanced-usage.md)","10-optimization/hqq/references/troubleshooting.md":"# References Skill\n\nPath: `10-optimization/hqq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/troubleshooting.md)","10-optimization/ml-training-recipes/references/architecture.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/architecture.md)","10-optimization/ml-training-recipes/references/biomedical.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/biomedical.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/biomedical.md)","10-optimization/ml-training-recipes/references/domain-specific.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/domain-specific.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/domain-specific.md)","10-optimization/ml-training-recipes/references/experiment-loop.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/experiment-loop.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/experiment-loop.md)","10-optimization/ml-training-recipes/references/optimizers.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/optimizers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/optimizers.md)","10-optimization/ml-training-recipes/references/scaling-and-selection.md":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/scaling-and-selection.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/scaling-and-selection.md)","11-evaluation/bigcode-evaluation-harness/references/benchmarks.md":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/benchmarks.md)","11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md)","11-evaluation/bigcode-evaluation-harness/references/issues.md":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/issues.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/issues.md)","11-evaluation/lm-evaluation-harness/references/api-evaluation.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/api-evaluation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/api-evaluation.md)","11-evaluation/lm-evaluation-harness/references/benchmark-guide.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/benchmark-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/benchmark-guide.md)","11-evaluation/lm-evaluation-harness/references/custom-tasks.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/custom-tasks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/custom-tasks.md)","11-evaluation/lm-evaluation-harness/references/distributed-eval.md":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/distributed-eval.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/distributed-eval.md)","11-evaluation/nemo-evaluator/references/adapter-system.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/adapter-system.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/adapter-system.md)","11-evaluation/nemo-evaluator/references/configuration.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/configuration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/configuration.md)","11-evaluation/nemo-evaluator/references/custom-benchmarks.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/custom-benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/custom-benchmarks.md)","11-evaluation/nemo-evaluator/references/execution-backends.md":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/execution-backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/execution-backends.md)","12-inference-serving/llama-cpp/references/optimization.md":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/optimization.md)","12-inference-serving/llama-cpp/references/quantization.md":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/quantization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/quantization.md)","12-inference-serving/llama-cpp/references/server.md":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/server.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/server.md)","12-inference-serving/sglang/references/deployment.md":"# References Skill\n\nPath: `12-inference-serving/sglang/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/deployment.md)","12-inference-serving/sglang/references/radix-attention.md":"# References Skill\n\nPath: `12-inference-serving/sglang/references/radix-attention.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/radix-attention.md)","12-inference-serving/sglang/references/structured-generation.md":"# References Skill\n\nPath: `12-inference-serving/sglang/references/structured-generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/structured-generation.md)","12-inference-serving/tensorrt-llm/references/multi-gpu.md":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/multi-gpu.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/multi-gpu.md)","12-inference-serving/tensorrt-llm/references/optimization.md":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/optimization.md)","12-inference-serving/tensorrt-llm/references/serving.md":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/serving.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/serving.md)","12-inference-serving/vllm/references/optimization.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/optimization.md)","12-inference-serving/vllm/references/quantization.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/quantization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/quantization.md)","12-inference-serving/vllm/references/server-deployment.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/server-deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/server-deployment.md)","12-inference-serving/vllm/references/troubleshooting.md":"# References Skill\n\nPath: `12-inference-serving/vllm/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/troubleshooting.md)","13-mlops/mlflow/references/deployment.md":"# References Skill\n\nPath: `13-mlops/mlflow/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/deployment.md)","13-mlops/mlflow/references/model-registry.md":"# References Skill\n\nPath: `13-mlops/mlflow/references/model-registry.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/model-registry.md)","13-mlops/mlflow/references/tracking.md":"# References Skill\n\nPath: `13-mlops/mlflow/references/tracking.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/tracking.md)","13-mlops/swanlab/references/integrations.md":"# References Skill\n\nPath: `13-mlops/swanlab/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/integrations.md)","13-mlops/swanlab/references/visualization.md":"# References Skill\n\nPath: `13-mlops/swanlab/references/visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/visualization.md)","13-mlops/tensorboard/references/integrations.md":"# References Skill\n\nPath: `13-mlops/tensorboard/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/integrations.md)","13-mlops/tensorboard/references/profiling.md":"# References Skill\n\nPath: `13-mlops/tensorboard/references/profiling.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/profiling.md)","13-mlops/tensorboard/references/visualization.md":"# References Skill\n\nPath: `13-mlops/tensorboard/references/visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/visualization.md)","13-mlops/weights-and-biases/references/artifacts.md":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/artifacts.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/artifacts.md)","13-mlops/weights-and-biases/references/integrations.md":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/integrations.md)","13-mlops/weights-and-biases/references/sweeps.md":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/sweeps.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/sweeps.md)","14-agents/a-evolve/references/api.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/api.md)","14-agents/a-evolve/references/architecture.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/architecture.md)","14-agents/a-evolve/references/design-patterns.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/design-patterns.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/design-patterns.md)","14-agents/a-evolve/references/examples.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/examples.md)","14-agents/a-evolve/references/issues.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/issues.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/issues.md)","14-agents/a-evolve/references/README.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/README.md)","14-agents/a-evolve/references/releases.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/releases.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/releases.md)","14-agents/a-evolve/references/tutorials.md":"# References Skill\n\nPath: `14-agents/a-evolve/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/tutorials.md)","14-agents/autogpt/references/advanced-usage.md":"# References Skill\n\nPath: `14-agents/autogpt/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/advanced-usage.md)","14-agents/autogpt/references/troubleshooting.md":"# References Skill\n\nPath: `14-agents/autogpt/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/troubleshooting.md)","14-agents/crewai/references/flows.md":"# References Skill\n\nPath: `14-agents/crewai/references/flows.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/flows.md)","14-agents/crewai/references/tools.md":"# References Skill\n\nPath: `14-agents/crewai/references/tools.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/tools.md)","14-agents/crewai/references/troubleshooting.md":"# References Skill\n\nPath: `14-agents/crewai/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/troubleshooting.md)","14-agents/langchain/references/agents.md":"# References Skill\n\nPath: `14-agents/langchain/references/agents.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/agents.md)","14-agents/langchain/references/integration.md":"# References Skill\n\nPath: `14-agents/langchain/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/integration.md)","14-agents/langchain/references/rag.md":"# References Skill\n\nPath: `14-agents/langchain/references/rag.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/rag.md)","14-agents/llamaindex/references/agents.md":"# References Skill\n\nPath: `14-agents/llamaindex/references/agents.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/agents.md)","14-agents/llamaindex/references/data_connectors.md":"# References Skill\n\nPath: `14-agents/llamaindex/references/data_connectors.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/data_connectors.md)","14-agents/llamaindex/references/query_engines.md":"# References Skill\n\nPath: `14-agents/llamaindex/references/query_engines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/query_engines.md)","15-rag/chroma/references/integration.md":"# References Skill\n\nPath: `15-rag/chroma/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/references/integration.md)","15-rag/faiss/references/index_types.md":"# References Skill\n\nPath: `15-rag/faiss/references/index_types.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/references/index_types.md)","15-rag/pinecone/references/deployment.md":"# References Skill\n\nPath: `15-rag/pinecone/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/references/deployment.md)","15-rag/qdrant/references/advanced-usage.md":"# References Skill\n\nPath: `15-rag/qdrant/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/advanced-usage.md)","15-rag/qdrant/references/troubleshooting.md":"# References Skill\n\nPath: `15-rag/qdrant/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/troubleshooting.md)","15-rag/sentence-transformers/references/models.md":"# References Skill\n\nPath: `15-rag/sentence-transformers/references/models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/references/models.md)","16-prompt-engineering/dspy/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/examples.md)","16-prompt-engineering/dspy/references/modules.md":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/modules.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/modules.md)","16-prompt-engineering/dspy/references/optimizers.md":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/optimizers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/optimizers.md)","16-prompt-engineering/guidance/references/backends.md":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/backends.md)","16-prompt-engineering/guidance/references/constraints.md":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/constraints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/constraints.md)","16-prompt-engineering/guidance/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/examples.md)","16-prompt-engineering/instructor/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/examples.md)","16-prompt-engineering/instructor/references/providers.md":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/providers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/providers.md)","16-prompt-engineering/instructor/references/validation.md":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/validation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/validation.md)","16-prompt-engineering/outlines/references/backends.md":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/backends.md)","16-prompt-engineering/outlines/references/examples.md":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/examples.md)","16-prompt-engineering/outlines/references/json_generation.md":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/json_generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/json_generation.md)","17-observability/langsmith/references/advanced-usage.md":"# References Skill\n\nPath: `17-observability/langsmith/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/advanced-usage.md)","17-observability/langsmith/references/troubleshooting.md":"# References Skill\n\nPath: `17-observability/langsmith/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/troubleshooting.md)","17-observability/phoenix/references/advanced-usage.md":"# References Skill\n\nPath: `17-observability/phoenix/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/advanced-usage.md)","17-observability/phoenix/references/troubleshooting.md":"# References Skill\n\nPath: `17-observability/phoenix/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/troubleshooting.md)","18-multimodal/audiocraft/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/audiocraft/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/advanced-usage.md)","18-multimodal/audiocraft/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/audiocraft/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/troubleshooting.md)","18-multimodal/blip-2/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/blip-2/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/advanced-usage.md)","18-multimodal/blip-2/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/blip-2/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/troubleshooting.md)","18-multimodal/clip/references/applications.md":"# References Skill\n\nPath: `18-multimodal/clip/references/applications.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/references/applications.md)","18-multimodal/cosmos-policy/references/libero-commands.md":"# References Skill\n\nPath: `18-multimodal/cosmos-policy/references/libero-commands.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/libero-commands.md)","18-multimodal/cosmos-policy/references/robocasa-commands.md":"# References Skill\n\nPath: `18-multimodal/cosmos-policy/references/robocasa-commands.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/robocasa-commands.md)","18-multimodal/llava/references/training.md":"# References Skill\n\nPath: `18-multimodal/llava/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/references/training.md)","18-multimodal/openpi/references/checkpoints-and-env-map.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/checkpoints-and-env-map.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/checkpoints-and-env-map.md)","18-multimodal/openpi/references/config-recipes.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/config-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/config-recipes.md)","18-multimodal/openpi/references/pytorch-gotchas.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/pytorch-gotchas.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/pytorch-gotchas.md)","18-multimodal/openpi/references/remote-client-pattern.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/remote-client-pattern.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/remote-client-pattern.md)","18-multimodal/openpi/references/training-debugging.md":"# References Skill\n\nPath: `18-multimodal/openpi/references/training-debugging.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/training-debugging.md)","18-multimodal/openvla-oft/references/aloha-workflow.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/aloha-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/aloha-workflow.md)","18-multimodal/openvla-oft/references/config-troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/config-troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/config-troubleshooting.md)","18-multimodal/openvla-oft/references/libero-workflow.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/libero-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/libero-workflow.md)","18-multimodal/openvla-oft/references/paper-and-checkpoints.md":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/paper-and-checkpoints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/paper-and-checkpoints.md)","18-multimodal/segment-anything/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/segment-anything/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/advanced-usage.md)","18-multimodal/segment-anything/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/segment-anything/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/troubleshooting.md)","18-multimodal/stable-diffusion/references/advanced-usage.md":"# References Skill\n\nPath: `18-multimodal/stable-diffusion/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/advanced-usage.md)","18-multimodal/stable-diffusion/references/troubleshooting.md":"# References Skill\n\nPath: `18-multimodal/stable-diffusion/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/troubleshooting.md)","18-multimodal/whisper/references/languages.md":"# References Skill\n\nPath: `18-multimodal/whisper/references/languages.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/references/languages.md)","19-emerging-techniques/knowledge-distillation/references/minillm.md":"# References Skill\n\nPath: `19-emerging-techniques/knowledge-distillation/references/minillm.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/references/minillm.md)","19-emerging-techniques/long-context/references/extension_methods.md":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/extension_methods.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/extension_methods.md)","19-emerging-techniques/long-context/references/fine_tuning.md":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/fine_tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/fine_tuning.md)","19-emerging-techniques/long-context/references/rope.md":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/rope.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/rope.md)","19-emerging-techniques/model-merging/references/coefficient-tuning.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/coefficient-tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/coefficient-tuning.md)","19-emerging-techniques/model-merging/references/evaluation.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/evaluation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/evaluation.md)","19-emerging-techniques/model-merging/references/examples.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/examples.md)","19-emerging-techniques/model-merging/references/methods.md":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/methods.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/methods.md)","19-emerging-techniques/model-pruning/references/wanda.md":"# References Skill\n\nPath: `19-emerging-techniques/model-pruning/references/wanda.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/references/wanda.md)","19-emerging-techniques/moe-training/references/architectures.md":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/architectures.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/architectures.md)","19-emerging-techniques/moe-training/references/inference.md":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/inference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/inference.md)","19-emerging-techniques/moe-training/references/training.md":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/training.md)","19-emerging-techniques/speculative-decoding/references/lookahead.md":"# References Skill\n\nPath: `19-emerging-techniques/speculative-decoding/references/lookahead.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/lookahead.md)","19-emerging-techniques/speculative-decoding/references/medusa.md":"# References Skill\n\nPath: `19-emerging-techniques/speculative-decoding/references/medusa.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/medusa.md)","20-ml-paper-writing/systems-paper-writing/references/checklist.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/checklist.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/checklist.md)","20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md)","20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md)","20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md)","20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md)","20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md":"# References Skill\n\nPath: `20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md)","20-ml-paper-writing/ml-paper-writing/references/checklists.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/checklists.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/checklists.md)","20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md)","20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md)","20-ml-paper-writing/ml-paper-writing/references/sources.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/sources.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/sources.md)","20-ml-paper-writing/ml-paper-writing/references/writing-guide.md":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/writing-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/writing-guide.md)","20-ml-paper-writing/academic-plotting/references/data-visualization.md":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/data-visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/data-visualization.md)","20-ml-paper-writing/academic-plotting/references/diagram-generation.md":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/diagram-generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/diagram-generation.md)","20-ml-paper-writing/academic-plotting/references/style-guide.md":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/style-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/style-guide.md)","22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md)","22-agent-native-research-artifact/research-manager/references/event-taxonomy.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/event-taxonomy.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/event-taxonomy.md)","22-agent-native-research-artifact/research-manager/references/provenance-tags.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/provenance-tags.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/provenance-tags.md)","22-agent-native-research-artifact/research-manager/references/session-protocol.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/session-protocol.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/session-protocol.md)","22-agent-native-research-artifact/compiler/references/ara-schema.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/ara-schema.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/ara-schema.md)","22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md)","22-agent-native-research-artifact/compiler/references/validation-checklist.md":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/validation-checklist.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/validation-checklist.md)"},"items":[{"name":"agent-continuity.md","path":"0-autoresearch-skill/references/agent-continuity.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/agent-continuity.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `0-autoresearch-skill/references/agent-continuity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/agent-continuity.md)","tokens":300,"sizeBytes":239},{"name":"progress-reporting.md","path":"0-autoresearch-skill/references/progress-reporting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/progress-reporting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `0-autoresearch-skill/references/progress-reporting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/progress-reporting.md)","tokens":300,"sizeBytes":243},{"name":"skill-routing.md","path":"0-autoresearch-skill/references/skill-routing.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/skill-routing.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `0-autoresearch-skill/references/skill-routing.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/skill-routing.md)","tokens":300,"sizeBytes":233},{"name":"SKILL.md","path":"0-autoresearch-skill/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/SKILL.md","title":"autoresearch","category":"anthropic-skill","format":"markdown","content":"---\nname: autoresearch\ndescription: Orchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for execution, supports continuous agent operation via Claude Code /loop and OpenClaw heartbeat, and produces research presentations and papers. Use when starting a research project, running autonomous experiments, or managing a multi-hypothesis research effort.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Autonomous Research, Two-Loop Architecture, Experiment Orchestration, Research Synthesis, Project Management]\n---\n\n# Autoresearch\n\nAutonomous research orchestration for AI coding agents. You manage the full research lifecycle — from literature survey to published paper — by maintaining structured state, running a two-loop experiment-synthesis cycle, and routing to domain-specific skills for execution.\n\nYou are a research project manager, not a domain expert. You orchestrate; the domain skills execute.\n\n**This runs fully autonomously.** Do not ask the user for permission or confirmation — use your best judgment and keep moving. Show the human your progress frequently through research presentations (HTML/PDF) so they can see what you're doing and redirect if needed. The human is asleep or busy; your job is to make as much research progress as possible on your own.\n\n## Getting Started\n\nUsers arrive in different states. Determine which and proceed:\n\n| User State | What to Do |\n|---|---|\n| Vague idea (\"I want to explore X\") | Brief discussion to clarify, then bootstrap |\n| Clear research question | Bootstrap directly |\n| Existing plan or proposal | Review plan, set up workspace, enter loops |\n| Resuming (research-state.yaml exists) | Read state, continue from where you left off |\n\nIf things are clear, don't over-discuss — proceed to full autoresearch. Most users want you to just start researching.\n\n**Step 0 — before anything else**: Set up the agent continuity loop. See [Agent Continuity](#agent-continuity-mandatory--set-up-first). This is MANDATORY. Without it, the research stops after one cycle.\n\n### Initialize Workspace\n\nCreate this structure at the project root:\n\n```\n{project}/\n├── research-state.yaml # Central state tracking\n├── research-log.md # Decision timeline\n├── findings.md # Evolving narrative synthesis\n├── literature/ # Papers, survey notes\n├── src/ # Reusable code (utils, plotting, shared modules)\n├── data/ # Raw result data (CSVs, JSONs, checkpoints)\n├── experiments/ # Per-hypothesis work\n│ └── {hypothesis-slug}/\n│ ├── protocol.md # What, why, and prediction\n│ ├── code/ # Experiment-specific code\n│ ├── results/ # Raw outputs, metrics, logs\n│ └── analysis.md # What we learned\n├── to_human/ # Progress presentations and reports for human review\n└── paper/ # Final paper (via ml-paper-writing)\n```\n\n- **`src/`**: When you write useful code (plotting functions, data loaders, evaluation helpers), move it here so it can be reused across experiments. Don't duplicate code in every experiment directory.\n- **`data/`**: Save raw result data (metric CSVs, training logs, small outputs) here in a structured way. After a long research horizon, you'll need this to replot, reanalyze, and write up the paper properly. Name files descriptively (e.g., `trajectory_H1_runs001-010.csv`). Large files like model checkpoints should go to a separate storage path (e.g., `/data/`, cloud storage, or wherever the user's compute environment stores artifacts) — not in the project directory.\n\nInitialize `research-state.yaml`, `research-log.md`, and `findings.md` from [templates/](templates/). Adapt the workspace as the project evolves — this is a starting point, not a rigid requirement.\n\n## The Two-Loop Architecture\n\nThis is the core engine. Everything else supports it.\n\n```\nBOOTSTRAP (once, lightweight)\n Scope question → search literature → form initial hypotheses\n\nINNER LOOP (fast, autonomous, repeating)\n Pick hypothesis → experiment → measure → record → learn → next\n Goal: run constrained experiments with clear measurable outcomes\n\nOUTER LOOP (periodic, reflective)\n Review results → find patterns → update findings.md →\n new hypotheses → decide direction\n Goal: synthesize understanding, find the story — this is where novelty comes from\n\nFINALIZE (when concluding)\n Write paper via ml-paper-writing → final presentation → archive\n```\n\nThe inner loop runs tight experiment cycles with clear measurable outcomes. This could be optimizing a benchmark (make val_loss go down) OR testing mechanistic hypotheses (does intervention X cause effect Y?). The outer loop steps back to ask: what do these results *mean*? What patterns emerge? What's the story? Research is open-ended — the two loops let you both optimize and discover.\n\nThere is no rigid boundary between the two loops — you decide when enough inner loop results have accumulated to warrant reflection. Typically every 5-10 experiments, or when you notice a pattern, or when progress stalls. The agent's judgment drives the rhythm.\n\n### Research is Non-Linear\n\nThe two-loop structure is a rhythm, not a railroad. At any point during research you can and should:\n\n- **Return to literature** when results surprise you, assumptions break, or you need context for a new direction — always save what you find to `literature/`\n- **Brainstorm new ideas** using `21-research-ideation/` skills when you're stuck or when results open unexpected questions\n- **Pivot the question entirely** if experiments reveal the original question was wrong or less interesting than what you found\n\nThis is normal. Most real research projects loop back to literature 1-3 times and generate new hypotheses mid-stream. Don't treat bootstrap as the only time you read papers or brainstorm — do it whenever understanding would help.\n\n## Bootstrap: Literature and Hypotheses\n\nBefore entering the loops, understand the landscape. Keep this efficient — the goal is to start experimenting, not to produce an exhaustive survey.\n\n1. **Search literature** for the research question. Use multiple sources — never stop at one:\n - **Exa MCP** (`web_search_exa`) if available — best for broad discovery and finding relevant papers quickly\n - **Semantic Scholar** (`pip install semanticscholar`) — best for ML/AI papers, citation graphs, and specific paper lookup. See `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code examples\n - **arXiv** (`pip install arxiv`) — best for recent preprints and open-access papers\n - **CrossRef** — best for DOI lookup and BibTeX retrieval\n - Keep searching until you have good coverage. If one source comes up empty, try another with different keywords\n\n **Save everything to `literature/`**: For every paper you find, save a summary to `literature/` — title, authors, year, key findings, relevance to your question, and the URL/DOI. Create one file per paper and a running `literature/survey.md` with all summaries. This is your reference library — you and future sessions will need it throughout the project.\n\n2. **Identify gaps** from the literature\n - What's been tried? What hasn't? Where do existing methods break?\n - What do Discussion sections flag as future work?\n\n3. **Form initial hypotheses** — invoke `21-research-ideation/` skills\n - `brainstorming-research-ideas` for structured diverge-converge workflow\n - `creative-thinking-for-research` for deeper cognitive frameworks\n - Each hypothesis must be testable with a clear prediction\n\n4. **Define the evaluation**\n - Set the proxy metric and baseline before running experiments\n - The metric should be computable quickly (minutes, not hours)\n - Lock evaluation criteria upfront to prevent unconscious metric gaming\n\n5. **Record** in research-state.yaml, log the bootstrap in research-log.md\n\n## The Inner Loop\n\nRapid iteration with clear measurable outcomes. Two flavors:\n\n- **Optimization**: make a metric go up/down (val_loss, accuracy, throughput). Think Karpathy's autoresearch.\n- **Discovery**: test mechanistic hypotheses about why something works. The metric is a measurement (does grokking happen faster? does entropy increase before forgetting?), not just a target to optimize.\n\n```\n1. Pick the highest-priority untested hypothesis\n2. Write a protocol: what change, what prediction, why\n Lock it: commit to git BEFORE running (research(protocol): {hypothesis})\n This creates temporal proof your plan existed before results\n3. Run the experiment (invoke the relevant domain skill)\n4. Sanity check before trusting results:\n - Did training converge? No NaN/Inf?\n - Does baseline reproduce expected performance?\n - Data loading correct? (spot-check a few samples)\n5. Measure the proxy metric\n6. Record in experiments/{hypothesis-slug}/\n Label clearly: CONFIRMATORY (in your protocol) vs EXPLORATORY (discovered during execution)\n7. If positive: keep, note WHY it worked\n8. If negative: this is progress — note what it rules out and what it suggests\n9. Update research-state.yaml\n10. If stuck: search literature or invoke ideation skills — don't just keep trying random things\n```\n\n**Never stop.** Even if something fails, find a path forward. Debug, adjust, simplify, or pivot — but keep the research moving. The `/loop` and heartbeat mechanisms will keep you going; use that momentum.\n\n### Route to Domain Skills\n\nWhen you need domain-specific execution, search the skills library:\n\n| Research Activity | Look In |\n|---|---|\n| Data preparation | `05-data-processing/` |\n| Model training / fine-tuning | `01-model-architecture/`, `03-fine-tuning/`, `06-post-training/` |\n| Distributed training | `08-distributed-training/` |\n| Optimization (quantization, attention) | `10-optimization/` |\n| Evaluation / benchmarks | `11-evaluation/` |\n| Inference / serving | `12-inference-serving/` |\n| Interpretability analysis | `04-mechanistic-interpretability/` |\n| Experiment tracking (W&B, MLflow) | `13-mlops/` |\n| Cloud compute | `09-infrastructure/` |\n\nRead the relevant SKILL.md before starting — it has workflows, common issues, and code examples. See [references/skill-routing.md](references/skill-routing.md) for a complete guide.\n\n### Track the Experiment Trajectory\n\nMaintain a running record of measurable outcomes across experiments:\n\n```json\n{\n \"experiment_id\": \"run_014\",\n \"hypothesis\": \"H3\",\n \"metric_value\": 0.847,\n \"baseline\": 0.812,\n \"delta\": \"+0.035\",\n \"wall_time_min\": 23,\n \"change_summary\": \"Added cosine annealing warmup schedule\"\n}\n```\n\nThis trajectory produces the optimization plot (like Karpathy's progress chart) — include it in progress reports. Humans love seeing the upward curve.\n\n## The Outer Loop\n\nStep back from individual experiments. Synthesize.\n\n```\n1. Review all results since last reflection\n2. Cluster by type: what kinds of changes worked? Which didn't?\n3. Ask WHY — identify the mechanism behind successes and failures\n4. Update findings.md with current understanding\n5. Search literature if results were surprising or assumptions need revisiting\n6. Generate new hypotheses if warranted (invoke 21-research-ideation/ skills)\n7. Decide direction (see criteria below)\n8. Update research-state.yaml with new direction\n9. Log the reflection in research-log.md\n10. If there's something meaningful, generate a progress presentation\n```\n\n### Deciding Direction\n\nDon't just pick randomly — use these criteria:\n\n**DEEPEN** — a supported result raises follow-up questions\n- Does the effect hold under different conditions? What's the mechanism?\n- Action: generate sub-hypotheses (H1.1, H1.2) → back to inner loop\n\n**BROADEN** — current results are solid, but adjacent questions are untested\n- New questions emerged. The current contribution is clear but more is possible.\n- Action: generate new root hypotheses → back to inner loop\n\n**PIVOT** — results invalidate key assumptions or something more interesting appeared\n- A core assumption was wrong, or an unexpected finding is more promising than the original question.\n- Action: return to literature with new questions → re-bootstrap\n\n**CONCLUDE** — sufficient evidence for a contribution\n- At least one hypothesis is strongly supported (or a coherent set of negative results)\n- Key ablations completed, error analysis done\n- findings.md reads like a paper backbone — a human could write the abstract from it\n- No critical open questions that would change the story\n\nNote: coherent negative results are a valid contribution. \"X does NOT work because Y\" is publishable if the reasoning is rigorous.\n\n### findings.md Is Your Project Memory\n\nThis file serves two purposes: it's the research narrative for humans AND your accumulated knowledge base as an agent. Read it at the start of every session, /loop tick, or heartbeat to remember what you've learned.\n\nAfter every outer loop, update it to answer:\n\n- What do we know so far? (Current Understanding)\n- What patterns explain our results? (Patterns and Insights)\n- What specific things did we learn not to repeat? (Lessons and Constraints)\n- What remains open? (Open Questions)\n\nThe \"Lessons and Constraints\" section is especially important — it captures specific actionable learnings like \"weight decay > 0.1 diverges at this scale\" or \"baseline only reproduces with batch_size=64.\" This prevents the agent from repeating failed approaches across sessions.\n\n**Quality test**: After 30 inner loop experiments, a human should be able to read findings.md and write a paper abstract from it. If they can't, the outer loop isn't synthesizing — it's just logging.\n\n## Agent Continuity (MANDATORY — Set Up First)\n\n**Before doing anything else**, set up the wall-clock loop. This is what keeps the research running continuously. Without it, the agent stops after one cycle.\n\n### Claude Code\n\n**First action — run this immediately:**\n\n```\n/loop 20m Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a report in to_human/ and opening the HTML/PDF. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\n```\n\nThis fires every 20 minutes regardless of what's happening. It's a rhythm that keeps you working — not a research phase boundary. If your previous work isn't done, just continue it.\n\n### OpenClaw\n\n**First action — set up a 20-minute cron job in the current session. You MUST do this before any research work.**\n\nUse the `cron.add` tool to create a recurring job bound to this chat session:\n\n```json\n{\n \"name\": \"autoresearch-loop\",\n \"schedule\": { \"kind\": \"every\", \"everyMs\": 1200000 },\n \"sessionTarget\": \"current\",\n \"payload\": {\n \"kind\": \"agentTurn\",\n \"message\": \"Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a PDF report in to_human/ and sending it to the user via Telegram, WhatsApp, or Slack. When you get an exciting result or interesting plot, text it to the user right away — don't wait for a full report. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\"\n }\n}\n```\n\nKey details:\n- `sessionTarget: \"current\"` binds the cron to this chat session so it maintains conversation context across ticks\n- `everyMs: 1200000` = 20 minutes\n- After creation, verify with `cron.list` that the job exists and is enabled\n- If the cron fires while you're mid-experiment, just continue — the tick is a nudge, not a restart\n\n### What the Loop Does\n\nThe `/loop` and cron job are purely **wall-clock rhythm**. They are completely separate from your research loops (inner/outer). On each tick:\n\n1. Read `research-state.yaml` and `findings.md` — remember where you are\n2. Check if anything is broken (failed experiments, stalled training, errors)\n3. If on track → keep working on whatever you were doing\n4. If stuck or something's wrong → step back, diagnose, fix, then continue\n5. Never idle. Always be making progress.\n\n## Progress Reporting\n\nWhen you have something meaningful to share, create a research presentation — not just a status dashboard, but a compelling story.\n\n**When to report** (your judgment):\n- After an outer loop that found a significant pattern\n- When the optimization trajectory shows clear progress (include the plot!)\n- After a pivot in direction\n- Before requesting human input on a decision\n- When concluding\n\n**What to include** (adapt to what's compelling):\n- The research question and why it matters\n- Key results with visualizations (plots, metric tables)\n- The optimization trajectory chart (metric over experiments)\n- What was tried and why (selective, not exhaustive)\n- Current understanding (the findings narrative)\n- What's planned next\n\nFor Claude Code: generate HTML and `open` it. If HTML fails to open or render, convert to PDF as fallback (use `weasyprint`, `playwright pdf`, or `wkhtmltopdf`). For OpenClaw: generate PDF directly.\n\nSee [references/progress-reporting.md](references/progress-reporting.md) for template scaffolding and the optimization plot approach. Use the template as a starting point — be creative with what you show.\n\n## Git Protocol\n\nCommit at natural research milestones:\n\n| When | Message Pattern |\n|---|---|\n| Workspace initialized | `research(init): {project} — {question}` |\n| Experiment protocol locked | `research(protocol): {hypothesis}` |\n| Significant results | `research(results): {hypothesis} — {outcome}` |\n| Outer loop direction change | `research(reflect): {direction} — {reason}` |\n| Paper draft complete | `research(paper): {title}` |\n\n**Hard rule**: Protocol commits MUST precede result commits. Never combine them. The git history is your lightweight pre-registration — it proves what you planned before you saw results. Don't commit after every experiment — commit when there's meaningful progress.\n\n## Concluding: Paper Writing\n\nWhen the outer loop decides to CONCLUDE:\n\n1. Ensure findings.md has a clear, well-supported narrative\n2. Study 2-3 top related papers to learn their format, style, and section structure\n3. Invoke the `20-ml-paper-writing` skill — it has LaTeX templates for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, and systems venues\n4. Feed it the accumulated literature, experimental results, and findings\n5. Follow its citation verification workflow — never hallucinate references\n6. Generate a final comprehensive research presentation\n\nProceed autonomously through the writing process. If the ml-paper-writing skill suggests human collaboration points, adapt and keep going — produce the best draft you can. The human will review and provide feedback.\n\n## Research Discipline\n\nPrinciples to enforce continuously — not tied to any specific phase:\n\n- **Lock before you run**: Commit your experiment protocol to git before executing. This proves your plan existed before you saw results. Never combine protocol + results in one commit.\n- **Confirmatory vs exploratory**: Results matching your locked protocol are confirmatory. Everything else is exploratory — interesting but requiring more skepticism.\n- **Negative results are progress**: A refuted hypothesis tells you something. Log what it rules out and what it suggests. Don't treat it as failure.\n- **Sanity check before analysis**: Verify training converged, baselines reproduce, and data is correct before trusting your primary metric.\n- **Return to literature when confused**: Don't guess — search. If results surprise you or assumptions break, go find papers. Use Exa MCP for discovery, Semantic Scholar for specific ML/AI paper lookup, arXiv for preprints.\n- **Never stop**: Don't wait for human approval on routine decisions. If a skill or tool suggests collaboration, adapt and keep going. Find the best path forward autonomously. The human will see your progress reports and can redirect if needed.\n- **Use whatever compute is available**: Adapt to the user's environment — local GPU, cluster job submission, cloud instances, or just CPU. If no GPU is available, use CPU and adjust experiment scale accordingly. Don't block on compute availability.\n\n## Quality Standards\n\n**Good agent behavior:**\n- Hypotheses have mechanistic reasoning (\"X because Y, predicting Z\"), not just \"try X\"\n- findings.md builds a coherent narrative, not a flat list of results\n- Negative results are recorded with what they rule out\n- The agent updates its model when experiments contradict expectations\n- Progress reports tell a research story with compelling visualizations\n\n**Bad agent behavior:**\n- Pure hyperparameter sweeps without interpretation\n- findings.md is just experiment logs copy-pasted\n- Agent never revisits its assumptions after failures\n- Optimizing metrics without understanding why changes work\n\n## When to Use vs Alternatives\n\n**Use autoresearch when:**\n- You have a research question explorable through experiments\n- There's a measurable proxy metric for inner loop optimization\n- The real contribution requires synthesis beyond the metric\n- You want continuous autonomous research operation\n\n**Use individual domain skills instead when:**\n- You have a specific one-off task (train a model, run eval, write a paper)\n- No iterative experimentation needed\n\n## Common Issues\n\n**Inner loop stalls (no metric improvement)**\nRun an outer loop. Is the metric the right one? Is the search space exhausted? Consider broadening or pivoting. Search literature for new approaches.\n\n**Stuck and not making progress**\nDon't keep trying random changes. Step back: search literature for related work, invoke `21-research-ideation/` brainstorming skills, or run an outer loop reflection. Being stuck means you need new information or a new perspective, not more experiments.\n\n**Results contradict baseline expectations**\nInvestigate, don't ignore. Return to literature — your protocol might have an error, the published baseline may be wrong, or conditions differ. Update findings.md with what you learn.\n\n**Agent loses context between ticks**\nEnsure research-state.yaml and findings.md are updated after every action. These files are your memory across sessions.\n\n**Can't find relevant papers**\nTry multiple approaches in order: Exa MCP for broad search, Semantic Scholar for specific ML/AI paper lookup (`pip install semanticscholar`), arXiv for preprints (`pip install arxiv`). Check `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code. Note: Google Scholar has no official API — use Semantic Scholar instead for programmatic search.\n\n**No GPU available**\nUse CPU and scale experiments down. Many research tasks (analysis, interpretability, small model training) run fine on CPU. Adjust experiment design to fit available compute rather than blocking.\n\n**Experiments take longer than /loop interval**\nNormal. On the next tick, check if it finished. If not, keep waiting or do something else useful (update notes, search papers). Adjust interval if needed.\n\n**Not sure when to conclude**\nThree questions: Do you have a strongly supported finding? Can you explain WHY it works? Would findings.md make a convincing paper abstract? If yes to all: conclude.\n\n## Advanced Topics\n\n- **Detailed agent continuity**: [references/agent-continuity.md](references/agent-continuity.md)\n- **Progress presentation templates**: [references/progress-reporting.md](references/progress-reporting.md)\n- **Complete skill routing**: [references/skill-routing.md](references/skill-routing.md)\n","frontmatter":{"name":"autoresearch","description":"Orchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for execution, supports continuous agent operation via Claude Code /loop and OpenClaw heartbeat, and produces research presentations and papers. Use when starting a research project, running autonomous experiments, or managing a multi-hypothesis research effort.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Autonomous Research","Two-Loop Architecture","Experiment Orchestration","Research Synthesis","Project Management"]},"isInternal":false,"tokens":5337,"sizeBytes":24945},{"name":"custom-models.md","path":"01-model-architecture/litgpt/references/custom-models.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/custom-models.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/custom-models.md)","tokens":300,"sizeBytes":249},{"name":"distributed-training.md","path":"01-model-architecture/litgpt/references/distributed-training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/distributed-training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/distributed-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/distributed-training.md)","tokens":300,"sizeBytes":263},{"name":"supported-models.md","path":"01-model-architecture/litgpt/references/supported-models.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/supported-models.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/supported-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/supported-models.md)","tokens":300,"sizeBytes":255},{"name":"training-recipes.md","path":"01-model-architecture/litgpt/references/training-recipes.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/training-recipes.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/litgpt/references/training-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/training-recipes.md)","tokens":300,"sizeBytes":255},{"name":"SKILL.md","path":"01-model-architecture/litgpt/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/SKILL.md","title":"implementing-llms-litgpt","category":"anthropic-skill","format":"markdown","content":"---\nname: implementing-llms-litgpt\ndescription: Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, LitGPT, Lightning AI, LLM Implementation, LoRA, QLoRA, Fine-Tuning, Llama, Gemma, Phi, Mistral, Educational]\ndependencies: [litgpt, torch, transformers]\n---\n\n# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft/phi-2\")\n\n# Generate text\nresult = llm.generate(\n \"What is the capital of France?\",\n max_new_tokens=50,\n temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama/Meta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft/phi-2\n\n# Download Gemma 2B\nlitgpt download google/gemma-2b\n```\n\nModels are saved to `checkpoints/` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n {\n \"instruction\": \"What is the capital of France?\",\n \"input\": \"\",\n \"output\": \"The capital of France is Paris.\"\n },\n {\n \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n \"input\": \"\",\n \"output\": \"Hola, ¿cómo estás?\"\n }\n]\n```\n\nSave as `data/my_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n meta-llama/Meta-Llama-3-8B \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --train.max_steps 1000 \\\n --train.learning_rate 2e-5 \\\n --train.micro_batch_size 1 \\\n --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --lora_alpha 32 \\\n --lora_dropout 0.05 \\\n --train.max_steps 1000 \\\n --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out/finetune/` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out/finetune/logs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out/finetune/lightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality/size tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\ # LoRA rank (8-64, higher=more capacity)\n --lora_alpha 32 \\ # LoRA scaling (typically 2×r)\n --lora_dropout 0.05 \\ # Prevent overfitting\n --lora_query true \\ # Apply LoRA to query projection\n --lora_key false \\ # Usually not needed\n --lora_value true \\ # Apply LoRA to value projection\n --lora_projection true \\ # Apply LoRA to output projection\n --lora_mlp false \\ # Usually not needed\n --lora_head false # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --train.epochs 3 \\\n --train.learning_rate 1e-4 \\\n --train.micro_batch_size 4 \\\n --train.global_batch_size 32 \\\n --out_dir out/phi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n out/phi2-lora/final \\\n --out_dir out/phi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out/phi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts/prepare_dataset.py \\\n --source_path data/my_corpus.txt \\\n --checkpoint_dir checkpoints/tokenizer \\\n --destination_path data/pretrain \\\n --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config/pythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n --config config/pythia-160m.yaml \\\n --data.data_dir data/pretrain \\\n --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir data/pretrain \\\n --devices 8 \\\n --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir /shared/data/pretrain \\\n --devices 8 \\\n --num_nodes 8 \\\n --train.global_batch_size 512 \\\n --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out/phi2-lora/final\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --dtype bfloat16 \\\n --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --quantize bnb.nf4-dq # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts/convert_lit_checkpoint.py \\\n --checkpoint_path out/phi2-lora/final \\\n --output_path models/phi2.gguf \\\n --model_name microsoft/phi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out/phi2-lora/final\")\n\n@app.post(\"/generate\")\ndef generate(prompt: str, max_tokens: int = 100):\n result = llm.generate(\n prompt,\n max_new_tokens=max_tokens,\n temperature=0.7\n )\n return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl/TRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n ... \\\n --train.gradient_accumulation_iters 4 # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30/40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32 # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama/Meta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints/\n# Should see: meta-llama/Meta-Llama-3-8B/\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8 # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\ # Disable this\n--lora_mlp false # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references/supported-models.md](references/supported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references/training-recipes.md](references/training-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references/distributed-training.md](references/distributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references/custom-models.md](references/custom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n - Inference (Phi-2): 6GB\n - LoRA fine-tuning (7B): 16GB\n - Full fine-tuning (7B): 40GB+\n - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https://github.com/Lightning-AI/litgpt\n- Docs: https://lightning.ai/docs/litgpt\n- Tutorials: https://lightning.ai/docs/litgpt/tutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n","frontmatter":{"name":"implementing-llms-litgpt","description":"Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Model Architecture","LitGPT","Lightning AI","LLM Implementation","LoRA","QLoRA","Fine-Tuning","Llama","Gemma","Phi","Mistral","Educational"],"dependencies":["litgpt","torch","transformers"]},"isInternal":false,"tokens":3208,"sizeBytes":11010},{"name":"architecture-details.md","path":"01-model-architecture/mamba/references/architecture-details.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/architecture-details.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/mamba/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/architecture-details.md)","tokens":300,"sizeBytes":261},{"name":"benchmarks.md","path":"01-model-architecture/mamba/references/benchmarks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/benchmarks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/mamba/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/benchmarks.md)","tokens":300,"sizeBytes":241},{"name":"training-guide.md","path":"01-model-architecture/mamba/references/training-guide.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/training-guide.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/mamba/references/training-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/training-guide.md)","tokens":300,"sizeBytes":249},{"name":"SKILL.md","path":"01-model-architecture/mamba/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/SKILL.md","title":"mamba-architecture","category":"anthropic-skill","format":"markdown","content":"---\nname: mamba-architecture\ndescription: State-space model with O(n) complexity vs Transformers' O(n²). 5× faster inference, million-token sequences, no KV cache. Selective SSM with hardware-aware design. Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head). Models 130M-2.8B on HuggingFace.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Mamba, State Space Models, SSM, Linear Complexity, Long Context, Efficient Inference, Hardware-Aware, Alternative To Transformers]\ndependencies: [mamba-ssm, torch, transformers, causal-conv1d]\n---\n\n# Mamba - Selective State Space Models\n\n## Quick start\n\nMamba is a state-space model architecture achieving O(n) linear complexity for sequence modeling.\n\n**Installation**:\n```bash\n# Install causal-conv1d (optional, for efficiency)\npip install causal-conv1d>=1.4.0\n\n# Install Mamba\npip install mamba-ssm\n# Or both together\npip install mamba-ssm[causal-conv1d]\n```\n\n**Prerequisites**: Linux, NVIDIA GPU, PyTorch 1.12+, CUDA 11.6+\n\n**Basic usage** (Mamba block):\n```python\nimport torch\nfrom mamba_ssm import Mamba\n\nbatch, length, dim = 2, 64, 16\nx = torch.randn(batch, length, dim).to(\"cuda\")\n\nmodel = Mamba(\n d_model=dim, # Model dimension\n d_state=16, # SSM state dimension\n d_conv=4, # Conv1d kernel size\n expand=2 # Expansion factor\n).to(\"cuda\")\n\ny = model(x) # O(n) complexity!\nassert y.shape == x.shape\n```\n\n## Common workflows\n\n### Workflow 1: Language model with Mamba-2\n\n**Complete LM with generation**:\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nfrom mamba_ssm.models.config_mamba import MambaConfig\nimport torch\n\n# Configure Mamba-2 LM\nconfig = MambaConfig(\n d_model=1024, # Hidden dimension\n n_layer=24, # Number of layers\n vocab_size=50277, # Vocabulary size\n ssm_cfg=dict(\n layer=\"Mamba2\", # Use Mamba-2\n d_state=128, # Larger state for Mamba-2\n headdim=64, # Head dimension\n ngroups=1 # Number of groups\n )\n)\n\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\n\n# Generate text\ninput_ids = torch.randint(0, 1000, (1, 20), device=\"cuda\", dtype=torch.long)\noutput = model.generate(\n input_ids=input_ids,\n max_length=100,\n temperature=0.7,\n top_p=0.9\n)\n```\n\n### Workflow 2: Use pretrained Mamba models\n\n**Load from HuggingFace**:\n```python\nfrom transformers import AutoTokenizer\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\n\n# Load pretrained model\nmodel_name = \"state-spaces/mamba-2.8b\"\ntokenizer = AutoTokenizer.from_pretrained(\"EleutherAI/gpt-neox-20b\") # Use compatible tokenizer\nmodel = MambaLMHeadModel.from_pretrained(model_name, device=\"cuda\", dtype=torch.float16)\n\n# Generate\nprompt = \"The future of AI is\"\ninput_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(\"cuda\")\noutput_ids = model.generate(\n input_ids=input_ids,\n max_length=200,\n temperature=0.7,\n top_p=0.9,\n repetition_penalty=1.2\n)\ngenerated_text = tokenizer.decode(output_ids[0])\nprint(generated_text)\n```\n\n**Available models**:\n- `state-spaces/mamba-130m`\n- `state-spaces/mamba-370m`\n- `state-spaces/mamba-790m`\n- `state-spaces/mamba-1.4b`\n- `state-spaces/mamba-2.8b`\n\n### Workflow 3: Mamba-1 vs Mamba-2\n\n**Mamba-1** (smaller state):\n```python\nfrom mamba_ssm import Mamba\n\nmodel = Mamba(\n d_model=256,\n d_state=16, # Smaller state dimension\n d_conv=4,\n expand=2\n).to(\"cuda\")\n```\n\n**Mamba-2** (multi-head, larger state):\n```python\nfrom mamba_ssm import Mamba2\n\nmodel = Mamba2(\n d_model=256,\n d_state=128, # Larger state dimension\n d_conv=4,\n expand=2,\n headdim=64, # Head dimension for multi-head\n ngroups=1 # Parallel groups\n).to(\"cuda\")\n```\n\n**Key differences**:\n- **State size**: Mamba-1 (d_state=16) vs Mamba-2 (d_state=128)\n- **Architecture**: Mamba-2 has multi-head structure\n- **Normalization**: Mamba-2 uses RMSNorm\n- **Distributed**: Mamba-2 supports tensor parallelism\n\n### Workflow 4: Benchmark vs Transformers\n\n**Generation speed comparison**:\n```bash\n# Benchmark Mamba\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"state-spaces/mamba-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n\n# Benchmark Transformer\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"EleutherAI/pythia-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n```\n\n**Expected results**:\n- **Mamba**: 5× faster inference\n- **Memory**: No KV cache needed\n- **Scaling**: Linear with sequence length\n\n## When to use vs alternatives\n\n**Use Mamba when**:\n- Need long sequences (100K+ tokens)\n- Want faster inference than Transformers\n- Memory-constrained (no KV cache)\n- Building streaming applications\n- Linear scaling important\n\n**Advantages**:\n- **O(n) complexity**: Linear vs quadratic\n- **5× faster inference**: No attention overhead\n- **No KV cache**: Lower memory usage\n- **Million-token sequences**: Hardware-efficient\n- **Streaming**: Constant memory per token\n\n**Use alternatives instead**:\n- **Transformers**: Need best-in-class performance, have compute\n- **RWKV**: Want RNN+Transformer hybrid\n- **RetNet**: Need retention-based architecture\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or use gradient checkpointing:\n```python\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\nmodel.gradient_checkpointing_enable() # Enable checkpointing\n```\n\n**Issue: Slow installation**\n\nInstall binary wheels (not source):\n```bash\npip install mamba-ssm --no-build-isolation\n```\n\n**Issue: Missing causal-conv1d**\n\nInstall separately:\n```bash\npip install causal-conv1d>=1.4.0\n```\n\n**Issue: Model not loading from HuggingFace**\n\nUse `MambaLMHeadModel.from_pretrained` (not `AutoModel`):\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nmodel = MambaLMHeadModel.from_pretrained(\"state-spaces/mamba-2.8b\")\n```\n\n## Advanced topics\n\n**Selective SSM**: See [references/selective-ssm.md](references/selective-ssm.md) for mathematical formulation, state-space equations, and how selectivity enables O(n) complexity.\n\n**Mamba-2 architecture**: See [references/mamba2-details.md](references/mamba2-details.md) for multi-head structure, tensor parallelism, and distributed training setup.\n\n**Performance optimization**: See [references/performance.md](references/performance.md) for hardware-aware design, CUDA kernels, and memory efficiency techniques.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA with CUDA 11.6+\n- **VRAM**:\n - 130M model: 2GB\n - 370M model: 4GB\n - 790M model: 8GB\n - 1.4B model: 14GB\n - 2.8B model: 28GB (FP16)\n- **Inference**: 5× faster than Transformers\n- **Memory**: No KV cache (lower than Transformers)\n\n**Performance** (vs Transformers):\n- **Speed**: 5× faster inference\n- **Memory**: 50% less (no KV cache)\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (Mamba-1): https://arxiv.org/abs/2312.00752 (Dec 2023)\n- Paper (Mamba-2): https://arxiv.org/abs/2405.21060 (May 2024)\n- GitHub: https://github.com/state-spaces/mamba ⭐ 13,000+\n- Models: https://huggingface.co/state-spaces\n- Docs: Repository README and wiki\n\n\n","frontmatter":{"name":"mamba-architecture","description":"State-space model with O(n) complexity vs Transformers' O(n²). 5× faster inference, million-token sequences, no KV cache. Selective SSM with hardware-aware design. Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head). Models 130M-2.8B on HuggingFace.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Model Architecture","Mamba","State Space Models","SSM","Linear Complexity","Long Context","Efficient Inference","Hardware-Aware","Alternative To Transformers"],"dependencies":["mamba-ssm","torch","transformers","causal-conv1d"]},"isInternal":false,"tokens":2070,"sizeBytes":7368},{"name":"architecture.md","path":"01-model-architecture/nanogpt/references/architecture.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/architecture.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/architecture.md)","tokens":300,"sizeBytes":249},{"name":"data.md","path":"01-model-architecture/nanogpt/references/data.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/data.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/data.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/data.md)","tokens":300,"sizeBytes":233},{"name":"training.md","path":"01-model-architecture/nanogpt/references/training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/nanogpt/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/training.md)","tokens":300,"sizeBytes":241},{"name":"SKILL.md","path":"01-model-architecture/nanogpt/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/SKILL.md","title":"nanogpt","category":"anthropic-skill","format":"markdown","content":"---\nname: nanogpt\ndescription: Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, NanoGPT, GPT-2, Educational, Andrej Karpathy, Transformer, Minimalist, From Scratch, Training]\ndependencies: [torch, transformers, datasets, tiktoken, wandb]\n---\n\n# nanoGPT - Minimalist GPT Training\n\n## Quick start\n\nnanoGPT is a simplified GPT implementation designed for learning and experimentation.\n\n**Installation**:\n```bash\npip install torch numpy transformers datasets tiktoken wandb tqdm\n```\n\n**Train on Shakespeare** (CPU-friendly):\n```bash\n# Prepare data\npython data/shakespeare_char/prepare.py\n\n# Train (5 minutes on CPU)\npython train.py config/train_shakespeare_char.py\n\n# Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Output**:\n```\nROMEO:\nWhat say'st thou? Shall I speak, and be a man?\n\nJULIET:\nI am afeard, and yet I'll speak; for thou art\nOne that hath been a man, and yet I know not\nWhat thou art.\n```\n\n## Common workflows\n\n### Workflow 1: Character-level Shakespeare\n\n**Complete training pipeline**:\n```bash\n# Step 1: Prepare data (creates train.bin, val.bin)\npython data/shakespeare_char/prepare.py\n\n# Step 2: Train small model\npython train.py config/train_shakespeare_char.py\n\n# Step 3: Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Config** (`config/train_shakespeare_char.py`):\n```python\n# Model config\nn_layer = 6 # 6 transformer layers\nn_head = 6 # 6 attention heads\nn_embd = 384 # 384-dim embeddings\nblock_size = 256 # 256 char context\n\n# Training config\nbatch_size = 64\nlearning_rate = 1e-3\nmax_iters = 5000\neval_interval = 500\n\n# Hardware\ndevice = 'cpu' # Or 'cuda'\ncompile = False # Set True for PyTorch 2.0\n```\n\n**Training time**: ~5 minutes (CPU), ~1 minute (GPU)\n\n### Workflow 2: Reproduce GPT-2 (124M)\n\n**Multi-GPU training on OpenWebText**:\n```bash\n# Step 1: Prepare OpenWebText (takes ~1 hour)\npython data/openwebtext/prepare.py\n\n# Step 2: Train GPT-2 124M with DDP (8 GPUs)\ntorchrun --standalone --nproc_per_node=8 \\\n train.py config/train_gpt2.py\n\n# Step 3: Sample from trained model\npython sample.py --out_dir=out\n```\n\n**Config** (`config/train_gpt2.py`):\n```python\n# GPT-2 (124M) architecture\nn_layer = 12\nn_head = 12\nn_embd = 768\nblock_size = 1024\ndropout = 0.0\n\n# Training\nbatch_size = 12\ngradient_accumulation_steps = 5 * 8 # Total batch ~0.5M tokens\nlearning_rate = 6e-4\nmax_iters = 600000\nlr_decay_iters = 600000\n\n# System\ncompile = True # PyTorch 2.0\n```\n\n**Training time**: ~4 days (8× A100)\n\n### Workflow 3: Fine-tune pretrained GPT-2\n\n**Start from OpenAI checkpoint**:\n```python\n# In train.py or config\ninit_from = 'gpt2' # Options: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n\n# Model loads OpenAI weights automatically\npython train.py config/finetune_shakespeare.py\n```\n\n**Example config** (`config/finetune_shakespeare.py`):\n```python\n# Start from GPT-2\ninit_from = 'gpt2'\n\n# Dataset\ndataset = 'shakespeare_char'\nbatch_size = 1\nblock_size = 1024\n\n# Fine-tuning\nlearning_rate = 3e-5 # Lower LR for fine-tuning\nmax_iters = 2000\nwarmup_iters = 100\n\n# Regularization\nweight_decay = 1e-1\n```\n\n### Workflow 4: Custom dataset\n\n**Train on your own text**:\n```python\n# data/custom/prepare.py\nimport numpy as np\n\n# Load your data\nwith open('my_data.txt', 'r') as f:\n text = f.read()\n\n# Create character mappings\nchars = sorted(list(set(text)))\nstoi = {ch: i for i, ch in enumerate(chars)}\nitos = {i: ch for i, ch in enumerate(chars)}\n\n# Tokenize\ndata = np.array([stoi[ch] for ch in text], dtype=np.uint16)\n\n# Split train/val\nn = len(data)\ntrain_data = data[:int(n*0.9)]\nval_data = data[int(n*0.9):]\n\n# Save\ntrain_data.tofile('data/custom/train.bin')\nval_data.tofile('data/custom/val.bin')\n```\n\n**Train**:\n```bash\npython data/custom/prepare.py\npython train.py --dataset=custom\n```\n\n## When to use vs alternatives\n\n**Use nanoGPT when**:\n- Learning how GPT works\n- Experimenting with transformer variants\n- Teaching/education purposes\n- Quick prototyping\n- Limited compute (can run on CPU)\n\n**Simplicity advantages**:\n- **~300 lines**: Entire model in `model.py`\n- **~300 lines**: Training loop in `train.py`\n- **Hackable**: Easy to modify\n- **No abstractions**: Pure PyTorch\n\n**Use alternatives instead**:\n- **HuggingFace Transformers**: Production use, many models\n- **Megatron-LM**: Large-scale distributed training\n- **LitGPT**: More architectures, production-ready\n- **PyTorch Lightning**: Need high-level framework\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or context length:\n```python\nbatch_size = 1 # Reduce from 12\nblock_size = 512 # Reduce from 1024\ngradient_accumulation_steps = 40 # Increase to maintain effective batch\n```\n\n**Issue: Training too slow**\n\nEnable compilation (PyTorch 2.0+):\n```python\ncompile = True # 2× speedup\n```\n\nUse mixed precision:\n```python\ndtype = 'bfloat16' # Or 'float16'\n```\n\n**Issue: Poor generation quality**\n\nTrain longer:\n```python\nmax_iters = 10000 # Increase from 5000\n```\n\nLower temperature:\n```python\n# In sample.py\ntemperature = 0.7 # Lower from 1.0\ntop_k = 200 # Add top-k sampling\n```\n\n**Issue: Can't load GPT-2 weights**\n\nInstall transformers:\n```bash\npip install transformers\n```\n\nCheck model name:\n```python\ninit_from = 'gpt2' # Valid: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n```\n\n## Advanced topics\n\n**Model architecture**: See [references/architecture.md](references/architecture.md) for GPT block structure, multi-head attention, and MLP layers explained simply.\n\n**Training loop**: See [references/training.md](references/training.md) for learning rate schedule, gradient accumulation, and distributed data parallel setup.\n\n**Data preparation**: See [references/data.md](references/data.md) for tokenization strategies (character-level vs BPE) and binary format details.\n\n## Hardware requirements\n\n- **Shakespeare (char-level)**:\n - CPU: 5 minutes\n - GPU (T4): 1 minute\n - VRAM: <1GB\n\n- **GPT-2 (124M)**:\n - 1× A100: ~1 week\n - 8× A100: ~4 days\n - VRAM: ~16GB per GPU\n\n- **GPT-2 Medium (350M)**:\n - 8× A100: ~2 weeks\n - VRAM: ~40GB per GPU\n\n**Performance**:\n- With `compile=True`: 2× speedup\n- With `dtype=bfloat16`: 50% memory reduction\n\n## Resources\n\n- GitHub: https://github.com/karpathy/nanoGPT ⭐ 48,000+\n- Video: \"Let's build GPT\" by Andrej Karpathy\n- Paper: \"Attention is All You Need\" (Vaswani et al.)\n- OpenWebText: https://huggingface.co/datasets/Skylion007/openwebtext\n- Educational: Best for understanding transformers from scratch\n\n\n","frontmatter":{"name":"nanogpt","description":"Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Model Architecture","NanoGPT","GPT-2","Educational","Andrej Karpathy","Transformer","Minimalist","From Scratch","Training"],"dependencies":["torch","transformers","datasets","tiktoken","wandb"]},"isInternal":false,"tokens":1957,"sizeBytes":6752},{"name":"architecture-details.md","path":"01-model-architecture/rwkv/references/architecture-details.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/architecture-details.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/architecture-details.md)","tokens":300,"sizeBytes":259},{"name":"rwkv7.md","path":"01-model-architecture/rwkv/references/rwkv7.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/rwkv7.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/rwkv7.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/rwkv7.md)","tokens":300,"sizeBytes":229},{"name":"state-management.md","path":"01-model-architecture/rwkv/references/state-management.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/state-management.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/rwkv/references/state-management.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/state-management.md)","tokens":300,"sizeBytes":251},{"name":"SKILL.md","path":"01-model-architecture/rwkv/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/SKILL.md","title":"rwkv-architecture","category":"anthropic-skill","format":"markdown","content":"---\nname: rwkv-architecture\ndescription: RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [RWKV, Model Architecture, RNN, Transformer Hybrid, Linear Complexity, Infinite Context, Efficient Inference, Linux Foundation, Alternative Architecture]\ndependencies: [rwkv, torch, transformers]\n---\n\n# RWKV - Receptance Weighted Key Value\n\n## Quick start\n\nRWKV (RwaKuv) combines Transformer parallelization (training) with RNN efficiency (inference).\n\n**Installation**:\n```bash\n# Install PyTorch\npip install torch --upgrade --extra-index-url https://download.pytorch.org/whl/cu121\n\n# Install dependencies\npip install pytorch-lightning==1.9.5 deepspeed wandb ninja --upgrade\n\n# Install RWKV\npip install rwkv\n```\n\n**Basic usage** (GPT mode + RNN mode):\n```python\nimport os\nfrom rwkv.model import RWKV\n\nos.environ[\"RWKV_JIT_ON\"] = '1'\nos.environ[\"RWKV_CUDA_ON\"] = '1' # Use CUDA kernel for speed\n\n# Load model\nmodel = RWKV(\n model='/path/to/RWKV-4-Pile-1B5-20220903-8040',\n strategy='cuda fp16'\n)\n\n# GPT mode (parallel processing)\nout, state = model.forward([187, 510, 1563, 310, 247], None)\nprint(out.detach().cpu().numpy()) # Logits\n\n# RNN mode (sequential processing, same result)\nout, state = model.forward([187, 510], None) # First 2 tokens\nout, state = model.forward([1563], state) # Next token\nout, state = model.forward([310, 247], state) # Last tokens\nprint(out.detach().cpu().numpy()) # Same logits as above!\n```\n\n## Common workflows\n\n### Workflow 1: Text generation (streaming)\n\n**Efficient token-by-token generation**:\n```python\nfrom rwkv.model import RWKV\nfrom rwkv.utils import PIPELINE\n\nmodel = RWKV(model='RWKV-4-Pile-14B-20230313-ctx8192-test1050', strategy='cuda fp16')\npipeline = PIPELINE(model, \"20B_tokenizer.json\")\n\n# Initial prompt\nprompt = \"The future of AI is\"\nstate = None\n\n# Generate token by token\nfor token in prompt:\n out, state = pipeline.model.forward(pipeline.encode(token), state)\n\n# Continue generation\nfor _ in range(100):\n out, state = pipeline.model.forward(None, state)\n token = pipeline.sample_logits(out)\n print(pipeline.decode(token), end='', flush=True)\n```\n\n**Key advantage**: Constant memory per token (no growing KV cache)\n\n### Workflow 2: Long context processing (infinite context)\n\n**Process million-token sequences**:\n```python\nmodel = RWKV(model='RWKV-4-Pile-14B', strategy='cuda fp16')\n\n# Process very long document\nstate = None\nlong_document = load_document() # e.g., 1M tokens\n\n# Stream through entire document\nfor chunk in chunks(long_document, chunk_size=1024):\n out, state = model.forward(chunk, state)\n\n# State now contains information from entire 1M token document\n# Memory usage: O(1) (constant, not O(n)!)\n```\n\n### Workflow 3: Fine-tuning RWKV\n\n**Standard fine-tuning workflow**:\n```python\n# Training script\nimport pytorch_lightning as pl\nfrom rwkv.model import RWKV\nfrom rwkv.trainer import RWKVTrainer\n\n# Configure model\nconfig = {\n 'n_layer': 24,\n 'n_embd': 1024,\n 'vocab_size': 50277,\n 'ctx_len': 1024\n}\n\n# Setup trainer\ntrainer = pl.Trainer(\n accelerator='gpu',\n devices=8,\n precision='bf16',\n strategy='deepspeed_stage_2',\n max_epochs=1\n)\n\n# Train\nmodel = RWKV(config)\ntrainer.fit(model, train_dataloader)\n```\n\n### Workflow 4: RWKV vs Transformer comparison\n\n**Memory comparison** (1M token sequence):\n```python\n# Transformer (GPT)\n# Memory: O(n²) for attention\n# KV cache: 1M × hidden_dim × n_layers × 2 (keys + values)\n# Example: 1M × 4096 × 24 × 2 = ~400GB (impractical!)\n\n# RWKV\n# Memory: O(1) per token\n# State: hidden_dim × n_layers = 4096 × 24 = ~400KB\n# 1,000,000× more efficient!\n```\n\n**Speed comparison** (inference):\n```python\n# Transformer: O(n) per token (quadratic overall)\n# First token: 1 computation\n# Second token: 2 computations\n# ...\n# 1000th token: 1000 computations\n\n# RWKV: O(1) per token (linear overall)\n# Every token: 1 computation\n# 1000th token: 1 computation (same as first!)\n```\n\n## When to use vs alternatives\n\n**Use RWKV when**:\n- Need very long context (100K+ tokens)\n- Want constant memory usage\n- Building streaming applications\n- Need RNN efficiency with Transformer performance\n- Memory-constrained deployment\n\n**Key advantages**:\n- **Linear time**: O(n) vs O(n²) for Transformers\n- **No KV cache**: Constant memory per token\n- **Infinite context**: No fixed window limit\n- **Parallelizable training**: Like GPT\n- **Sequential inference**: Like RNN\n\n**Use alternatives instead**:\n- **Transformers**: Need absolute best performance, have compute\n- **Mamba**: Want state-space models\n- **RetNet**: Need retention mechanism\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: Out of memory during training**\n\nUse gradient checkpointing and DeepSpeed:\n```python\ntrainer = pl.Trainer(\n strategy='deepspeed_stage_3', # Full ZeRO-3\n precision='bf16'\n)\n```\n\n**Issue: Slow inference**\n\nEnable CUDA kernel:\n```python\nos.environ[\"RWKV_CUDA_ON\"] = '1'\n```\n\n**Issue: Model not loading**\n\nCheck model path and strategy:\n```python\nmodel = RWKV(\n model='/absolute/path/to/model.pth',\n strategy='cuda fp16' # Or 'cpu fp32' for CPU\n)\n```\n\n**Issue: State management in RNN mode**\n\nAlways pass state between forward calls:\n```python\n# WRONG: State lost\nout1, _ = model.forward(tokens1, None)\nout2, _ = model.forward(tokens2, None) # No context from tokens1!\n\n# CORRECT: State preserved\nout1, state = model.forward(tokens1, None)\nout2, state = model.forward(tokens2, state) # Has context from tokens1\n```\n\n## Advanced topics\n\n**Time-mixing and channel-mixing**: See [references/architecture-details.md](references/architecture-details.md) for WKV operation, time-decay mechanism, and receptance gates.\n\n**State management**: See [references/state-management.md](references/state-management.md) for att_x_prev, att_kv, ffn_x_prev states, and numerical stability considerations.\n\n**RWKV-7 improvements**: See [references/rwkv7.md](references/rwkv7.md) for latest architectural improvements (March 2025) and multimodal capabilities.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.6+) or CPU\n- **VRAM** (FP16):\n - 169M model: 1GB\n - 430M model: 2GB\n - 1.5B model: 4GB\n - 3B model: 8GB\n - 7B model: 16GB\n - 14B model: 32GB\n- **Inference**: O(1) memory per token\n- **Training**: Parallelizable like GPT\n\n**Performance** (vs Transformers):\n- **Speed**: Similar training, faster inference\n- **Memory**: 1000× less for long sequences\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (RWKV): https://arxiv.org/abs/2305.13048 (May 2023)\n- Paper (RWKV-7): https://arxiv.org/abs/2503.14456 (March 2025)\n- GitHub: https://github.com/BlinkDL/RWKV-LM ⭐ 12,000+\n- Docs: https://wiki.rwkv.com/\n- Models: https://huggingface.co/BlinkDL\n- Linux Foundation AI: Official project\n- Production: Microsoft Windows, Office integration, NeMo support\n\n\n","frontmatter":{"name":"rwkv-architecture","description":"RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["RWKV","Model Architecture","RNN","Transformer Hybrid","Linear Complexity","Infinite Context","Efficient Inference","Linux Foundation","Alternative Architecture"],"dependencies":["rwkv","torch","transformers"]},"isInternal":false,"tokens":1978,"sizeBytes":7099},{"name":"checkpoint.md","path":"01-model-architecture/torchtitan/references/checkpoint.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/checkpoint.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/checkpoint.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/checkpoint.md)","tokens":300,"sizeBytes":251},{"name":"custom-models.md","path":"01-model-architecture/torchtitan/references/custom-models.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/custom-models.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/custom-models.md)","tokens":300,"sizeBytes":257},{"name":"float8.md","path":"01-model-architecture/torchtitan/references/float8.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/float8.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/float8.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/float8.md)","tokens":300,"sizeBytes":243},{"name":"fsdp.md","path":"01-model-architecture/torchtitan/references/fsdp.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/fsdp.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `01-model-architecture/torchtitan/references/fsdp.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/fsdp.md)","tokens":300,"sizeBytes":239},{"name":"SKILL.md","path":"01-model-architecture/torchtitan/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/SKILL.md","title":"distributed-llm-pretraining-torchtitan","category":"anthropic-skill","format":"markdown","content":"---\nname: distributed-llm-pretraining-torchtitan\ndescription: Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Distributed Training, TorchTitan, FSDP2, Tensor Parallel, Pipeline Parallel, Context Parallel, Float8, Llama, Pretraining]\ndependencies: [torch>=2.6.0, torchtitan>=0.2.0, torchao>=0.5.0]\n---\n\n# TorchTitan - PyTorch Native Distributed LLM Pretraining\n\n## Quick start\n\nTorchTitan is PyTorch's official platform for large-scale LLM pretraining with composable 4D parallelism (FSDP2, TP, PP, CP), achieving 65%+ speedups over baselines on H100 GPUs.\n\n**Installation**:\n```bash\n# From PyPI (stable)\npip install torchtitan\n\n# From source (latest features, requires PyTorch nightly)\ngit clone https://github.com/pytorch/torchtitan\ncd torchtitan\npip install -r requirements.txt\n```\n\n**Download tokenizer**:\n```bash\n# Get HF token from https://huggingface.co/settings/tokens\npython scripts/download_hf_assets.py --repo_id meta-llama/Llama-3.1-8B --assets tokenizer --hf_token=...\n```\n\n**Start training on 8 GPUs**:\n```bash\nCONFIG_FILE=\"./torchtitan/models/llama3/train_configs/llama3_8b.toml\" ./run_train.sh\n```\n\n## Common workflows\n\n### Workflow 1: Pretrain Llama 3.1 8B on single node\n\nCopy this checklist:\n\n```\nSingle Node Pretraining:\n- [ ] Step 1: Download tokenizer\n- [ ] Step 2: Configure training\n- [ ] Step 3: Launch training\n- [ ] Step 4: Monitor and checkpoint\n```\n\n**Step 1: Download tokenizer**\n\n```bash\npython scripts/download_hf_assets.py \\\n --repo_id meta-llama/Llama-3.1-8B \\\n --assets tokenizer \\\n --hf_token=YOUR_HF_TOKEN\n```\n\n**Step 2: Configure training**\n\nEdit or create a TOML config file:\n\n```toml\n# llama3_8b_custom.toml\n[job]\ndump_folder = \"./outputs\"\ndescription = \"Llama 3.1 8B training\"\n\n[model]\nname = \"llama3\"\nflavor = \"8B\"\nhf_assets_path = \"./assets/hf/Llama-3.1-8B\"\n\n[optimizer]\nname = \"AdamW\"\nlr = 3e-4\n\n[lr_scheduler]\nwarmup_steps = 200\n\n[training]\nlocal_batch_size = 2\nseq_len = 8192\nmax_norm = 1.0\nsteps = 1000\ndataset = \"c4\"\n\n[parallelism]\ndata_parallel_shard_degree = -1 # Use all GPUs for FSDP\n\n[activation_checkpoint]\nmode = \"selective\"\nselective_ac_option = \"op\"\n\n[checkpoint]\nenable = true\nfolder = \"checkpoint\"\ninterval = 500\n```\n\n**Step 3: Launch training**\n\n```bash\n# 8 GPUs on single node\nCONFIG_FILE=\"./llama3_8b_custom.toml\" ./run_train.sh\n\n# Or explicitly with torchrun\ntorchrun --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_8b_custom.toml\n```\n\n**Step 4: Monitor and checkpoint**\n\nTensorBoard logs are saved to `./outputs/tb/`:\n```bash\ntensorboard --logdir ./outputs/tb\n```\n\n### Workflow 2: Multi-node training with SLURM\n\n```\nMulti-Node Training:\n- [ ] Step 1: Configure parallelism for scale\n- [ ] Step 2: Set up SLURM script\n- [ ] Step 3: Submit job\n- [ ] Step 4: Resume from checkpoint\n```\n\n**Step 1: Configure parallelism for scale**\n\nFor 70B model on 256 GPUs (32 nodes):\n```toml\n[parallelism]\ndata_parallel_shard_degree = 32 # FSDP across 32 ranks\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 1 # No PP for 70B\ncontext_parallel_degree = 1 # Increase for long sequences\n```\n\n**Step 2: Set up SLURM script**\n\n```bash\n#!/bin/bash\n#SBATCH --job-name=llama70b\n#SBATCH --nodes=32\n#SBATCH --ntasks-per-node=8\n#SBATCH --gpus-per-node=8\n\nsrun torchrun \\\n --nnodes=32 \\\n --nproc_per_node=8 \\\n --rdzv_backend=c10d \\\n --rdzv_endpoint=$MASTER_ADDR:$MASTER_PORT \\\n -m torchtitan.train \\\n --job.config_file ./llama3_70b.toml\n```\n\n**Step 3: Submit job**\n\n```bash\nsbatch multinode_trainer.slurm\n```\n\n**Step 4: Resume from checkpoint**\n\nTraining auto-resumes if checkpoint exists in configured folder.\n\n### Workflow 3: Enable Float8 training for H100s\n\nFloat8 provides 30-50% speedup on H100 GPUs.\n\n```\nFloat8 Training:\n- [ ] Step 1: Install torchao\n- [ ] Step 2: Configure Float8\n- [ ] Step 3: Launch with compile\n```\n\n**Step 1: Install torchao**\n\n```bash\nUSE_CPP=0 pip install git+https://github.com/pytorch/ao.git\n```\n\n**Step 2: Configure Float8**\n\nAdd to your TOML config:\n```toml\n[model]\nconverters = [\"quantize.linear.float8\"]\n\n[quantize.linear.float8]\nenable_fsdp_float8_all_gather = true\nprecompute_float8_dynamic_scale_for_fsdp = true\nfilter_fqns = [\"output\"] # Exclude output layer\n\n[compile]\nenable = true\ncomponents = [\"model\", \"loss\"]\n```\n\n**Step 3: Launch with compile**\n\n```bash\nCONFIG_FILE=\"./llama3_8b.toml\" ./run_train.sh \\\n --model.converters=\"quantize.linear.float8\" \\\n --quantize.linear.float8.enable_fsdp_float8_all_gather \\\n --compile.enable\n```\n\n### Workflow 4: 4D parallelism for 405B models\n\n```\n4D Parallelism (FSDP + TP + PP + CP):\n- [ ] Step 1: Create seed checkpoint\n- [ ] Step 2: Configure 4D parallelism\n- [ ] Step 3: Launch on 512 GPUs\n```\n\n**Step 1: Create seed checkpoint**\n\nRequired for consistent initialization across PP stages:\n```bash\nNGPU=1 CONFIG_FILE=./llama3_405b.toml ./run_train.sh \\\n --checkpoint.enable \\\n --checkpoint.create_seed_checkpoint \\\n --parallelism.data_parallel_shard_degree 1 \\\n --parallelism.tensor_parallel_degree 1 \\\n --parallelism.pipeline_parallel_degree 1\n```\n\n**Step 2: Configure 4D parallelism**\n\n```toml\n[parallelism]\ndata_parallel_shard_degree = 8 # FSDP\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 8 # PP across nodes\ncontext_parallel_degree = 1 # CP for long sequences\n\n[training]\nlocal_batch_size = 32\nseq_len = 8192\n```\n\n**Step 3: Launch on 512 GPUs**\n\n```bash\n# 64 nodes x 8 GPUs = 512 GPUs\nsrun torchrun --nnodes=64 --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_405b.toml\n```\n\n## When to use vs alternatives\n\n**Use TorchTitan when:**\n- Pretraining LLMs from scratch (8B to 405B+)\n- Need PyTorch-native solution without third-party dependencies\n- Require composable 4D parallelism (FSDP2, TP, PP, CP)\n- Training on H100s with Float8 support\n- Want interoperable checkpoints with torchtune/HuggingFace\n\n**Use alternatives instead:**\n- **Megatron-LM**: Maximum performance for NVIDIA-only deployments\n- **DeepSpeed**: Broader ZeRO optimization ecosystem, inference support\n- **Axolotl/TRL**: Fine-tuning rather than pretraining\n- **LitGPT**: Educational, smaller-scale training\n\n## Common issues\n\n**Issue: Out of memory on large models**\n\nEnable activation checkpointing and reduce batch size:\n```toml\n[activation_checkpoint]\nmode = \"full\" # Instead of \"selective\"\n\n[training]\nlocal_batch_size = 1\n```\n\nOr use gradient accumulation:\n```toml\n[training]\nlocal_batch_size = 1\nglobal_batch_size = 32 # Accumulates gradients\n```\n\n**Issue: TP causes high memory with async collectives**\n\nSet environment variable:\n```bash\nexport TORCH_NCCL_AVOID_RECORD_STREAMS=1\n```\n\n**Issue: Float8 training not faster**\n\nFloat8 only benefits large GEMMs. Filter small layers:\n```toml\n[quantize.linear.float8]\nfilter_fqns = [\"attention.wk\", \"attention.wv\", \"output\", \"auto_filter_small_kn\"]\n```\n\n**Issue: Checkpoint loading fails after parallelism change**\n\nUse DCP's resharding capability:\n```bash\n# Convert sharded checkpoint to single file\npython -m torch.distributed.checkpoint.format_utils \\\n dcp_to_torch checkpoint/step-1000 checkpoint.pt\n```\n\n**Issue: Pipeline parallelism initialization**\n\nCreate seed checkpoint first (see Workflow 4, Step 1).\n\n## Supported models\n\n| Model | Sizes | Status |\n|-------|-------|--------|\n| Llama 3.1 | 8B, 70B, 405B | Production |\n| Llama 4 | Various | Experimental |\n| DeepSeek V3 | 16B, 236B, 671B (MoE) | Experimental |\n| GPT-OSS | 20B, 120B (MoE) | Experimental |\n| Qwen 3 | Various | Experimental |\n| Flux | Diffusion | Experimental |\n\n## Performance benchmarks (H100)\n\n| Model | GPUs | Parallelism | TPS/GPU | Techniques |\n|-------|------|-------------|---------|------------|\n| Llama 8B | 8 | FSDP | 5,762 | Baseline |\n| Llama 8B | 8 | FSDP+compile+FP8 | 8,532 | +48% |\n| Llama 70B | 256 | FSDP+TP+AsyncTP | 876 | 2D parallel |\n| Llama 405B | 512 | FSDP+TP+PP | 128 | 3D parallel |\n\n## Advanced topics\n\n**FSDP2 configuration**: See [references/fsdp.md](references/fsdp.md) for detailed FSDP2 vs FSDP1 comparison and ZeRO equivalents.\n\n**Float8 training**: See [references/float8.md](references/float8.md) for tensorwise vs rowwise scaling recipes.\n\n**Checkpointing**: See [references/checkpoint.md](references/checkpoint.md) for HuggingFace conversion and async checkpointing.\n\n**Adding custom models**: See [references/custom-models.md](references/custom-models.md) for TrainSpec protocol.\n\n## Resources\n\n- GitHub: https://github.com/pytorch/torchtitan\n- Paper: https://arxiv.org/abs/2410.06511\n- ICLR 2025: https://iclr.cc/virtual/2025/poster/29620\n- PyTorch Forum: https://discuss.pytorch.org/c/distributed/torchtitan/44\n\n","frontmatter":{"name":"distributed-llm-pretraining-torchtitan","description":"Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Model Architecture","Distributed Training","TorchTitan","FSDP2","Tensor Parallel","Pipeline Parallel","Context Parallel","Float8","Llama","Pretraining"],"dependencies":["torch>=2.6.0","torchtitan>=0.2.0","torchao>=0.5.0"]},"isInternal":false,"tokens":2631,"sizeBytes":8927},{"name":"algorithms.md","path":"02-tokenization/huggingface-tokenizers/references/algorithms.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/algorithms.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/algorithms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/algorithms.md)","tokens":300,"sizeBytes":263},{"name":"integration.md","path":"02-tokenization/huggingface-tokenizers/references/integration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/integration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/integration.md)","tokens":300,"sizeBytes":265},{"name":"pipeline.md","path":"02-tokenization/huggingface-tokenizers/references/pipeline.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/pipeline.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/pipeline.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/pipeline.md)","tokens":300,"sizeBytes":259},{"name":"training.md","path":"02-tokenization/huggingface-tokenizers/references/training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/training.md)","tokens":300,"sizeBytes":259},{"name":"SKILL.md","path":"02-tokenization/huggingface-tokenizers/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/SKILL.md","title":"huggingface-tokenizers","category":"anthropic-skill","format":"markdown","content":"---\nname: huggingface-tokenizers\ndescription: Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Tokenization, HuggingFace, BPE, WordPiece, Unigram, Fast Tokenization, Rust, Custom Tokenizer, Alignment Tracking, Production]\ndependencies: [tokenizers, transformers, datasets]\n---\n\n# HuggingFace Tokenizers - Fast Tokenization for NLP\n\nFast, production-ready tokenizers with Rust performance and Python ease-of-use.\n\n## When to use HuggingFace Tokenizers\n\n**Use HuggingFace Tokenizers when:**\n- Need extremely fast tokenization (<20s per GB of text)\n- Training custom tokenizers from scratch\n- Want alignment tracking (token → original text position)\n- Building production NLP pipelines\n- Need to tokenize large corpora efficiently\n\n**Performance**:\n- **Speed**: <20 seconds to tokenize 1GB on CPU\n- **Implementation**: Rust core with Python/Node.js bindings\n- **Efficiency**: 10-100× faster than pure Python implementations\n\n**Use alternatives instead**:\n- **SentencePiece**: Language-independent, used by T5/ALBERT\n- **tiktoken**: OpenAI's BPE tokenizer for GPT models\n- **transformers AutoTokenizer**: Loading pretrained only (uses this library internally)\n\n## Quick start\n\n### Installation\n\n```bash\n# Install tokenizers\npip install tokenizers\n\n# With transformers integration\npip install tokenizers transformers\n```\n\n### Load pretrained tokenizer\n\n```python\nfrom tokenizers import Tokenizer\n\n# Load from HuggingFace Hub\ntokenizer = Tokenizer.from_pretrained(\"bert-base-uncased\")\n\n# Encode text\noutput = tokenizer.encode(\"Hello, how are you?\")\nprint(output.tokens) # ['hello', ',', 'how', 'are', 'you', '?']\nprint(output.ids) # [7592, 1010, 2129, 2024, 2017, 1029]\n\n# Decode back\ntext = tokenizer.decode(output.ids)\nprint(text) # \"hello, how are you?\"\n```\n\n### Train custom BPE tokenizer\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import BPE\nfrom tokenizers.trainers import BpeTrainer\nfrom tokenizers.pre_tokenizers import Whitespace\n\n# Initialize tokenizer with BPE model\ntokenizer = Tokenizer(BPE(unk_token=\"[UNK]\"))\ntokenizer.pre_tokenizer = Whitespace()\n\n# Configure trainer\ntrainer = BpeTrainer(\n vocab_size=30000,\n special_tokens=[\"[UNK]\", \"[CLS]\", \"[SEP]\", \"[PAD]\", \"[MASK]\"],\n min_frequency=2\n)\n\n# Train on files\nfiles = [\"train.txt\", \"validation.txt\"]\ntokenizer.train(files, trainer)\n\n# Save\ntokenizer.save(\"my-tokenizer.json\")\n```\n\n**Training time**: ~1-2 minutes for 100MB corpus, ~10-20 minutes for 1GB\n\n### Batch encoding with padding\n\n```python\n# Enable padding\ntokenizer.enable_padding(pad_id=3, pad_token=\"[PAD]\")\n\n# Encode batch\ntexts = [\"Hello world\", \"This is a longer sentence\"]\nencodings = tokenizer.encode_batch(texts)\n\nfor encoding in encodings:\n print(encoding.ids)\n# [101, 7592, 2088, 102, 3, 3, 3]\n# [101, 2023, 2003, 1037, 2936, 6251, 102]\n```\n\n## Tokenization algorithms\n\n### BPE (Byte-Pair Encoding)\n\n**How it works**:\n1. Start with character-level vocabulary\n2. Find most frequent character pair\n3. Merge into new token, add to vocabulary\n4. Repeat until vocabulary size reached\n\n**Used by**: GPT-2, GPT-3, RoBERTa, BART, DeBERTa\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import BPE\nfrom tokenizers.trainers import BpeTrainer\nfrom tokenizers.pre_tokenizers import ByteLevel\n\ntokenizer = Tokenizer(BPE(unk_token=\"<|endoftext|>\"))\ntokenizer.pre_tokenizer = ByteLevel()\n\ntrainer = BpeTrainer(\n vocab_size=50257,\n special_tokens=[\"<|endoftext|>\"],\n min_frequency=2\n)\n\ntokenizer.train(files=[\"data.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Handles OOV words well (breaks into subwords)\n- Flexible vocabulary size\n- Good for morphologically rich languages\n\n**Trade-offs**:\n- Tokenization depends on merge order\n- May split common words unexpectedly\n\n### WordPiece\n\n**How it works**:\n1. Start with character vocabulary\n2. Score merge pairs: `frequency(pair) / (frequency(first) × frequency(second))`\n3. Merge highest scoring pair\n4. Repeat until vocabulary size reached\n\n**Used by**: BERT, DistilBERT, MobileBERT\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import WordPiece\nfrom tokenizers.trainers import WordPieceTrainer\nfrom tokenizers.pre_tokenizers import Whitespace\nfrom tokenizers.normalizers import BertNormalizer\n\ntokenizer = Tokenizer(WordPiece(unk_token=\"[UNK]\"))\ntokenizer.normalizer = BertNormalizer(lowercase=True)\ntokenizer.pre_tokenizer = Whitespace()\n\ntrainer = WordPieceTrainer(\n vocab_size=30522,\n special_tokens=[\"[UNK]\", \"[CLS]\", \"[SEP]\", \"[PAD]\", \"[MASK]\"],\n continuing_subword_prefix=\"##\"\n)\n\ntokenizer.train(files=[\"corpus.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Prioritizes meaningful merges (high score = semantically related)\n- Used successfully in BERT (state-of-the-art results)\n\n**Trade-offs**:\n- Unknown words become `[UNK]` if no subword match\n- Saves vocabulary, not merge rules (larger files)\n\n### Unigram\n\n**How it works**:\n1. Start with large vocabulary (all substrings)\n2. Compute loss for corpus with current vocabulary\n3. Remove tokens with minimal impact on loss\n4. Repeat until vocabulary size reached\n\n**Used by**: ALBERT, T5, mBART, XLNet (via SentencePiece)\n\n```python\nfrom tokenizers import Tokenizer\nfrom tokenizers.models import Unigram\nfrom tokenizers.trainers import UnigramTrainer\n\ntokenizer = Tokenizer(Unigram())\n\ntrainer = UnigramTrainer(\n vocab_size=8000,\n special_tokens=[\"\", \"\", \"\"],\n unk_token=\"\"\n)\n\ntokenizer.train(files=[\"data.txt\"], trainer=trainer)\n```\n\n**Advantages**:\n- Probabilistic (finds most likely tokenization)\n- Works well for languages without word boundaries\n- Handles diverse linguistic contexts\n\n**Trade-offs**:\n- Computationally expensive to train\n- More hyperparameters to tune\n\n## Tokenization pipeline\n\nComplete pipeline: **Normalization → Pre-tokenization → Model → Post-processing**\n\n### Normalization\n\nClean and standardize text:\n\n```python\nfrom tokenizers.normalizers import NFD, StripAccents, Lowercase, Sequence\n\ntokenizer.normalizer = Sequence([\n NFD(), # Unicode normalization (decompose)\n Lowercase(), # Convert to lowercase\n StripAccents() # Remove accents\n])\n\n# Input: \"Héllo WORLD\"\n# After normalization: \"hello world\"\n```\n\n**Common normalizers**:\n- `NFD`, `NFC`, `NFKD`, `NFKC` - Unicode normalization forms\n- `Lowercase()` - Convert to lowercase\n- `StripAccents()` - Remove accents (é → e)\n- `Strip()` - Remove whitespace\n- `Replace(pattern, content)` - Regex replacement\n\n### Pre-tokenization\n\nSplit text into word-like units:\n\n```python\nfrom tokenizers.pre_tokenizers import Whitespace, Punctuation, Sequence, ByteLevel\n\n# Split on whitespace and punctuation\ntokenizer.pre_tokenizer = Sequence([\n Whitespace(),\n Punctuation()\n])\n\n# Input: \"Hello, world!\"\n# After pre-tokenization: [\"Hello\", \",\", \"world\", \"!\"]\n```\n\n**Common pre-tokenizers**:\n- `Whitespace()` - Split on spaces, tabs, newlines\n- `ByteLevel()` - GPT-2 style byte-level splitting\n- `Punctuation()` - Isolate punctuation\n- `Digits(individual_digits=True)` - Split digits individually\n- `Metaspace()` - Replace spaces with ▁ (SentencePiece style)\n\n### Post-processing\n\nAdd special tokens for model input:\n\n```python\nfrom tokenizers.processors import TemplateProcessing\n\n# BERT-style: [CLS] sentence [SEP]\ntokenizer.post_processor = TemplateProcessing(\n single=\"[CLS] $A [SEP]\",\n pair=\"[CLS] $A [SEP] $B [SEP]\",\n special_tokens=[\n (\"[CLS]\", 1),\n (\"[SEP]\", 2),\n ],\n)\n```\n\n**Common patterns**:\n```python\n# GPT-2: sentence <|endoftext|>\nTemplateProcessing(\n single=\"$A <|endoftext|>\",\n special_tokens=[(\"<|endoftext|>\", 50256)]\n)\n\n# RoBERTa: sentence \nTemplateProcessing(\n single=\" $A \",\n pair=\" $A $B \",\n special_tokens=[(\"\", 0), (\"\", 2)]\n)\n```\n\n## Alignment tracking\n\nTrack token positions in original text:\n\n```python\noutput = tokenizer.encode(\"Hello, world!\")\n\n# Get token offsets\nfor token, offset in zip(output.tokens, output.offsets):\n start, end = offset\n print(f\"{token:10} → [{start:2}, {end:2}): {text[start:end]!r}\")\n\n# Output:\n# hello → [ 0, 5): 'Hello'\n# , → [ 5, 6): ','\n# world → [ 7, 12): 'world'\n# ! → [12, 13): '!'\n```\n\n**Use cases**:\n- Named entity recognition (map predictions back to text)\n- Question answering (extract answer spans)\n- Token classification (align labels to original positions)\n\n## Integration with transformers\n\n### Load with AutoTokenizer\n\n```python\nfrom transformers import AutoTokenizer\n\n# AutoTokenizer automatically uses fast tokenizers\ntokenizer = AutoTokenizer.from_pretrained(\"bert-base-uncased\")\n\n# Check if using fast tokenizer\nprint(tokenizer.is_fast) # True\n\n# Access underlying tokenizers.Tokenizer\nfast_tokenizer = tokenizer.backend_tokenizer\nprint(type(fast_tokenizer)) # \n```\n\n### Convert custom tokenizer to transformers\n\n```python\nfrom tokenizers import Tokenizer\nfrom transformers import PreTrainedTokenizerFast\n\n# Train custom tokenizer\ntokenizer = Tokenizer(BPE())\n# ... train tokenizer ...\ntokenizer.save(\"my-tokenizer.json\")\n\n# Wrap for transformers\ntransformers_tokenizer = PreTrainedTokenizerFast(\n tokenizer_file=\"my-tokenizer.json\",\n unk_token=\"[UNK]\",\n pad_token=\"[PAD]\",\n cls_token=\"[CLS]\",\n sep_token=\"[SEP]\",\n mask_token=\"[MASK]\"\n)\n\n# Use like any transformers tokenizer\noutputs = transformers_tokenizer(\n \"Hello world\",\n padding=True,\n truncation=True,\n max_length=512,\n return_tensors=\"pt\"\n)\n```\n\n## Common patterns\n\n### Train from iterator (large datasets)\n\n```python\nfrom datasets import load_dataset\n\n# Load dataset\ndataset = load_dataset(\"wikitext\", \"wikitext-103-raw-v1\", split=\"train\")\n\n# Create batch iterator\ndef batch_iterator(batch_size=1000):\n for i in range(0, len(dataset), batch_size):\n yield dataset[i:i + batch_size][\"text\"]\n\n# Train tokenizer\ntokenizer.train_from_iterator(\n batch_iterator(),\n trainer=trainer,\n length=len(dataset) # For progress bar\n)\n```\n\n**Performance**: Processes 1GB in ~10-20 minutes\n\n### Enable truncation and padding\n\n```python\n# Enable truncation\ntokenizer.enable_truncation(max_length=512)\n\n# Enable padding\ntokenizer.enable_padding(\n pad_id=tokenizer.token_to_id(\"[PAD]\"),\n pad_token=\"[PAD]\",\n length=512 # Fixed length, or None for batch max\n)\n\n# Encode with both\noutput = tokenizer.encode(\"This is a long sentence that will be truncated...\")\nprint(len(output.ids)) # 512\n```\n\n### Multi-processing\n\n```python\nfrom tokenizers import Tokenizer\nfrom multiprocessing import Pool\n\n# Load tokenizer\ntokenizer = Tokenizer.from_file(\"tokenizer.json\")\n\ndef encode_batch(texts):\n return tokenizer.encode_batch(texts)\n\n# Process large corpus in parallel\nwith Pool(8) as pool:\n # Split corpus into chunks\n chunk_size = 1000\n chunks = [corpus[i:i+chunk_size] for i in range(0, len(corpus), chunk_size)]\n\n # Encode in parallel\n results = pool.map(encode_batch, chunks)\n```\n\n**Speedup**: 5-8× with 8 cores\n\n## Performance benchmarks\n\n### Training speed\n\n| Corpus Size | BPE (30k vocab) | WordPiece (30k) | Unigram (8k) |\n|-------------|-----------------|-----------------|--------------|\n| 10 MB | 15 sec | 18 sec | 25 sec |\n| 100 MB | 1.5 min | 2 min | 4 min |\n| 1 GB | 15 min | 20 min | 40 min |\n\n**Hardware**: 16-core CPU, tested on English Wikipedia\n\n### Tokenization speed\n\n| Implementation | 1 GB corpus | Throughput |\n|----------------|-------------|---------------|\n| Pure Python | ~20 minutes | ~50 MB/min |\n| HF Tokenizers | ~15 seconds | ~4 GB/min |\n| **Speedup** | **80×** | **80×** |\n\n**Test**: English text, average sentence length 20 words\n\n### Memory usage\n\n| Task | Memory |\n|-------------------------|---------|\n| Load tokenizer | ~10 MB |\n| Train BPE (30k vocab) | ~200 MB |\n| Encode 1M sentences | ~500 MB |\n\n## Supported models\n\nPre-trained tokenizers available via `from_pretrained()`:\n\n**BERT family**:\n- `bert-base-uncased`, `bert-large-cased`\n- `distilbert-base-uncased`\n- `roberta-base`, `roberta-large`\n\n**GPT family**:\n- `gpt2`, `gpt2-medium`, `gpt2-large`\n- `distilgpt2`\n\n**T5 family**:\n- `t5-small`, `t5-base`, `t5-large`\n- `google/flan-t5-xxl`\n\n**Other**:\n- `facebook/bart-base`, `facebook/mbart-large-cc25`\n- `albert-base-v2`, `albert-xlarge-v2`\n- `xlm-roberta-base`, `xlm-roberta-large`\n\nBrowse all: https://huggingface.co/models?library=tokenizers\n\n## References\n\n- **[Training Guide](references/training.md)** - Train custom tokenizers, configure trainers, handle large datasets\n- **[Algorithms Deep Dive](references/algorithms.md)** - BPE, WordPiece, Unigram explained in detail\n- **[Pipeline Components](references/pipeline.md)** - Normalizers, pre-tokenizers, post-processors, decoders\n- **[Transformers Integration](references/integration.md)** - AutoTokenizer, PreTrainedTokenizerFast, special tokens\n\n## Resources\n\n- **Docs**: https://huggingface.co/docs/tokenizers\n- **GitHub**: https://github.com/huggingface/tokenizers ⭐ 9,000+\n- **Version**: 0.20.0+\n- **Course**: https://huggingface.co/learn/nlp-course/chapter6/1\n- **Paper**: BPE (Sennrich et al., 2016), WordPiece (Schuster & Nakajima, 2012)\n\n\n","frontmatter":{"name":"huggingface-tokenizers","description":"Fast tokenizers optimized for research and production. Rust-based implementation tokenizes 1GB in <20 seconds. Supports BPE, WordPiece, and Unigram algorithms. Train custom vocabularies, track alignments, handle padding/truncation. Integrates seamlessly with transformers. Use when you need high-performance tokenization or custom tokenizer training.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Tokenization","HuggingFace","BPE","WordPiece","Unigram","Fast Tokenization","Rust","Custom Tokenizer","Alignment Tracking","Production"],"dependencies":["tokenizers","transformers","datasets"]},"isInternal":false,"tokens":3411,"sizeBytes":13674},{"name":"algorithms.md","path":"02-tokenization/sentencepiece/references/algorithms.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/algorithms.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `02-tokenization/sentencepiece/references/algorithms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/algorithms.md)","tokens":300,"sizeBytes":245},{"name":"training.md","path":"02-tokenization/sentencepiece/references/training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `02-tokenization/sentencepiece/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/references/training.md)","tokens":300,"sizeBytes":241},{"name":"SKILL.md","path":"02-tokenization/sentencepiece/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/SKILL.md","title":"sentencepiece","category":"anthropic-skill","format":"markdown","content":"---\nname: sentencepiece\ndescription: Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Tokenization, SentencePiece, Language-Independent, BPE, Unigram, Multilingual, CJK Languages, Unicode, Deterministic, Google]\ndependencies: [sentencepiece, transformers]\n---\n\n# SentencePiece - Language-Independent Tokenization\n\nUnsupervised tokenizer that works on raw text without language-specific preprocessing.\n\n## When to use SentencePiece\n\n**Use SentencePiece when:**\n- Building multilingual models (no language-specific rules)\n- Working with CJK languages (Chinese, Japanese, Korean)\n- Need reproducible tokenization (deterministic vocabulary)\n- Want to train on raw text (no pre-tokenization needed)\n- Require lightweight deployment (6MB memory, 50k sentences/sec)\n\n**Performance**:\n- **Speed**: 50,000 sentences/sec\n- **Memory**: ~6MB for loaded model\n- **Languages**: All (language-independent)\n\n**Use alternatives instead**:\n- **HuggingFace Tokenizers**: Faster training, more flexibility\n- **tiktoken**: OpenAI models (GPT-3.5/4)\n- **BERT WordPiece**: English-centric tasks\n\n## Quick start\n\n### Installation\n\n```bash\n# Python\npip install sentencepiece\n\n# C++ (requires CMake)\ngit clone https://github.com/google/sentencepiece.git\ncd sentencepiece\nmkdir build && cd build\ncmake .. && make -j $(nproc)\nsudo make install\n```\n\n### Train model\n\n```bash\n# Command-line (BPE with 8000 vocab)\nspm_train --input=data.txt --model_prefix=m --vocab_size=8000 --model_type=bpe\n\n# Python API\nimport sentencepiece as spm\n\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='m',\n vocab_size=8000,\n model_type='bpe'\n)\n```\n\n**Training time**: ~1-2 minutes for 100MB corpus\n\n### Encode and decode\n\n```python\nimport sentencepiece as spm\n\n# Load model\nsp = spm.SentencePieceProcessor(model_file='m.model')\n\n# Encode to pieces\npieces = sp.encode('This is a test', out_type=str)\nprint(pieces) # ['▁This', '▁is', '▁a', '▁test']\n\n# Encode to IDs\nids = sp.encode('This is a test', out_type=int)\nprint(ids) # [284, 47, 11, 1243]\n\n# Decode\ntext = sp.decode(ids)\nprint(text) # \"This is a test\"\n```\n\n## Language-independent design\n\n### Whitespace as symbol (▁)\n\n```python\ntext = \"Hello world\"\npieces = sp.encode(text, out_type=str)\nprint(pieces) # ['▁Hello', '▁world']\n\n# Decode preserves spaces\ndecoded = sp.decode_pieces(pieces)\nprint(decoded) # \"Hello world\"\n```\n\n**Key principle**: Treat text as raw Unicode, whitespace = ▁ (meta symbol)\n\n## Tokenization algorithms\n\n### BPE (Byte-Pair Encoding)\n\n```python\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='bpe_model',\n vocab_size=16000,\n model_type='bpe'\n)\n```\n\n**Used by**: mBART\n\n### Unigram (default)\n\n```python\nspm.SentencePieceTrainer.train(\n input='data.txt',\n model_prefix='unigram_model',\n vocab_size=8000,\n model_type='unigram'\n)\n```\n\n**Used by**: T5, ALBERT, XLNet\n\n## Training configuration\n\n### Essential parameters\n\n```python\nspm.SentencePieceTrainer.train(\n input='corpus.txt',\n model_prefix='m',\n vocab_size=32000,\n model_type='unigram',\n character_coverage=0.9995, # 1.0 for CJK\n user_defined_symbols=['[SEP]', '[CLS]'],\n unk_piece='',\n num_threads=16\n)\n```\n\n### Character coverage\n\n| Language Type | Coverage | Rationale |\n|---------------|----------|-----------|\n| English | 0.9995 | Most common chars |\n| CJK (Chinese) | 1.0 | All characters needed |\n| Multilingual | 0.9995 | Balance |\n\n## Encoding options\n\n### Subword regularization\n\n```python\n# Sample different tokenizations\nfor _ in range(3):\n pieces = sp.encode('tokenization', out_type=str, enable_sampling=True, alpha=0.1)\n print(pieces)\n\n# Output (different each time):\n# ['▁token', 'ization']\n# ['▁tok', 'en', 'ization']\n```\n\n**Use case**: Data augmentation for robustness.\n\n## Common patterns\n\n### T5-style training\n\n```python\nspm.SentencePieceTrainer.train(\n input='c4_corpus.txt',\n model_prefix='t5',\n vocab_size=32000,\n model_type='unigram',\n user_defined_symbols=[f'' for i in range(100)],\n unk_id=2,\n eos_id=1,\n pad_id=0\n)\n```\n\n### Integration with transformers\n\n```python\nfrom transformers import T5Tokenizer\n\n# T5 uses SentencePiece internally\ntokenizer = T5Tokenizer.from_pretrained('t5-base')\ninputs = tokenizer('translate English to French: Hello', return_tensors='pt')\n```\n\n## Performance benchmarks\n\n### Training speed\n\n| Corpus | BPE (16k) | Unigram (8k) |\n|--------|-----------|--------------|\n| 100 MB | 1-2 min | 3-4 min |\n| 1 GB | 10-15 min | 30-40 min |\n\n### Tokenization speed\n\n- **SentencePiece**: 50,000 sentences/sec\n- **HF Tokenizers**: 200,000 sentences/sec (4× faster)\n\n## Supported models\n\n**T5 family**: `t5-base`, `t5-large` (32k vocab, Unigram)\n**ALBERT**: `albert-base-v2` (30k vocab, Unigram)\n**XLNet**: `xlnet-base-cased` (32k vocab, Unigram)\n**mBART**: `facebook/mbart-large-50` (250k vocab, BPE)\n\n## References\n\n- **[Training Guide](references/training.md)** - Detailed options, corpus preparation\n- **[Algorithms](references/algorithms.md)** - BPE vs Unigram, subword regularization\n\n## Resources\n\n- **GitHub**: https://github.com/google/sentencepiece ⭐ 10,000+\n- **Paper**: https://arxiv.org/abs/1808.06226 (EMNLP 2018)\n- **Version**: 0.2.0+\n\n\n","frontmatter":{"name":"sentencepiece","description":"Language-independent tokenizer treating text as raw Unicode. Supports BPE and Unigram algorithms. Fast (50k sentences/sec), lightweight (6MB memory), deterministic vocabulary. Used by T5, ALBERT, XLNet, mBART. Train on raw text without pre-tokenization. Use when you need multilingual support, CJK languages, or reproducible tokenization.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Tokenization","SentencePiece","Language-Independent","BPE","Unigram","Multilingual","CJK Languages","Unicode","Deterministic","Google"],"dependencies":["sentencepiece","transformers"]},"isInternal":false,"tokens":1542,"sizeBytes":5632},{"name":"api.md","path":"03-fine-tuning/axolotl/references/api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/api.md)","tokens":300,"sizeBytes":217},{"name":"dataset-formats.md","path":"03-fine-tuning/axolotl/references/dataset-formats.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/dataset-formats.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/dataset-formats.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/dataset-formats.md)","tokens":300,"sizeBytes":241},{"name":"index.md","path":"03-fine-tuning/axolotl/references/index.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/index.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/index.md)","tokens":300,"sizeBytes":221},{"name":"other.md","path":"03-fine-tuning/axolotl/references/other.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/other.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/axolotl/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/references/other.md)","tokens":300,"sizeBytes":221},{"name":"SKILL.md","path":"03-fine-tuning/axolotl/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/SKILL.md","title":"axolotl","category":"anthropic-skill","format":"markdown","content":"---\nname: axolotl\ndescription: Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, Axolotl, LLM, LoRA, QLoRA, DPO, KTO, ORPO, GRPO, YAML, HuggingFace, DeepSpeed, Multimodal]\ndependencies: [axolotl, torch, transformers, datasets, peft, accelerate, deepspeed]\n---\n\n# Axolotl Skill\n\nComprehensive assistance with axolotl development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with axolotl\n- Asking about axolotl features or APIs\n- Implementing axolotl solutions\n- Debugging axolotl code\n- Learning axolotl best practices\n\n## Quick Reference\n\n### Common Patterns\n\n**Pattern 1:** To validate that acceptable data transfer speeds exist for your training job, running NCCL Tests can help pinpoint bottlenecks, for example:\n\n```\n./build/all_reduce_perf -b 8 -e 128M -f 2 -g 3\n```\n\n**Pattern 2:** Configure your model to use FSDP in the Axolotl yaml. For example:\n\n```\nfsdp_version: 2\nfsdp_config:\n offload_params: true\n state_dict_type: FULL_STATE_DICT\n auto_wrap_policy: TRANSFORMER_BASED_WRAP\n transformer_layer_cls_to_wrap: LlamaDecoderLayer\n reshard_after_forward: true\n```\n\n**Pattern 3:** The context_parallel_size should be a divisor of the total number of GPUs. For example:\n\n```\ncontext_parallel_size\n```\n\n**Pattern 4:** For example: - With 8 GPUs and no sequence parallelism: 8 different batches processed per step - With 8 GPUs and context_parallel_size=4: Only 2 different batches processed per step (each split across 4 GPUs) - If your per-GPU micro_batch_size is 2, the global batch size decreases from 16 to 4\n\n```\ncontext_parallel_size=4\n```\n\n**Pattern 5:** Setting save_compressed: true in your configuration enables saving models in a compressed format, which: - Reduces disk space usage by approximately 40% - Maintains compatibility with vLLM for accelerated inference - Maintains compatibility with llmcompressor for further optimization (example: quantization)\n\n```\nsave_compressed: true\n```\n\n**Pattern 6:** Note It is not necessary to place your integration in the integrations folder. It can be in any location, so long as it’s installed in a package in your python env. See this repo for an example: https://github.com/axolotl-ai-cloud/diff-transformer\n\n```\nintegrations\n```\n\n**Pattern 7:** Handle both single-example and batched data. - single example: sample[‘input_ids’] is a list[int] - batched data: sample[‘input_ids’] is a list[list[int]]\n\n```\nutils.trainer.drop_long_seq(sample, sequence_len=2048, min_sequence_len=2)\n```\n\n### Example Code Patterns\n\n**Example 1** (python):\n```python\ncli.cloud.modal_.ModalCloud(config, app=None)\n```\n\n**Example 2** (python):\n```python\ncli.cloud.modal_.run_cmd(cmd, run_folder, volumes=None)\n```\n\n**Example 3** (python):\n```python\ncore.trainers.base.AxolotlTrainer(\n *_args,\n bench_data_collator=None,\n eval_data_collator=None,\n dataset_tags=None,\n **kwargs,\n)\n```\n\n**Example 4** (python):\n```python\ncore.trainers.base.AxolotlTrainer.log(logs, start_time=None)\n```\n\n**Example 5** (python):\n```python\nprompt_strategies.input_output.RawInputOutputPrompter()\n```\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **api.md** - Api documentation\n- **dataset-formats.md** - Dataset-Formats documentation\n- **other.md** - Other documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n","frontmatter":{"name":"axolotl","description":"Expert guidance for fine-tuning LLMs with Axolotl - YAML configs, 100+ models, LoRA/QLoRA, DPO/KTO/ORPO/GRPO, multimodal support","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Fine-Tuning","Axolotl","LLM","LoRA","QLoRA","DPO","KTO","ORPO","GRPO","YAML","HuggingFace","DeepSpeed","Multimodal"],"dependencies":["axolotl","torch","transformers","datasets","peft","accelerate","deepspeed"]},"isInternal":false,"tokens":1159,"sizeBytes":4797},{"name":"_images.md","path":"03-fine-tuning/llama-factory/references/_images.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/_images.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/_images.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/_images.md)","tokens":300,"sizeBytes":237},{"name":"advanced.md","path":"03-fine-tuning/llama-factory/references/advanced.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/advanced.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/advanced.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/advanced.md)","tokens":300,"sizeBytes":239},{"name":"getting_started.md","path":"03-fine-tuning/llama-factory/references/getting_started.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/getting_started.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/getting_started.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/getting_started.md)","tokens":300,"sizeBytes":253},{"name":"index.md","path":"03-fine-tuning/llama-factory/references/index.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/index.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/index.md)","tokens":300,"sizeBytes":233},{"name":"other.md","path":"03-fine-tuning/llama-factory/references/other.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/other.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/llama-factory/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/references/other.md)","tokens":300,"sizeBytes":233},{"name":"SKILL.md","path":"03-fine-tuning/llama-factory/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/SKILL.md","title":"llama-factory","category":"anthropic-skill","format":"markdown","content":"---\nname: llama-factory\ndescription: Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, LLaMA Factory, LLM, WebUI, No-Code, QLoRA, LoRA, Multimodal, HuggingFace, Llama, Qwen, Gemma]\ndependencies: [llmtuner, torch, transformers, datasets, peft, accelerate, gradio]\n---\n\n# Llama-Factory Skill\n\nComprehensive assistance with llama-factory development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with llama-factory\n- Asking about llama-factory features or APIs\n- Implementing llama-factory solutions\n- Debugging llama-factory code\n- Learning llama-factory best practices\n\n## Quick Reference\n\n### Common Patterns\n\n*Quick reference patterns will be added as you use the skill.*\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **_images.md** - Images documentation\n- **advanced.md** - Advanced documentation\n- **getting_started.md** - Getting Started documentation\n- **other.md** - Other documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n","frontmatter":{"name":"llama-factory","description":"Expert guidance for fine-tuning LLMs with LLaMA-Factory - WebUI no-code, 100+ models, 2/3/4/5/6/8-bit QLoRA, multimodal support","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Fine-Tuning","LLaMA Factory","LLM","WebUI","No-Code","QLoRA","LoRA","Multimodal","HuggingFace","Llama","Qwen","Gemma"],"dependencies":["llmtuner","torch","transformers","datasets","peft","accelerate","gradio"]},"isInternal":false,"tokens":526,"sizeBytes":2472},{"name":"advanced-usage.md","path":"03-fine-tuning/peft/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/peft/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/advanced-usage.md)","tokens":300,"sizeBytes":233},{"name":"troubleshooting.md","path":"03-fine-tuning/peft/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/peft/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/references/troubleshooting.md)","tokens":300,"sizeBytes":235},{"name":"SKILL.md","path":"03-fine-tuning/peft/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/SKILL.md","title":"peft-fine-tuning","category":"anthropic-skill","format":"markdown","content":"---\nname: peft-fine-tuning\ndescription: Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, PEFT, LoRA, QLoRA, Parameter-Efficient, Adapters, Low-Rank, Memory Optimization, Multi-Adapter]\ndependencies: [peft>=0.13.0, transformers>=4.45.0, torch>=2.0.0, bitsandbytes>=0.43.0]\n---\n\n# PEFT (Parameter-Efficient Fine-Tuning)\n\nFine-tune LLMs by training <1% of parameters using LoRA, QLoRA, and 25+ adapter methods.\n\n## When to use PEFT\n\n**Use PEFT/LoRA when:**\n- Fine-tuning 7B-70B models on consumer GPUs (RTX 4090, A100)\n- Need to train <1% parameters (6MB adapters vs 14GB full model)\n- Want fast iteration with multiple task-specific adapters\n- Deploying multiple fine-tuned variants from one base model\n\n**Use QLoRA (PEFT + quantization) when:**\n- Fine-tuning 70B models on single 24GB GPU\n- Memory is the primary constraint\n- Can accept ~5% quality trade-off vs full fine-tuning\n\n**Use full fine-tuning instead when:**\n- Training small models (<1B parameters)\n- Need maximum quality and have compute budget\n- Significant domain shift requires updating all weights\n\n## Quick start\n\n### Installation\n\n```bash\n# Basic installation\npip install peft\n\n# With quantization support (recommended)\npip install peft bitsandbytes\n\n# Full stack\npip install peft transformers accelerate bitsandbytes datasets\n```\n\n### LoRA fine-tuning (standard)\n\n```python\nfrom transformers import AutoModelForCausalLM, AutoTokenizer, TrainingArguments, Trainer\nfrom peft import get_peft_model, LoraConfig, TaskType\nfrom datasets import load_dataset\n\n# Load base model\nmodel_name = \"meta-llama/Llama-3.1-8B\"\nmodel = AutoModelForCausalLM.from_pretrained(model_name, torch_dtype=\"auto\", device_map=\"auto\")\ntokenizer = AutoTokenizer.from_pretrained(model_name)\ntokenizer.pad_token = tokenizer.eos_token\n\n# LoRA configuration\nlora_config = LoraConfig(\n task_type=TaskType.CAUSAL_LM,\n r=16, # Rank (8-64, higher = more capacity)\n lora_alpha=32, # Scaling factor (typically 2*r)\n lora_dropout=0.05, # Dropout for regularization\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\"], # Attention layers\n bias=\"none\" # Don't train biases\n)\n\n# Apply LoRA\nmodel = get_peft_model(model, lora_config)\nmodel.print_trainable_parameters()\n# Output: trainable params: 13,631,488 || all params: 8,043,307,008 || trainable%: 0.17%\n\n# Prepare dataset\ndataset = load_dataset(\"databricks/databricks-dolly-15k\", split=\"train\")\n\ndef tokenize(example):\n text = f\"### Instruction:\\n{example['instruction']}\\n\\n### Response:\\n{example['response']}\"\n return tokenizer(text, truncation=True, max_length=512, padding=\"max_length\")\n\ntokenized = dataset.map(tokenize, remove_columns=dataset.column_names)\n\n# Training\ntraining_args = TrainingArguments(\n output_dir=\"./lora-llama\",\n num_train_epochs=3,\n per_device_train_batch_size=4,\n gradient_accumulation_steps=4,\n learning_rate=2e-4,\n fp16=True,\n logging_steps=10,\n save_strategy=\"epoch\"\n)\n\ntrainer = Trainer(\n model=model,\n args=training_args,\n train_dataset=tokenized,\n data_collator=lambda data: {\"input_ids\": torch.stack([f[\"input_ids\"] for f in data]),\n \"attention_mask\": torch.stack([f[\"attention_mask\"] for f in data]),\n \"labels\": torch.stack([f[\"input_ids\"] for f in data])}\n)\n\ntrainer.train()\n\n# Save adapter only (6MB vs 16GB)\nmodel.save_pretrained(\"./lora-llama-adapter\")\n```\n\n### QLoRA fine-tuning (memory-efficient)\n\n```python\nfrom transformers import AutoModelForCausalLM, BitsAndBytesConfig\nfrom peft import get_peft_model, LoraConfig, prepare_model_for_kbit_training\n\n# 4-bit quantization config\nbnb_config = BitsAndBytesConfig(\n load_in_4bit=True,\n bnb_4bit_quant_type=\"nf4\", # NormalFloat4 (best for LLMs)\n bnb_4bit_compute_dtype=\"bfloat16\", # Compute in bf16\n bnb_4bit_use_double_quant=True # Nested quantization\n)\n\n# Load quantized model\nmodel = AutoModelForCausalLM.from_pretrained(\n \"meta-llama/Llama-3.1-70B\",\n quantization_config=bnb_config,\n device_map=\"auto\"\n)\n\n# Prepare for training (enables gradient checkpointing)\nmodel = prepare_model_for_kbit_training(model)\n\n# LoRA config for QLoRA\nlora_config = LoraConfig(\n r=64, # Higher rank for 70B\n lora_alpha=128,\n lora_dropout=0.1,\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"],\n bias=\"none\",\n task_type=\"CAUSAL_LM\"\n)\n\nmodel = get_peft_model(model, lora_config)\n# 70B model now fits on single 24GB GPU!\n```\n\n## LoRA parameter selection\n\n### Rank (r) - capacity vs efficiency\n\n| Rank | Trainable Params | Memory | Quality | Use Case |\n|------|-----------------|--------|---------|----------|\n| 4 | ~3M | Minimal | Lower | Simple tasks, prototyping |\n| **8** | ~7M | Low | Good | **Recommended starting point** |\n| **16** | ~14M | Medium | Better | **General fine-tuning** |\n| 32 | ~27M | Higher | High | Complex tasks |\n| 64 | ~54M | High | Highest | Domain adaptation, 70B models |\n\n### Alpha (lora_alpha) - scaling factor\n\n```python\n# Rule of thumb: alpha = 2 * rank\nLoraConfig(r=16, lora_alpha=32) # Standard\nLoraConfig(r=16, lora_alpha=16) # Conservative (lower learning rate effect)\nLoraConfig(r=16, lora_alpha=64) # Aggressive (higher learning rate effect)\n```\n\n### Target modules by architecture\n\n```python\n# Llama / Mistral / Qwen\ntarget_modules = [\"q_proj\", \"v_proj\", \"k_proj\", \"o_proj\", \"gate_proj\", \"up_proj\", \"down_proj\"]\n\n# GPT-2 / GPT-Neo\ntarget_modules = [\"c_attn\", \"c_proj\", \"c_fc\"]\n\n# Falcon\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# BLOOM\ntarget_modules = [\"query_key_value\", \"dense\", \"dense_h_to_4h\", \"dense_4h_to_h\"]\n\n# Auto-detect all linear layers\ntarget_modules = \"all-linear\" # PEFT 0.6.0+\n```\n\n## Loading and merging adapters\n\n### Load trained adapter\n\n```python\nfrom peft import PeftModel, AutoPeftModelForCausalLM\nfrom transformers import AutoModelForCausalLM\n\n# Option 1: Load with PeftModel\nbase_model = AutoModelForCausalLM.from_pretrained(\"meta-llama/Llama-3.1-8B\")\nmodel = PeftModel.from_pretrained(base_model, \"./lora-llama-adapter\")\n\n# Option 2: Load directly (recommended)\nmodel = AutoPeftModelForCausalLM.from_pretrained(\n \"./lora-llama-adapter\",\n device_map=\"auto\"\n)\n```\n\n### Merge adapter into base model\n\n```python\n# Merge for deployment (no adapter overhead)\nmerged_model = model.merge_and_unload()\n\n# Save merged model\nmerged_model.save_pretrained(\"./llama-merged\")\ntokenizer.save_pretrained(\"./llama-merged\")\n\n# Push to Hub\nmerged_model.push_to_hub(\"username/llama-finetuned\")\n```\n\n### Multi-adapter serving\n\n```python\nfrom peft import PeftModel\n\n# Load base with first adapter\nmodel = AutoPeftModelForCausalLM.from_pretrained(\"./adapter-task1\")\n\n# Load additional adapters\nmodel.load_adapter(\"./adapter-task2\", adapter_name=\"task2\")\nmodel.load_adapter(\"./adapter-task3\", adapter_name=\"task3\")\n\n# Switch between adapters at runtime\nmodel.set_adapter(\"task1\") # Use task1 adapter\noutput1 = model.generate(**inputs)\n\nmodel.set_adapter(\"task2\") # Switch to task2\noutput2 = model.generate(**inputs)\n\n# Disable adapters (use base model)\nwith model.disable_adapter():\n base_output = model.generate(**inputs)\n```\n\n## PEFT methods comparison\n\n| Method | Trainable % | Memory | Speed | Best For |\n|--------|------------|--------|-------|----------|\n| **LoRA** | 0.1-1% | Low | Fast | General fine-tuning |\n| **QLoRA** | 0.1-1% | Very Low | Medium | Memory-constrained |\n| AdaLoRA | 0.1-1% | Low | Medium | Automatic rank selection |\n| IA3 | 0.01% | Minimal | Fastest | Few-shot adaptation |\n| Prefix Tuning | 0.1% | Low | Medium | Generation control |\n| Prompt Tuning | 0.001% | Minimal | Fast | Simple task adaptation |\n| P-Tuning v2 | 0.1% | Low | Medium | NLU tasks |\n\n### IA3 (minimal parameters)\n\n```python\nfrom peft import IA3Config\n\nia3_config = IA3Config(\n target_modules=[\"q_proj\", \"v_proj\", \"k_proj\", \"down_proj\"],\n feedforward_modules=[\"down_proj\"]\n)\nmodel = get_peft_model(model, ia3_config)\n# Trains only 0.01% of parameters!\n```\n\n### Prefix Tuning\n\n```python\nfrom peft import PrefixTuningConfig\n\nprefix_config = PrefixTuningConfig(\n task_type=\"CAUSAL_LM\",\n num_virtual_tokens=20, # Prepended tokens\n prefix_projection=True # Use MLP projection\n)\nmodel = get_peft_model(model, prefix_config)\n```\n\n## Integration patterns\n\n### With TRL (SFTTrainer)\n\n```python\nfrom trl import SFTTrainer, SFTConfig\nfrom peft import LoraConfig\n\nlora_config = LoraConfig(r=16, lora_alpha=32, target_modules=\"all-linear\")\n\ntrainer = SFTTrainer(\n model=model,\n args=SFTConfig(output_dir=\"./output\", max_seq_length=512),\n train_dataset=dataset,\n peft_config=lora_config, # Pass LoRA config directly\n)\ntrainer.train()\n```\n\n### With Axolotl (YAML config)\n\n```yaml\n# axolotl config.yaml\nadapter: lora\nlora_r: 16\nlora_alpha: 32\nlora_dropout: 0.05\nlora_target_modules:\n - q_proj\n - v_proj\n - k_proj\n - o_proj\nlora_target_linear: true # Target all linear layers\n```\n\n### With vLLM (inference)\n\n```python\nfrom vllm import LLM\nfrom vllm.lora.request import LoRARequest\n\n# Load base model with LoRA support\nllm = LLM(model=\"meta-llama/Llama-3.1-8B\", enable_lora=True)\n\n# Serve with adapter\noutputs = llm.generate(\n prompts,\n lora_request=LoRARequest(\"adapter1\", 1, \"./lora-adapter\")\n)\n```\n\n## Performance benchmarks\n\n### Memory usage (Llama 3.1 8B)\n\n| Method | GPU Memory | Trainable Params |\n|--------|-----------|------------------|\n| Full fine-tuning | 60+ GB | 8B (100%) |\n| LoRA r=16 | 18 GB | 14M (0.17%) |\n| QLoRA r=16 | 6 GB | 14M (0.17%) |\n| IA3 | 16 GB | 800K (0.01%) |\n\n### Training speed (A100 80GB)\n\n| Method | Tokens/sec | vs Full FT |\n|--------|-----------|------------|\n| Full FT | 2,500 | 1x |\n| LoRA | 3,200 | 1.3x |\n| QLoRA | 2,100 | 0.84x |\n\n### Quality (MMLU benchmark)\n\n| Model | Full FT | LoRA | QLoRA |\n|-------|---------|------|-------|\n| Llama 2-7B | 45.3 | 44.8 | 44.1 |\n| Llama 2-13B | 54.8 | 54.2 | 53.5 |\n\n## Common issues\n\n### CUDA OOM during training\n\n```python\n# Solution 1: Enable gradient checkpointing\nmodel.gradient_checkpointing_enable()\n\n# Solution 2: Reduce batch size + increase accumulation\nTrainingArguments(\n per_device_train_batch_size=1,\n gradient_accumulation_steps=16\n)\n\n# Solution 3: Use QLoRA\nfrom transformers import BitsAndBytesConfig\nbnb_config = BitsAndBytesConfig(load_in_4bit=True, bnb_4bit_quant_type=\"nf4\")\n```\n\n### Adapter not applying\n\n```python\n# Verify adapter is active\nprint(model.active_adapters) # Should show adapter name\n\n# Check trainable parameters\nmodel.print_trainable_parameters()\n\n# Ensure model in training mode\nmodel.train()\n```\n\n### Quality degradation\n\n```python\n# Increase rank\nLoraConfig(r=32, lora_alpha=64)\n\n# Target more modules\ntarget_modules = \"all-linear\"\n\n# Use more training data and epochs\nTrainingArguments(num_train_epochs=5)\n\n# Lower learning rate\nTrainingArguments(learning_rate=1e-4)\n```\n\n## Best practices\n\n1. **Start with r=8-16**, increase if quality insufficient\n2. **Use alpha = 2 * rank** as starting point\n3. **Target attention + MLP layers** for best quality/efficiency\n4. **Enable gradient checkpointing** for memory savings\n5. **Save adapters frequently** (small files, easy rollback)\n6. **Evaluate on held-out data** before merging\n7. **Use QLoRA for 70B+ models** on consumer hardware\n\n## References\n\n- **[Advanced Usage](references/advanced-usage.md)** - DoRA, LoftQ, rank stabilization, custom modules\n- **[Troubleshooting](references/troubleshooting.md)** - Common errors, debugging, optimization\n\n## Resources\n\n- **GitHub**: https://github.com/huggingface/peft\n- **Docs**: https://huggingface.co/docs/peft\n- **LoRA Paper**: arXiv:2106.09685\n- **QLoRA Paper**: arXiv:2305.14314\n- **Models**: https://huggingface.co/models?library=peft\n","frontmatter":{"name":"peft-fine-tuning","description":"Parameter-efficient fine-tuning for LLMs using LoRA, QLoRA, and 25+ methods. Use when fine-tuning large models (7B-70B) with limited GPU memory, when you need to train <1% of parameters with minimal accuracy loss, or for multi-adapter serving. HuggingFace's official library integrated with transformers ecosystem.","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Fine-Tuning","PEFT","LoRA","QLoRA","Parameter-Efficient","Adapters","Low-Rank","Memory Optimization","Multi-Adapter"],"dependencies":["peft>=0.13.0","transformers>=4.45.0","torch>=2.0.0","bitsandbytes>=0.43.0"]},"isInternal":false,"tokens":3455,"sizeBytes":12210},{"name":"index.md","path":"03-fine-tuning/unsloth/references/index.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/index.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/index.md)","tokens":300,"sizeBytes":221},{"name":"llms-full.md","path":"03-fine-tuning/unsloth/references/llms-full.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-full.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms-full.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-full.md)","tokens":300,"sizeBytes":229},{"name":"llms-txt.md","path":"03-fine-tuning/unsloth/references/llms-txt.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-txt.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms-txt.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms-txt.md)","tokens":300,"sizeBytes":227},{"name":"llms.md","path":"03-fine-tuning/unsloth/references/llms.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `03-fine-tuning/unsloth/references/llms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/references/llms.md)","tokens":300,"sizeBytes":219},{"name":"SKILL.md","path":"03-fine-tuning/unsloth/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/SKILL.md","title":"unsloth","category":"anthropic-skill","format":"markdown","content":"---\nname: unsloth\ndescription: Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Fine-Tuning, Unsloth, Fast Training, LoRA, QLoRA, Memory-Efficient, Optimization, Llama, Mistral, Gemma, Qwen]\ndependencies: [unsloth, torch, transformers, trl, datasets, peft]\n---\n\n# Unsloth Skill\n\nComprehensive assistance with unsloth development, generated from official documentation.\n\n## When to Use This Skill\n\nThis skill should be triggered when:\n- Working with unsloth\n- Asking about unsloth features or APIs\n- Implementing unsloth solutions\n- Debugging unsloth code\n- Learning unsloth best practices\n\n## Quick Reference\n\n### Common Patterns\n\n*Quick reference patterns will be added as you use the skill.*\n\n## Reference Files\n\nThis skill includes comprehensive documentation in `references/`:\n\n- **llms-txt.md** - Llms-Txt documentation\n\nUse `view` to read specific reference files when detailed information is needed.\n\n## Working with This Skill\n\n### For Beginners\nStart with the getting_started or tutorials reference files for foundational concepts.\n\n### For Specific Features\nUse the appropriate category reference file (api, guides, etc.) for detailed information.\n\n### For Code Examples\nThe quick reference section above contains common patterns extracted from the official docs.\n\n## Resources\n\n### references/\nOrganized documentation extracted from official sources. These files contain:\n- Detailed explanations\n- Code examples with language annotations\n- Links to original documentation\n- Table of contents for quick navigation\n\n### scripts/\nAdd helper scripts here for common automation tasks.\n\n### assets/\nAdd templates, boilerplate, or example projects here.\n\n## Notes\n\n- This skill was automatically generated from official documentation\n- Reference files preserve the structure and examples from source docs\n- Code examples include language detection for better syntax highlighting\n- Quick reference patterns are extracted from common usage examples in the docs\n\n## Updating\n\nTo refresh this skill with updated documentation:\n1. Re-run the scraper with the same configuration\n2. The skill will be rebuilt with the latest information\n\n\n\n\n\n","frontmatter":{"name":"unsloth","description":"Expert guidance for fast fine-tuning with Unsloth - 2-5x faster training, 50-80% less memory, LoRA/QLoRA optimization","version":"1.0.0","author":"Orchestra Research","license":"MIT","tags":["Fine-Tuning","Unsloth","Fast Training","LoRA","QLoRA","Memory-Efficient","Optimization","Llama","Mistral","Gemma","Qwen"],"dependencies":["unsloth","torch","transformers","trl","datasets","peft"]},"isInternal":false,"tokens":481,"sizeBytes":2307},{"name":"api.md","path":"04-mechanistic-interpretability/nnsight/references/api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/api.md)","tokens":300,"sizeBytes":251},{"name":"README.md","path":"04-mechanistic-interpretability/nnsight/references/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/README.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/README.md)","tokens":300,"sizeBytes":257},{"name":"tutorials.md","path":"04-mechanistic-interpretability/nnsight/references/tutorials.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/tutorials.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/nnsight/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/references/tutorials.md)","tokens":300,"sizeBytes":263},{"name":"SKILL.md","path":"04-mechanistic-interpretability/nnsight/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/SKILL.md","title":"nnsight","category":"anthropic-skill","format":"markdown","content":"# nnsight\n\nPath: `04-mechanistic-interpretability/nnsight/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/SKILL.md)","tokens":300,"sizeBytes":224},{"name":"api.md","path":"04-mechanistic-interpretability/pyvene/references/api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/api.md)","tokens":300,"sizeBytes":249},{"name":"README.md","path":"04-mechanistic-interpretability/pyvene/references/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/README.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/README.md)","tokens":300,"sizeBytes":255},{"name":"tutorials.md","path":"04-mechanistic-interpretability/pyvene/references/tutorials.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/tutorials.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/pyvene/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/references/tutorials.md)","tokens":300,"sizeBytes":261},{"name":"SKILL.md","path":"04-mechanistic-interpretability/pyvene/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/SKILL.md","title":"pyvene","category":"anthropic-skill","format":"markdown","content":"# pyvene\n\nPath: `04-mechanistic-interpretability/pyvene/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/SKILL.md)","tokens":300,"sizeBytes":221},{"name":"api.md","path":"04-mechanistic-interpretability/saelens/references/api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/api.md)","tokens":300,"sizeBytes":251},{"name":"README.md","path":"04-mechanistic-interpretability/saelens/references/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/README.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/README.md)","tokens":300,"sizeBytes":257},{"name":"tutorials.md","path":"04-mechanistic-interpretability/saelens/references/tutorials.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/tutorials.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/saelens/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/references/tutorials.md)","tokens":300,"sizeBytes":263},{"name":"SKILL.md","path":"04-mechanistic-interpretability/saelens/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/SKILL.md","title":"saelens","category":"anthropic-skill","format":"markdown","content":"# saelens\n\nPath: `04-mechanistic-interpretability/saelens/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/SKILL.md)","tokens":300,"sizeBytes":224},{"name":"api.md","path":"04-mechanistic-interpretability/transformer-lens/references/api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/api.md)","tokens":300,"sizeBytes":269},{"name":"README.md","path":"04-mechanistic-interpretability/transformer-lens/references/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/README.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/README.md)","tokens":300,"sizeBytes":275},{"name":"tutorials.md","path":"04-mechanistic-interpretability/transformer-lens/references/tutorials.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/tutorials.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `04-mechanistic-interpretability/transformer-lens/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/references/tutorials.md)","tokens":300,"sizeBytes":281},{"name":"SKILL.md","path":"04-mechanistic-interpretability/transformer-lens/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/SKILL.md","title":"transformer-lens","category":"anthropic-skill","format":"markdown","content":"# transformer-lens\n\nPath: `04-mechanistic-interpretability/transformer-lens/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/SKILL.md)","tokens":300,"sizeBytes":251},{"name":"deduplication.md","path":"05-data-processing/nemo-curator/references/deduplication.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/deduplication.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `05-data-processing/nemo-curator/references/deduplication.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/deduplication.md)","tokens":300,"sizeBytes":255},{"name":"filtering.md","path":"05-data-processing/nemo-curator/references/filtering.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/filtering.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `05-data-processing/nemo-curator/references/filtering.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/references/filtering.md)","tokens":300,"sizeBytes":247},{"name":"SKILL.md","path":"05-data-processing/nemo-curator/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/SKILL.md","title":"nemo-curator","category":"anthropic-skill","format":"markdown","content":"# nemo-curator\n\nPath: `05-data-processing/nemo-curator/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/SKILL.md)","tokens":300,"sizeBytes":213},{"name":"integration.md","path":"05-data-processing/ray-data/references/integration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/integration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `05-data-processing/ray-data/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/integration.md)","tokens":300,"sizeBytes":243},{"name":"transformations.md","path":"05-data-processing/ray-data/references/transformations.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/transformations.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `05-data-processing/ray-data/references/transformations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/references/transformations.md)","tokens":300,"sizeBytes":251},{"name":"SKILL.md","path":"05-data-processing/ray-data/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/SKILL.md","title":"ray-data","category":"anthropic-skill","format":"markdown","content":"# ray-data\n\nPath: `05-data-processing/ray-data/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/SKILL.md)","tokens":300,"sizeBytes":201},{"name":"SKILL.md","path":"06-post-training/grpo-rl-training/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/SKILL.md","title":"grpo-rl-training","category":"anthropic-skill","format":"markdown","content":"# grpo-rl-training\n\nPath: `06-post-training/grpo-rl-training/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/SKILL.md)","tokens":300,"sizeBytes":221},{"name":"api-reference.md","path":"06-post-training/miles/references/api-reference.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/api-reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/miles/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/api-reference.md)","tokens":300,"sizeBytes":237},{"name":"troubleshooting.md","path":"06-post-training/miles/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/miles/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/references/troubleshooting.md)","tokens":300,"sizeBytes":241},{"name":"SKILL.md","path":"06-post-training/miles/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/SKILL.md","title":"miles","category":"anthropic-skill","format":"markdown","content":"# miles\n\nPath: `06-post-training/miles/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/SKILL.md)","tokens":300,"sizeBytes":188},{"name":"algorithm-comparison.md","path":"06-post-training/openrlhf/references/algorithm-comparison.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/algorithm-comparison.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/openrlhf/references/algorithm-comparison.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/algorithm-comparison.md)","tokens":300,"sizeBytes":257},{"name":"custom-rewards.md","path":"06-post-training/openrlhf/references/custom-rewards.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/custom-rewards.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/openrlhf/references/custom-rewards.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/custom-rewards.md)","tokens":300,"sizeBytes":245},{"name":"hybrid-engine.md","path":"06-post-training/openrlhf/references/hybrid-engine.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/hybrid-engine.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/openrlhf/references/hybrid-engine.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/hybrid-engine.md)","tokens":300,"sizeBytes":243},{"name":"multi-node-training.md","path":"06-post-training/openrlhf/references/multi-node-training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/multi-node-training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/openrlhf/references/multi-node-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/references/multi-node-training.md)","tokens":300,"sizeBytes":255},{"name":"SKILL.md","path":"06-post-training/openrlhf/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/SKILL.md","title":"openrlhf","category":"anthropic-skill","format":"markdown","content":"# openrlhf\n\nPath: `06-post-training/openrlhf/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/SKILL.md)","tokens":300,"sizeBytes":197},{"name":"datasets.md","path":"06-post-training/simpo/references/datasets.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/datasets.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/simpo/references/datasets.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/datasets.md)","tokens":300,"sizeBytes":227},{"name":"hyperparameters.md","path":"06-post-training/simpo/references/hyperparameters.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/hyperparameters.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/simpo/references/hyperparameters.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/hyperparameters.md)","tokens":300,"sizeBytes":241},{"name":"loss-functions.md","path":"06-post-training/simpo/references/loss-functions.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/loss-functions.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/simpo/references/loss-functions.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/references/loss-functions.md)","tokens":300,"sizeBytes":239},{"name":"SKILL.md","path":"06-post-training/simpo/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/SKILL.md","title":"simpo","category":"anthropic-skill","format":"markdown","content":"# simpo\n\nPath: `06-post-training/simpo/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/SKILL.md)","tokens":300,"sizeBytes":188},{"name":"api-reference.md","path":"06-post-training/slime/references/api-reference.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/api-reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/slime/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/api-reference.md)","tokens":300,"sizeBytes":237},{"name":"troubleshooting.md","path":"06-post-training/slime/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/slime/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/references/troubleshooting.md)","tokens":300,"sizeBytes":241},{"name":"SKILL.md","path":"06-post-training/slime/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/SKILL.md","title":"slime","category":"anthropic-skill","format":"markdown","content":"# slime\n\nPath: `06-post-training/slime/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/SKILL.md)","tokens":300,"sizeBytes":188},{"name":"api-reference.md","path":"06-post-training/torchforge/references/api-reference.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/api-reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/torchforge/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/api-reference.md)","tokens":300,"sizeBytes":247},{"name":"troubleshooting.md","path":"06-post-training/torchforge/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/torchforge/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/references/troubleshooting.md)","tokens":300,"sizeBytes":251},{"name":"SKILL.md","path":"06-post-training/torchforge/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/SKILL.md","title":"torchforge","category":"anthropic-skill","format":"markdown","content":"# torchforge\n\nPath: `06-post-training/torchforge/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/SKILL.md)","tokens":300,"sizeBytes":203},{"name":"dpo-variants.md","path":"06-post-training/trl-fine-tuning/references/dpo-variants.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/dpo-variants.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/dpo-variants.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/dpo-variants.md)","tokens":300,"sizeBytes":255},{"name":"online-rl.md","path":"06-post-training/trl-fine-tuning/references/online-rl.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/online-rl.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/online-rl.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/online-rl.md)","tokens":300,"sizeBytes":249},{"name":"reward-modeling.md","path":"06-post-training/trl-fine-tuning/references/reward-modeling.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/reward-modeling.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/reward-modeling.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/reward-modeling.md)","tokens":300,"sizeBytes":261},{"name":"sft-training.md","path":"06-post-training/trl-fine-tuning/references/sft-training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/sft-training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/trl-fine-tuning/references/sft-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/references/sft-training.md)","tokens":300,"sizeBytes":255},{"name":"SKILL.md","path":"06-post-training/trl-fine-tuning/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/SKILL.md","title":"trl-fine-tuning","category":"anthropic-skill","format":"markdown","content":"# trl-fine-tuning\n\nPath: `06-post-training/trl-fine-tuning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/SKILL.md)","tokens":300,"sizeBytes":218},{"name":"api-reference.md","path":"06-post-training/verl/references/api-reference.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/api-reference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/verl/references/api-reference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/api-reference.md)","tokens":300,"sizeBytes":235},{"name":"troubleshooting.md","path":"06-post-training/verl/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `06-post-training/verl/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/references/troubleshooting.md)","tokens":300,"sizeBytes":239},{"name":"SKILL.md","path":"06-post-training/verl/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/SKILL.md","title":"verl","category":"anthropic-skill","format":"markdown","content":"# verl\n\nPath: `06-post-training/verl/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/SKILL.md)","tokens":300,"sizeBytes":185},{"name":"SKILL.md","path":"07-safety-alignment/constitutional-ai/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/SKILL.md","title":"constitutional-ai","category":"anthropic-skill","format":"markdown","content":"# constitutional-ai\n\nPath: `07-safety-alignment/constitutional-ai/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/SKILL.md)","tokens":300,"sizeBytes":230},{"name":"SKILL.md","path":"07-safety-alignment/llamaguard/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/SKILL.md","title":"llamaguard","category":"anthropic-skill","format":"markdown","content":"# llamaguard\n\nPath: `07-safety-alignment/llamaguard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/SKILL.md)","tokens":300,"sizeBytes":209},{"name":"SKILL.md","path":"07-safety-alignment/nemo-guardrails/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/SKILL.md","title":"nemo-guardrails","category":"anthropic-skill","format":"markdown","content":"# nemo-guardrails\n\nPath: `07-safety-alignment/nemo-guardrails/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/SKILL.md)","tokens":300,"sizeBytes":224},{"name":"SKILL.md","path":"07-safety-alignment/prompt-guard/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/SKILL.md","title":"prompt-guard","category":"anthropic-skill","format":"markdown","content":"# prompt-guard\n\nPath: `07-safety-alignment/prompt-guard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/SKILL.md)","tokens":300,"sizeBytes":215},{"name":"custom-plugins.md","path":"08-distributed-training/accelerate/references/custom-plugins.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/custom-plugins.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/custom-plugins.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/custom-plugins.md)","tokens":300,"sizeBytes":263},{"name":"megatron-integration.md","path":"08-distributed-training/accelerate/references/megatron-integration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/megatron-integration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/megatron-integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/megatron-integration.md)","tokens":300,"sizeBytes":275},{"name":"performance.md","path":"08-distributed-training/accelerate/references/performance.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/performance.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/accelerate/references/performance.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/references/performance.md)","tokens":300,"sizeBytes":257},{"name":"SKILL.md","path":"08-distributed-training/accelerate/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/SKILL.md","title":"accelerate","category":"anthropic-skill","format":"markdown","content":"# accelerate\n\nPath: `08-distributed-training/accelerate/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/SKILL.md)","tokens":300,"sizeBytes":217},{"name":"08.md","path":"08-distributed-training/deepspeed/references/08.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/08.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/08.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/08.md)","tokens":300,"sizeBytes":237},{"name":"09.md","path":"08-distributed-training/deepspeed/references/09.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/09.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/09.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/09.md)","tokens":300,"sizeBytes":237},{"name":"2020.md","path":"08-distributed-training/deepspeed/references/2020.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2020.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/2020.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2020.md)","tokens":300,"sizeBytes":241},{"name":"2023.md","path":"08-distributed-training/deepspeed/references/2023.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2023.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/2023.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/2023.md)","tokens":300,"sizeBytes":241},{"name":"assets.md","path":"08-distributed-training/deepspeed/references/assets.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/assets.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/assets.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/assets.md)","tokens":300,"sizeBytes":245},{"name":"index.md","path":"08-distributed-training/deepspeed/references/index.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/index.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/index.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/index.md)","tokens":300,"sizeBytes":243},{"name":"mii.md","path":"08-distributed-training/deepspeed/references/mii.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/mii.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/mii.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/mii.md)","tokens":300,"sizeBytes":239},{"name":"other.md","path":"08-distributed-training/deepspeed/references/other.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/other.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/other.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/other.md)","tokens":300,"sizeBytes":243},{"name":"tutorials.md","path":"08-distributed-training/deepspeed/references/tutorials.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/tutorials.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/deepspeed/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/references/tutorials.md)","tokens":300,"sizeBytes":251},{"name":"SKILL.md","path":"08-distributed-training/deepspeed/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/SKILL.md","title":"deepspeed","category":"anthropic-skill","format":"markdown","content":"# deepspeed\n\nPath: `08-distributed-training/deepspeed/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/SKILL.md)","tokens":300,"sizeBytes":214},{"name":"benchmarks.md","path":"08-distributed-training/megatron-core/references/benchmarks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/benchmarks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/benchmarks.md)","tokens":300,"sizeBytes":261},{"name":"parallelism-guide.md","path":"08-distributed-training/megatron-core/references/parallelism-guide.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/parallelism-guide.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/parallelism-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/parallelism-guide.md)","tokens":300,"sizeBytes":275},{"name":"production-examples.md","path":"08-distributed-training/megatron-core/references/production-examples.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/production-examples.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/production-examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/production-examples.md)","tokens":300,"sizeBytes":279},{"name":"training-recipes.md","path":"08-distributed-training/megatron-core/references/training-recipes.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/training-recipes.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/megatron-core/references/training-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/references/training-recipes.md)","tokens":300,"sizeBytes":273},{"name":"SKILL.md","path":"08-distributed-training/megatron-core/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/SKILL.md","title":"megatron-core","category":"anthropic-skill","format":"markdown","content":"# megatron-core\n\nPath: `08-distributed-training/megatron-core/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/SKILL.md)","tokens":300,"sizeBytes":226},{"name":"pytorch_dcp_async_recipe.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_async_recipe.md)","tokens":300,"sizeBytes":289},{"name":"pytorch_dcp_overview.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_overview.md)","tokens":300,"sizeBytes":281},{"name":"pytorch_dcp_recipe.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_dcp_recipe.md)","tokens":300,"sizeBytes":277},{"name":"pytorch_ddp_notes.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_ddp_notes.md)","tokens":300,"sizeBytes":275},{"name":"pytorch_device_mesh_tutorial.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_device_mesh_tutorial.md)","tokens":300,"sizeBytes":297},{"name":"pytorch_examples_fsdp2.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_examples_fsdp2.md)","tokens":300,"sizeBytes":285},{"name":"pytorch_fsdp1_api.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp1_api.md)","tokens":300,"sizeBytes":275},{"name":"pytorch_fsdp2_tutorial.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fsdp2_tutorial.md)","tokens":300,"sizeBytes":285},{"name":"pytorch_fully_shard_api.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_fully_shard_api.md)","tokens":300,"sizeBytes":287},{"name":"pytorch_tp_tutorial.md","path":"08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/pytorch_tp_tutorial.md)","tokens":300,"sizeBytes":279},{"name":"ray_train_fsdp2_example.md","path":"08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/ray_train_fsdp2_example.md)","tokens":300,"sizeBytes":287},{"name":"torchtitan_fsdp_notes.md","path":"08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/references/torchtitan_fsdp_notes.md)","tokens":300,"sizeBytes":283},{"name":"SKILL.md","path":"08-distributed-training/pytorch-fsdp2/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/SKILL.md","title":"pytorch-fsdp2","category":"anthropic-skill","format":"markdown","content":"# pytorch-fsdp2\n\nPath: `08-distributed-training/pytorch-fsdp2/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/SKILL.md)","tokens":300,"sizeBytes":226},{"name":"callbacks.md","path":"08-distributed-training/pytorch-lightning/references/callbacks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/callbacks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/callbacks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/callbacks.md)","tokens":300,"sizeBytes":267},{"name":"distributed.md","path":"08-distributed-training/pytorch-lightning/references/distributed.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/distributed.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/distributed.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/distributed.md)","tokens":300,"sizeBytes":271},{"name":"hyperparameter-tuning.md","path":"08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/references/hyperparameter-tuning.md)","tokens":300,"sizeBytes":291},{"name":"SKILL.md","path":"08-distributed-training/pytorch-lightning/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/SKILL.md","title":"pytorch-lightning","category":"anthropic-skill","format":"markdown","content":"# pytorch-lightning\n\nPath: `08-distributed-training/pytorch-lightning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/SKILL.md)","tokens":300,"sizeBytes":238},{"name":"multi-node.md","path":"08-distributed-training/ray-train/references/multi-node.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/references/multi-node.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `08-distributed-training/ray-train/references/multi-node.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/references/multi-node.md)","tokens":300,"sizeBytes":253},{"name":"SKILL.md","path":"08-distributed-training/ray-train/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/SKILL.md","title":"ray-train","category":"anthropic-skill","format":"markdown","content":"# ray-train\n\nPath: `08-distributed-training/ray-train/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/SKILL.md)","tokens":300,"sizeBytes":214},{"name":"advanced-usage.md","path":"09-infrastructure/lambda-labs/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `09-infrastructure/lambda-labs/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/advanced-usage.md)","tokens":300,"sizeBytes":253},{"name":"troubleshooting.md","path":"09-infrastructure/lambda-labs/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `09-infrastructure/lambda-labs/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/references/troubleshooting.md)","tokens":300,"sizeBytes":255},{"name":"SKILL.md","path":"09-infrastructure/lambda-labs/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/SKILL.md","title":"lambda-labs","category":"anthropic-skill","format":"markdown","content":"# lambda-labs\n\nPath: `09-infrastructure/lambda-labs/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/SKILL.md)","tokens":300,"sizeBytes":208},{"name":"advanced-usage.md","path":"09-infrastructure/modal/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `09-infrastructure/modal/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/advanced-usage.md)","tokens":300,"sizeBytes":241},{"name":"troubleshooting.md","path":"09-infrastructure/modal/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `09-infrastructure/modal/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/references/troubleshooting.md)","tokens":300,"sizeBytes":243},{"name":"SKILL.md","path":"09-infrastructure/modal/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/SKILL.md","title":"modal","category":"anthropic-skill","format":"markdown","content":"# modal\n\nPath: `09-infrastructure/modal/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/SKILL.md)","tokens":300,"sizeBytes":190},{"name":"advanced-usage.md","path":"09-infrastructure/skypilot/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `09-infrastructure/skypilot/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/advanced-usage.md)","tokens":300,"sizeBytes":247},{"name":"troubleshooting.md","path":"09-infrastructure/skypilot/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `09-infrastructure/skypilot/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/references/troubleshooting.md)","tokens":300,"sizeBytes":249},{"name":"SKILL.md","path":"09-infrastructure/skypilot/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/SKILL.md","title":"skypilot","category":"anthropic-skill","format":"markdown","content":"# skypilot\n\nPath: `09-infrastructure/skypilot/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/SKILL.md)","tokens":300,"sizeBytes":199},{"name":"advanced-usage.md","path":"10-optimization/awq/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/awq/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/advanced-usage.md)","tokens":300,"sizeBytes":233},{"name":"troubleshooting.md","path":"10-optimization/awq/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/awq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/references/troubleshooting.md)","tokens":300,"sizeBytes":235},{"name":"SKILL.md","path":"10-optimization/awq/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/SKILL.md","title":"awq","category":"anthropic-skill","format":"markdown","content":"# awq\n\nPath: `10-optimization/awq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/SKILL.md)","tokens":300,"sizeBytes":180},{"name":"memory-optimization.md","path":"10-optimization/bitsandbytes/references/memory-optimization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/memory-optimization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/memory-optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/memory-optimization.md)","tokens":300,"sizeBytes":261},{"name":"qlora-training.md","path":"10-optimization/bitsandbytes/references/qlora-training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/qlora-training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/qlora-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/qlora-training.md)","tokens":300,"sizeBytes":251},{"name":"quantization-formats.md","path":"10-optimization/bitsandbytes/references/quantization-formats.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/quantization-formats.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/bitsandbytes/references/quantization-formats.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/references/quantization-formats.md)","tokens":300,"sizeBytes":263},{"name":"SKILL.md","path":"10-optimization/bitsandbytes/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/SKILL.md","title":"bitsandbytes","category":"anthropic-skill","format":"markdown","content":"# bitsandbytes\n\nPath: `10-optimization/bitsandbytes/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/SKILL.md)","tokens":300,"sizeBytes":207},{"name":"benchmarks.md","path":"10-optimization/flash-attention/references/benchmarks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/benchmarks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/flash-attention/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/benchmarks.md)","tokens":300,"sizeBytes":249},{"name":"transformers-integration.md","path":"10-optimization/flash-attention/references/transformers-integration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/transformers-integration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/flash-attention/references/transformers-integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/references/transformers-integration.md)","tokens":300,"sizeBytes":277},{"name":"SKILL.md","path":"10-optimization/flash-attention/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/SKILL.md","title":"flash-attention","category":"anthropic-skill","format":"markdown","content":"# flash-attention\n\nPath: `10-optimization/flash-attention/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/SKILL.md)","tokens":300,"sizeBytes":216},{"name":"advanced-usage.md","path":"10-optimization/gguf/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/gguf/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/advanced-usage.md)","tokens":300,"sizeBytes":235},{"name":"troubleshooting.md","path":"10-optimization/gguf/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/gguf/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/references/troubleshooting.md)","tokens":300,"sizeBytes":237},{"name":"SKILL.md","path":"10-optimization/gguf/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/SKILL.md","title":"gguf","category":"anthropic-skill","format":"markdown","content":"# gguf\n\nPath: `10-optimization/gguf/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/SKILL.md)","tokens":300,"sizeBytes":183},{"name":"calibration.md","path":"10-optimization/gptq/references/calibration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/calibration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/gptq/references/calibration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/calibration.md)","tokens":300,"sizeBytes":229},{"name":"integration.md","path":"10-optimization/gptq/references/integration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/integration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/gptq/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/integration.md)","tokens":300,"sizeBytes":229},{"name":"troubleshooting.md","path":"10-optimization/gptq/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/gptq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/references/troubleshooting.md)","tokens":300,"sizeBytes":237},{"name":"SKILL.md","path":"10-optimization/gptq/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/SKILL.md","title":"gptq","category":"anthropic-skill","format":"markdown","content":"# gptq\n\nPath: `10-optimization/gptq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/SKILL.md)","tokens":300,"sizeBytes":183},{"name":"advanced-usage.md","path":"10-optimization/hqq/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/hqq/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/advanced-usage.md)","tokens":300,"sizeBytes":233},{"name":"troubleshooting.md","path":"10-optimization/hqq/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/hqq/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/references/troubleshooting.md)","tokens":300,"sizeBytes":235},{"name":"SKILL.md","path":"10-optimization/hqq/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/SKILL.md","title":"hqq","category":"anthropic-skill","format":"markdown","content":"# hqq\n\nPath: `10-optimization/hqq/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/SKILL.md)","tokens":300,"sizeBytes":180},{"name":"architecture.md","path":"10-optimization/ml-training-recipes/references/architecture.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/architecture.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/architecture.md)","tokens":300,"sizeBytes":261},{"name":"biomedical.md","path":"10-optimization/ml-training-recipes/references/biomedical.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/biomedical.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/biomedical.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/biomedical.md)","tokens":300,"sizeBytes":257},{"name":"domain-specific.md","path":"10-optimization/ml-training-recipes/references/domain-specific.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/domain-specific.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/domain-specific.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/domain-specific.md)","tokens":300,"sizeBytes":267},{"name":"experiment-loop.md","path":"10-optimization/ml-training-recipes/references/experiment-loop.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/experiment-loop.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/experiment-loop.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/experiment-loop.md)","tokens":300,"sizeBytes":267},{"name":"optimizers.md","path":"10-optimization/ml-training-recipes/references/optimizers.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/optimizers.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/optimizers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/optimizers.md)","tokens":300,"sizeBytes":257},{"name":"scaling-and-selection.md","path":"10-optimization/ml-training-recipes/references/scaling-and-selection.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/scaling-and-selection.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `10-optimization/ml-training-recipes/references/scaling-and-selection.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/references/scaling-and-selection.md)","tokens":300,"sizeBytes":279},{"name":"SKILL.md","path":"10-optimization/ml-training-recipes/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/SKILL.md","title":"ml-training-recipes","category":"anthropic-skill","format":"markdown","content":"# ml-training-recipes\n\nPath: `10-optimization/ml-training-recipes/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/SKILL.md)","tokens":300,"sizeBytes":228},{"name":"benchmarks.md","path":"11-evaluation/bigcode-evaluation-harness/references/benchmarks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/benchmarks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/benchmarks.md)","tokens":300,"sizeBytes":267},{"name":"custom-tasks.md","path":"11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/custom-tasks.md)","tokens":300,"sizeBytes":271},{"name":"issues.md","path":"11-evaluation/bigcode-evaluation-harness/references/issues.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/issues.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/bigcode-evaluation-harness/references/issues.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/references/issues.md)","tokens":300,"sizeBytes":259},{"name":"SKILL.md","path":"11-evaluation/bigcode-evaluation-harness/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/SKILL.md","title":"bigcode-evaluation-harness","category":"anthropic-skill","format":"markdown","content":"# bigcode-evaluation-harness\n\nPath: `11-evaluation/bigcode-evaluation-harness/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/SKILL.md)","tokens":300,"sizeBytes":245},{"name":"api-evaluation.md","path":"11-evaluation/lm-evaluation-harness/references/api-evaluation.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/api-evaluation.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/api-evaluation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/api-evaluation.md)","tokens":300,"sizeBytes":265},{"name":"benchmark-guide.md","path":"11-evaluation/lm-evaluation-harness/references/benchmark-guide.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/benchmark-guide.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/benchmark-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/benchmark-guide.md)","tokens":300,"sizeBytes":267},{"name":"custom-tasks.md","path":"11-evaluation/lm-evaluation-harness/references/custom-tasks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/custom-tasks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/custom-tasks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/custom-tasks.md)","tokens":300,"sizeBytes":261},{"name":"distributed-eval.md","path":"11-evaluation/lm-evaluation-harness/references/distributed-eval.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/distributed-eval.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/lm-evaluation-harness/references/distributed-eval.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/references/distributed-eval.md)","tokens":300,"sizeBytes":269},{"name":"SKILL.md","path":"11-evaluation/lm-evaluation-harness/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/SKILL.md","title":"lm-evaluation-harness","category":"anthropic-skill","format":"markdown","content":"# lm-evaluation-harness\n\nPath: `11-evaluation/lm-evaluation-harness/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/SKILL.md)","tokens":300,"sizeBytes":230},{"name":"adapter-system.md","path":"11-evaluation/nemo-evaluator/references/adapter-system.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/adapter-system.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/adapter-system.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/adapter-system.md)","tokens":300,"sizeBytes":251},{"name":"configuration.md","path":"11-evaluation/nemo-evaluator/references/configuration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/configuration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/configuration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/configuration.md)","tokens":300,"sizeBytes":249},{"name":"custom-benchmarks.md","path":"11-evaluation/nemo-evaluator/references/custom-benchmarks.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/custom-benchmarks.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/custom-benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/custom-benchmarks.md)","tokens":300,"sizeBytes":257},{"name":"execution-backends.md","path":"11-evaluation/nemo-evaluator/references/execution-backends.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/execution-backends.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `11-evaluation/nemo-evaluator/references/execution-backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/references/execution-backends.md)","tokens":300,"sizeBytes":259},{"name":"SKILL.md","path":"11-evaluation/nemo-evaluator/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/SKILL.md","title":"nemo-evaluator","category":"anthropic-skill","format":"markdown","content":"# nemo-evaluator\n\nPath: `11-evaluation/nemo-evaluator/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/SKILL.md)","tokens":300,"sizeBytes":209},{"name":"optimization.md","path":"12-inference-serving/llama-cpp/references/optimization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/optimization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/optimization.md)","tokens":300,"sizeBytes":251},{"name":"quantization.md","path":"12-inference-serving/llama-cpp/references/quantization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/quantization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/quantization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/quantization.md)","tokens":300,"sizeBytes":251},{"name":"server.md","path":"12-inference-serving/llama-cpp/references/server.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/server.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/llama-cpp/references/server.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/references/server.md)","tokens":300,"sizeBytes":239},{"name":"SKILL.md","path":"12-inference-serving/llama-cpp/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/SKILL.md","title":"llama-cpp","category":"anthropic-skill","format":"markdown","content":"# llama-cpp\n\nPath: `12-inference-serving/llama-cpp/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/SKILL.md)","tokens":300,"sizeBytes":208},{"name":"deployment.md","path":"12-inference-serving/sglang/references/deployment.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/deployment.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/sglang/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/deployment.md)","tokens":300,"sizeBytes":241},{"name":"radix-attention.md","path":"12-inference-serving/sglang/references/radix-attention.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/radix-attention.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/sglang/references/radix-attention.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/radix-attention.md)","tokens":300,"sizeBytes":251},{"name":"structured-generation.md","path":"12-inference-serving/sglang/references/structured-generation.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/structured-generation.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/sglang/references/structured-generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/references/structured-generation.md)","tokens":300,"sizeBytes":263},{"name":"SKILL.md","path":"12-inference-serving/sglang/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/SKILL.md","title":"sglang","category":"anthropic-skill","format":"markdown","content":"# sglang\n\nPath: `12-inference-serving/sglang/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/SKILL.md)","tokens":300,"sizeBytes":199},{"name":"multi-gpu.md","path":"12-inference-serving/tensorrt-llm/references/multi-gpu.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/multi-gpu.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/multi-gpu.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/multi-gpu.md)","tokens":300,"sizeBytes":251},{"name":"optimization.md","path":"12-inference-serving/tensorrt-llm/references/optimization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/optimization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/optimization.md)","tokens":300,"sizeBytes":257},{"name":"serving.md","path":"12-inference-serving/tensorrt-llm/references/serving.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/serving.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/tensorrt-llm/references/serving.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/references/serving.md)","tokens":300,"sizeBytes":247},{"name":"SKILL.md","path":"12-inference-serving/tensorrt-llm/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/SKILL.md","title":"tensorrt-llm","category":"anthropic-skill","format":"markdown","content":"# tensorrt-llm\n\nPath: `12-inference-serving/tensorrt-llm/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/SKILL.md)","tokens":300,"sizeBytes":217},{"name":"optimization.md","path":"12-inference-serving/vllm/references/optimization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/optimization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/vllm/references/optimization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/optimization.md)","tokens":300,"sizeBytes":241},{"name":"quantization.md","path":"12-inference-serving/vllm/references/quantization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/quantization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/vllm/references/quantization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/quantization.md)","tokens":300,"sizeBytes":241},{"name":"server-deployment.md","path":"12-inference-serving/vllm/references/server-deployment.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/server-deployment.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/vllm/references/server-deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/server-deployment.md)","tokens":300,"sizeBytes":251},{"name":"troubleshooting.md","path":"12-inference-serving/vllm/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `12-inference-serving/vllm/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/references/troubleshooting.md)","tokens":300,"sizeBytes":247},{"name":"SKILL.md","path":"12-inference-serving/vllm/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/SKILL.md","title":"vllm","category":"anthropic-skill","format":"markdown","content":"# vllm\n\nPath: `12-inference-serving/vllm/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/SKILL.md)","tokens":300,"sizeBytes":193},{"name":"deployment.md","path":"13-mlops/mlflow/references/deployment.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/deployment.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/mlflow/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/deployment.md)","tokens":300,"sizeBytes":217},{"name":"model-registry.md","path":"13-mlops/mlflow/references/model-registry.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/model-registry.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/mlflow/references/model-registry.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/model-registry.md)","tokens":300,"sizeBytes":225},{"name":"tracking.md","path":"13-mlops/mlflow/references/tracking.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/tracking.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/mlflow/references/tracking.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/references/tracking.md)","tokens":300,"sizeBytes":213},{"name":"SKILL.md","path":"13-mlops/mlflow/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/SKILL.md","title":"mlflow","category":"anthropic-skill","format":"markdown","content":"# mlflow\n\nPath: `13-mlops/mlflow/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/SKILL.md)","tokens":300,"sizeBytes":175},{"name":"integrations.md","path":"13-mlops/swanlab/references/integrations.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/integrations.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/swanlab/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/integrations.md)","tokens":300,"sizeBytes":223},{"name":"visualization.md","path":"13-mlops/swanlab/references/visualization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/visualization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/swanlab/references/visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/references/visualization.md)","tokens":300,"sizeBytes":225},{"name":"SKILL.md","path":"13-mlops/swanlab/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/SKILL.md","title":"swanlab","category":"anthropic-skill","format":"markdown","content":"# swanlab\n\nPath: `13-mlops/swanlab/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/swanlab/SKILL.md)","tokens":300,"sizeBytes":178},{"name":"integrations.md","path":"13-mlops/tensorboard/references/integrations.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/integrations.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/tensorboard/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/integrations.md)","tokens":300,"sizeBytes":231},{"name":"profiling.md","path":"13-mlops/tensorboard/references/profiling.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/profiling.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/tensorboard/references/profiling.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/profiling.md)","tokens":300,"sizeBytes":225},{"name":"visualization.md","path":"13-mlops/tensorboard/references/visualization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/visualization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/tensorboard/references/visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/references/visualization.md)","tokens":300,"sizeBytes":233},{"name":"SKILL.md","path":"13-mlops/tensorboard/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/SKILL.md","title":"tensorboard","category":"anthropic-skill","format":"markdown","content":"# tensorboard\n\nPath: `13-mlops/tensorboard/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/SKILL.md)","tokens":300,"sizeBytes":190},{"name":"artifacts.md","path":"13-mlops/weights-and-biases/references/artifacts.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/artifacts.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/artifacts.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/artifacts.md)","tokens":300,"sizeBytes":239},{"name":"integrations.md","path":"13-mlops/weights-and-biases/references/integrations.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/integrations.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/integrations.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/integrations.md)","tokens":300,"sizeBytes":245},{"name":"sweeps.md","path":"13-mlops/weights-and-biases/references/sweeps.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/sweeps.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `13-mlops/weights-and-biases/references/sweeps.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/references/sweeps.md)","tokens":300,"sizeBytes":233},{"name":"SKILL.md","path":"13-mlops/weights-and-biases/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/SKILL.md","title":"weights-and-biases","category":"anthropic-skill","format":"markdown","content":"# weights-and-biases\n\nPath: `13-mlops/weights-and-biases/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/SKILL.md)","tokens":300,"sizeBytes":211},{"name":"api.md","path":"14-agents/a-evolve/references/api.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/api.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/api.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/api.md)","tokens":300,"sizeBytes":209},{"name":"architecture.md","path":"14-agents/a-evolve/references/architecture.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/architecture.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/architecture.md)","tokens":300,"sizeBytes":227},{"name":"design-patterns.md","path":"14-agents/a-evolve/references/design-patterns.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/design-patterns.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/design-patterns.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/design-patterns.md)","tokens":300,"sizeBytes":233},{"name":"examples.md","path":"14-agents/a-evolve/references/examples.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/examples.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/examples.md)","tokens":300,"sizeBytes":219},{"name":"issues.md","path":"14-agents/a-evolve/references/issues.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/issues.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/issues.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/issues.md)","tokens":300,"sizeBytes":215},{"name":"README.md","path":"14-agents/a-evolve/references/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/README.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/README.md)","tokens":300,"sizeBytes":215},{"name":"releases.md","path":"14-agents/a-evolve/references/releases.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/releases.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/releases.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/releases.md)","tokens":300,"sizeBytes":219},{"name":"tutorials.md","path":"14-agents/a-evolve/references/tutorials.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/tutorials.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/a-evolve/references/tutorials.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/references/tutorials.md)","tokens":300,"sizeBytes":221},{"name":"SKILL.md","path":"14-agents/a-evolve/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/SKILL.md","title":"a-evolve","category":"anthropic-skill","format":"markdown","content":"# a-evolve\n\nPath: `14-agents/a-evolve/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/a-evolve/SKILL.md)","tokens":300,"sizeBytes":183},{"name":"advanced-usage.md","path":"14-agents/autogpt/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/autogpt/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/advanced-usage.md)","tokens":300,"sizeBytes":229},{"name":"troubleshooting.md","path":"14-agents/autogpt/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/autogpt/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/references/troubleshooting.md)","tokens":300,"sizeBytes":231},{"name":"SKILL.md","path":"14-agents/autogpt/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/SKILL.md","title":"autogpt","category":"anthropic-skill","format":"markdown","content":"# autogpt\n\nPath: `14-agents/autogpt/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/SKILL.md)","tokens":300,"sizeBytes":180},{"name":"flows.md","path":"14-agents/crewai/references/flows.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/flows.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/crewai/references/flows.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/flows.md)","tokens":300,"sizeBytes":209},{"name":"tools.md","path":"14-agents/crewai/references/tools.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/tools.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/crewai/references/tools.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/tools.md)","tokens":300,"sizeBytes":209},{"name":"troubleshooting.md","path":"14-agents/crewai/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/crewai/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/references/troubleshooting.md)","tokens":300,"sizeBytes":229},{"name":"SKILL.md","path":"14-agents/crewai/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/SKILL.md","title":"crewai","category":"anthropic-skill","format":"markdown","content":"# crewai\n\nPath: `14-agents/crewai/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/SKILL.md)","tokens":300,"sizeBytes":177},{"name":"agents.md","path":"14-agents/langchain/references/agents.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/agents.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/langchain/references/agents.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/agents.md)","tokens":300,"sizeBytes":217},{"name":"integration.md","path":"14-agents/langchain/references/integration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/integration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/langchain/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/integration.md)","tokens":300,"sizeBytes":227},{"name":"rag.md","path":"14-agents/langchain/references/rag.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/rag.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/langchain/references/rag.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/references/rag.md)","tokens":300,"sizeBytes":211},{"name":"SKILL.md","path":"14-agents/langchain/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/SKILL.md","title":"langchain","category":"anthropic-skill","format":"markdown","content":"# langchain\n\nPath: `14-agents/langchain/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/SKILL.md)","tokens":300,"sizeBytes":186},{"name":"agents.md","path":"14-agents/llamaindex/references/agents.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/agents.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/llamaindex/references/agents.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/agents.md)","tokens":300,"sizeBytes":219},{"name":"data_connectors.md","path":"14-agents/llamaindex/references/data_connectors.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/data_connectors.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/llamaindex/references/data_connectors.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/data_connectors.md)","tokens":300,"sizeBytes":237},{"name":"query_engines.md","path":"14-agents/llamaindex/references/query_engines.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/query_engines.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `14-agents/llamaindex/references/query_engines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/references/query_engines.md)","tokens":300,"sizeBytes":233},{"name":"SKILL.md","path":"14-agents/llamaindex/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/SKILL.md","title":"llamaindex","category":"anthropic-skill","format":"markdown","content":"# llamaindex\n\nPath: `14-agents/llamaindex/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/SKILL.md)","tokens":300,"sizeBytes":189},{"name":"integration.md","path":"15-rag/chroma/references/integration.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/references/integration.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `15-rag/chroma/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/references/integration.md)","tokens":300,"sizeBytes":215},{"name":"SKILL.md","path":"15-rag/chroma/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/SKILL.md","title":"chroma","category":"anthropic-skill","format":"markdown","content":"# chroma\n\nPath: `15-rag/chroma/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/SKILL.md)","tokens":300,"sizeBytes":171},{"name":"index_types.md","path":"15-rag/faiss/references/index_types.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/references/index_types.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `15-rag/faiss/references/index_types.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/references/index_types.md)","tokens":300,"sizeBytes":213},{"name":"SKILL.md","path":"15-rag/faiss/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/SKILL.md","title":"faiss","category":"anthropic-skill","format":"markdown","content":"# faiss\n\nPath: `15-rag/faiss/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/SKILL.md)","tokens":300,"sizeBytes":168},{"name":"deployment.md","path":"15-rag/pinecone/references/deployment.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/references/deployment.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `15-rag/pinecone/references/deployment.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/references/deployment.md)","tokens":300,"sizeBytes":217},{"name":"SKILL.md","path":"15-rag/pinecone/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/SKILL.md","title":"pinecone","category":"anthropic-skill","format":"markdown","content":"# pinecone\n\nPath: `15-rag/pinecone/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/SKILL.md)","tokens":300,"sizeBytes":177},{"name":"advanced-usage.md","path":"15-rag/qdrant/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `15-rag/qdrant/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/advanced-usage.md)","tokens":300,"sizeBytes":221},{"name":"troubleshooting.md","path":"15-rag/qdrant/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `15-rag/qdrant/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/references/troubleshooting.md)","tokens":300,"sizeBytes":223},{"name":"SKILL.md","path":"15-rag/qdrant/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/SKILL.md","title":"qdrant","category":"anthropic-skill","format":"markdown","content":"# qdrant\n\nPath: `15-rag/qdrant/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/SKILL.md)","tokens":300,"sizeBytes":171},{"name":"models.md","path":"15-rag/sentence-transformers/references/models.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/references/models.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `15-rag/sentence-transformers/references/models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/references/models.md)","tokens":300,"sizeBytes":235},{"name":"SKILL.md","path":"15-rag/sentence-transformers/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/SKILL.md","title":"sentence-transformers","category":"anthropic-skill","format":"markdown","content":"# sentence-transformers\n\nPath: `15-rag/sentence-transformers/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/SKILL.md)","tokens":300,"sizeBytes":216},{"name":"examples.md","path":"16-prompt-engineering/dspy/references/examples.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/examples.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/examples.md)","tokens":300,"sizeBytes":235},{"name":"modules.md","path":"16-prompt-engineering/dspy/references/modules.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/modules.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/modules.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/modules.md)","tokens":300,"sizeBytes":233},{"name":"optimizers.md","path":"16-prompt-engineering/dspy/references/optimizers.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/optimizers.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/dspy/references/optimizers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/references/optimizers.md)","tokens":300,"sizeBytes":239},{"name":"SKILL.md","path":"16-prompt-engineering/dspy/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/SKILL.md","title":"dspy","category":"anthropic-skill","format":"markdown","content":"# dspy\n\nPath: `16-prompt-engineering/dspy/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/SKILL.md)","tokens":300,"sizeBytes":195},{"name":"backends.md","path":"16-prompt-engineering/guidance/references/backends.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/backends.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/backends.md)","tokens":300,"sizeBytes":243},{"name":"constraints.md","path":"16-prompt-engineering/guidance/references/constraints.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/constraints.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/constraints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/constraints.md)","tokens":300,"sizeBytes":249},{"name":"examples.md","path":"16-prompt-engineering/guidance/references/examples.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/examples.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/guidance/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/references/examples.md)","tokens":300,"sizeBytes":243},{"name":"SKILL.md","path":"16-prompt-engineering/guidance/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/SKILL.md","title":"guidance","category":"anthropic-skill","format":"markdown","content":"# guidance\n\nPath: `16-prompt-engineering/guidance/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/SKILL.md)","tokens":300,"sizeBytes":207},{"name":"examples.md","path":"16-prompt-engineering/instructor/references/examples.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/examples.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/examples.md)","tokens":300,"sizeBytes":247},{"name":"providers.md","path":"16-prompt-engineering/instructor/references/providers.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/providers.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/providers.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/providers.md)","tokens":300,"sizeBytes":249},{"name":"validation.md","path":"16-prompt-engineering/instructor/references/validation.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/validation.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/instructor/references/validation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/references/validation.md)","tokens":300,"sizeBytes":251},{"name":"SKILL.md","path":"16-prompt-engineering/instructor/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/SKILL.md","title":"instructor","category":"anthropic-skill","format":"markdown","content":"# instructor\n\nPath: `16-prompt-engineering/instructor/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/SKILL.md)","tokens":300,"sizeBytes":213},{"name":"backends.md","path":"16-prompt-engineering/outlines/references/backends.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/backends.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/backends.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/backends.md)","tokens":300,"sizeBytes":243},{"name":"examples.md","path":"16-prompt-engineering/outlines/references/examples.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/examples.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/examples.md)","tokens":300,"sizeBytes":243},{"name":"json_generation.md","path":"16-prompt-engineering/outlines/references/json_generation.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/json_generation.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `16-prompt-engineering/outlines/references/json_generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/references/json_generation.md)","tokens":300,"sizeBytes":257},{"name":"SKILL.md","path":"16-prompt-engineering/outlines/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/SKILL.md","title":"outlines","category":"anthropic-skill","format":"markdown","content":"# outlines\n\nPath: `16-prompt-engineering/outlines/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/SKILL.md)","tokens":300,"sizeBytes":207},{"name":"advanced-usage.md","path":"17-observability/langsmith/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `17-observability/langsmith/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/advanced-usage.md)","tokens":300,"sizeBytes":247},{"name":"troubleshooting.md","path":"17-observability/langsmith/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `17-observability/langsmith/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/references/troubleshooting.md)","tokens":300,"sizeBytes":249},{"name":"SKILL.md","path":"17-observability/langsmith/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/SKILL.md","title":"langsmith","category":"anthropic-skill","format":"markdown","content":"# langsmith\n\nPath: `17-observability/langsmith/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/SKILL.md)","tokens":300,"sizeBytes":200},{"name":"advanced-usage.md","path":"17-observability/phoenix/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `17-observability/phoenix/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/advanced-usage.md)","tokens":300,"sizeBytes":243},{"name":"troubleshooting.md","path":"17-observability/phoenix/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `17-observability/phoenix/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/references/troubleshooting.md)","tokens":300,"sizeBytes":245},{"name":"SKILL.md","path":"17-observability/phoenix/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/SKILL.md","title":"phoenix","category":"anthropic-skill","format":"markdown","content":"# phoenix\n\nPath: `17-observability/phoenix/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/SKILL.md)","tokens":300,"sizeBytes":194},{"name":"advanced-usage.md","path":"18-multimodal/audiocraft/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/audiocraft/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/advanced-usage.md)","tokens":300,"sizeBytes":243},{"name":"troubleshooting.md","path":"18-multimodal/audiocraft/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/audiocraft/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/references/troubleshooting.md)","tokens":300,"sizeBytes":245},{"name":"SKILL.md","path":"18-multimodal/audiocraft/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/SKILL.md","title":"audiocraft","category":"anthropic-skill","format":"markdown","content":"# audiocraft\n\nPath: `18-multimodal/audiocraft/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/SKILL.md)","tokens":300,"sizeBytes":197},{"name":"advanced-usage.md","path":"18-multimodal/blip-2/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/blip-2/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/advanced-usage.md)","tokens":300,"sizeBytes":235},{"name":"troubleshooting.md","path":"18-multimodal/blip-2/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/blip-2/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/references/troubleshooting.md)","tokens":300,"sizeBytes":237},{"name":"SKILL.md","path":"18-multimodal/blip-2/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/SKILL.md","title":"blip-2","category":"anthropic-skill","format":"markdown","content":"# blip-2\n\nPath: `18-multimodal/blip-2/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/SKILL.md)","tokens":300,"sizeBytes":185},{"name":"applications.md","path":"18-multimodal/clip/references/applications.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/references/applications.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/clip/references/applications.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/references/applications.md)","tokens":300,"sizeBytes":227},{"name":"SKILL.md","path":"18-multimodal/clip/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/SKILL.md","title":"clip","category":"anthropic-skill","format":"markdown","content":"# clip\n\nPath: `18-multimodal/clip/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/SKILL.md)","tokens":300,"sizeBytes":179},{"name":"libero-commands.md","path":"18-multimodal/cosmos-policy/references/libero-commands.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/libero-commands.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/cosmos-policy/references/libero-commands.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/libero-commands.md)","tokens":300,"sizeBytes":251},{"name":"robocasa-commands.md","path":"18-multimodal/cosmos-policy/references/robocasa-commands.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/robocasa-commands.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/cosmos-policy/references/robocasa-commands.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/references/robocasa-commands.md)","tokens":300,"sizeBytes":255},{"name":"SKILL.md","path":"18-multimodal/cosmos-policy/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/SKILL.md","title":"cosmos-policy","category":"anthropic-skill","format":"markdown","content":"# cosmos-policy\n\nPath: `18-multimodal/cosmos-policy/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/SKILL.md)","tokens":300,"sizeBytes":206},{"name":"training.md","path":"18-multimodal/llava/references/training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/references/training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/llava/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/references/training.md)","tokens":300,"sizeBytes":221},{"name":"SKILL.md","path":"18-multimodal/llava/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/SKILL.md","title":"llava","category":"anthropic-skill","format":"markdown","content":"# llava\n\nPath: `18-multimodal/llava/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/SKILL.md)","tokens":300,"sizeBytes":182},{"name":"checkpoints-and-env-map.md","path":"18-multimodal/openpi/references/checkpoints-and-env-map.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/checkpoints-and-env-map.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openpi/references/checkpoints-and-env-map.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/checkpoints-and-env-map.md)","tokens":300,"sizeBytes":253},{"name":"config-recipes.md","path":"18-multimodal/openpi/references/config-recipes.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/config-recipes.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openpi/references/config-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/config-recipes.md)","tokens":300,"sizeBytes":235},{"name":"pytorch-gotchas.md","path":"18-multimodal/openpi/references/pytorch-gotchas.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/pytorch-gotchas.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openpi/references/pytorch-gotchas.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/pytorch-gotchas.md)","tokens":300,"sizeBytes":237},{"name":"remote-client-pattern.md","path":"18-multimodal/openpi/references/remote-client-pattern.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/remote-client-pattern.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openpi/references/remote-client-pattern.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/remote-client-pattern.md)","tokens":300,"sizeBytes":249},{"name":"training-debugging.md","path":"18-multimodal/openpi/references/training-debugging.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/training-debugging.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openpi/references/training-debugging.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/references/training-debugging.md)","tokens":300,"sizeBytes":243},{"name":"SKILL.md","path":"18-multimodal/openpi/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/SKILL.md","title":"openpi","category":"anthropic-skill","format":"markdown","content":"# openpi\n\nPath: `18-multimodal/openpi/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/SKILL.md)","tokens":300,"sizeBytes":185},{"name":"aloha-workflow.md","path":"18-multimodal/openvla-oft/references/aloha-workflow.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/aloha-workflow.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/aloha-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/aloha-workflow.md)","tokens":300,"sizeBytes":245},{"name":"config-troubleshooting.md","path":"18-multimodal/openvla-oft/references/config-troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/config-troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/config-troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/config-troubleshooting.md)","tokens":300,"sizeBytes":261},{"name":"libero-workflow.md","path":"18-multimodal/openvla-oft/references/libero-workflow.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/libero-workflow.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/libero-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/libero-workflow.md)","tokens":300,"sizeBytes":247},{"name":"paper-and-checkpoints.md","path":"18-multimodal/openvla-oft/references/paper-and-checkpoints.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/paper-and-checkpoints.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/openvla-oft/references/paper-and-checkpoints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/references/paper-and-checkpoints.md)","tokens":300,"sizeBytes":259},{"name":"SKILL.md","path":"18-multimodal/openvla-oft/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/SKILL.md","title":"openvla-oft","category":"anthropic-skill","format":"markdown","content":"# openvla-oft\n\nPath: `18-multimodal/openvla-oft/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/SKILL.md)","tokens":300,"sizeBytes":200},{"name":"advanced-usage.md","path":"18-multimodal/segment-anything/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/segment-anything/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/advanced-usage.md)","tokens":300,"sizeBytes":255},{"name":"troubleshooting.md","path":"18-multimodal/segment-anything/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/segment-anything/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/references/troubleshooting.md)","tokens":300,"sizeBytes":257},{"name":"SKILL.md","path":"18-multimodal/segment-anything/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/SKILL.md","title":"segment-anything","category":"anthropic-skill","format":"markdown","content":"# segment-anything\n\nPath: `18-multimodal/segment-anything/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/SKILL.md)","tokens":300,"sizeBytes":215},{"name":"advanced-usage.md","path":"18-multimodal/stable-diffusion/references/advanced-usage.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/advanced-usage.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/stable-diffusion/references/advanced-usage.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/advanced-usage.md)","tokens":300,"sizeBytes":255},{"name":"troubleshooting.md","path":"18-multimodal/stable-diffusion/references/troubleshooting.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/troubleshooting.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/stable-diffusion/references/troubleshooting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/references/troubleshooting.md)","tokens":300,"sizeBytes":257},{"name":"SKILL.md","path":"18-multimodal/stable-diffusion/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/SKILL.md","title":"stable-diffusion","category":"anthropic-skill","format":"markdown","content":"# stable-diffusion\n\nPath: `18-multimodal/stable-diffusion/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/SKILL.md)","tokens":300,"sizeBytes":215},{"name":"languages.md","path":"18-multimodal/whisper/references/languages.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/references/languages.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `18-multimodal/whisper/references/languages.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/references/languages.md)","tokens":300,"sizeBytes":227},{"name":"SKILL.md","path":"18-multimodal/whisper/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/SKILL.md","title":"whisper","category":"anthropic-skill","format":"markdown","content":"# whisper\n\nPath: `18-multimodal/whisper/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/SKILL.md)","tokens":300,"sizeBytes":188},{"name":"minillm.md","path":"19-emerging-techniques/knowledge-distillation/references/minillm.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/references/minillm.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/knowledge-distillation/references/minillm.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/references/minillm.md)","tokens":300,"sizeBytes":271},{"name":"SKILL.md","path":"19-emerging-techniques/knowledge-distillation/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/SKILL.md","title":"knowledge-distillation","category":"anthropic-skill","format":"markdown","content":"# knowledge-distillation\n\nPath: `19-emerging-techniques/knowledge-distillation/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/SKILL.md)","tokens":300,"sizeBytes":251},{"name":"extension_methods.md","path":"19-emerging-techniques/long-context/references/extension_methods.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/extension_methods.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/extension_methods.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/extension_methods.md)","tokens":300,"sizeBytes":271},{"name":"fine_tuning.md","path":"19-emerging-techniques/long-context/references/fine_tuning.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/fine_tuning.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/fine_tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/fine_tuning.md)","tokens":300,"sizeBytes":259},{"name":"rope.md","path":"19-emerging-techniques/long-context/references/rope.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/rope.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/long-context/references/rope.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/references/rope.md)","tokens":300,"sizeBytes":245},{"name":"SKILL.md","path":"19-emerging-techniques/long-context/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/SKILL.md","title":"long-context","category":"anthropic-skill","format":"markdown","content":"# long-context\n\nPath: `19-emerging-techniques/long-context/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/SKILL.md)","tokens":300,"sizeBytes":221},{"name":"coefficient-tuning.md","path":"19-emerging-techniques/model-merging/references/coefficient-tuning.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/coefficient-tuning.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/coefficient-tuning.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/coefficient-tuning.md)","tokens":300,"sizeBytes":275},{"name":"evaluation.md","path":"19-emerging-techniques/model-merging/references/evaluation.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/evaluation.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/evaluation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/evaluation.md)","tokens":300,"sizeBytes":259},{"name":"examples.md","path":"19-emerging-techniques/model-merging/references/examples.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/examples.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/examples.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/examples.md)","tokens":300,"sizeBytes":255},{"name":"methods.md","path":"19-emerging-techniques/model-merging/references/methods.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/methods.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/model-merging/references/methods.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/references/methods.md)","tokens":300,"sizeBytes":253},{"name":"SKILL.md","path":"19-emerging-techniques/model-merging/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/SKILL.md","title":"model-merging","category":"anthropic-skill","format":"markdown","content":"# model-merging\n\nPath: `19-emerging-techniques/model-merging/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/SKILL.md)","tokens":300,"sizeBytes":224},{"name":"wanda.md","path":"19-emerging-techniques/model-pruning/references/wanda.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/references/wanda.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/model-pruning/references/wanda.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/references/wanda.md)","tokens":300,"sizeBytes":249},{"name":"SKILL.md","path":"19-emerging-techniques/model-pruning/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/SKILL.md","title":"model-pruning","category":"anthropic-skill","format":"markdown","content":"# model-pruning\n\nPath: `19-emerging-techniques/model-pruning/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/SKILL.md)","tokens":300,"sizeBytes":224},{"name":"architectures.md","path":"19-emerging-techniques/moe-training/references/architectures.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/architectures.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/architectures.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/architectures.md)","tokens":300,"sizeBytes":263},{"name":"inference.md","path":"19-emerging-techniques/moe-training/references/inference.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/inference.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/inference.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/inference.md)","tokens":300,"sizeBytes":255},{"name":"training.md","path":"19-emerging-techniques/moe-training/references/training.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/training.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/moe-training/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/references/training.md)","tokens":300,"sizeBytes":253},{"name":"SKILL.md","path":"19-emerging-techniques/moe-training/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/SKILL.md","title":"moe-training","category":"anthropic-skill","format":"markdown","content":"# moe-training\n\nPath: `19-emerging-techniques/moe-training/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/SKILL.md)","tokens":300,"sizeBytes":221},{"name":"lookahead.md","path":"19-emerging-techniques/speculative-decoding/references/lookahead.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/lookahead.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/speculative-decoding/references/lookahead.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/lookahead.md)","tokens":300,"sizeBytes":271},{"name":"medusa.md","path":"19-emerging-techniques/speculative-decoding/references/medusa.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/medusa.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `19-emerging-techniques/speculative-decoding/references/medusa.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/references/medusa.md)","tokens":300,"sizeBytes":265},{"name":"SKILL.md","path":"19-emerging-techniques/speculative-decoding/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/SKILL.md","title":"speculative-decoding","category":"anthropic-skill","format":"markdown","content":"# speculative-decoding\n\nPath: `19-emerging-techniques/speculative-decoding/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/SKILL.md)","tokens":300,"sizeBytes":245},{"name":"data-visualization.md","path":"20-ml-paper-writing/academic-plotting/references/data-visualization.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/data-visualization.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/data-visualization.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/data-visualization.md)","tokens":300,"sizeBytes":277},{"name":"diagram-generation.md","path":"20-ml-paper-writing/academic-plotting/references/diagram-generation.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/diagram-generation.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/diagram-generation.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/diagram-generation.md)","tokens":300,"sizeBytes":277},{"name":"style-guide.md","path":"20-ml-paper-writing/academic-plotting/references/style-guide.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/style-guide.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/academic-plotting/references/style-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/references/style-guide.md)","tokens":300,"sizeBytes":263},{"name":"SKILL.md","path":"20-ml-paper-writing/academic-plotting/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/SKILL.md","title":"academic-plotting","category":"anthropic-skill","format":"markdown","content":"# academic-plotting\n\nPath: `20-ml-paper-writing/academic-plotting/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/SKILL.md)","tokens":300,"sizeBytes":230},{"name":"checklists.md","path":"20-ml-paper-writing/ml-paper-writing/references/checklists.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/checklists.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/checklists.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/checklists.md)","tokens":300,"sizeBytes":259},{"name":"citation-workflow.md","path":"20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/citation-workflow.md)","tokens":300,"sizeBytes":273},{"name":"reviewer-guidelines.md","path":"20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/reviewer-guidelines.md)","tokens":300,"sizeBytes":277},{"name":"sources.md","path":"20-ml-paper-writing/ml-paper-writing/references/sources.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/sources.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/sources.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/sources.md)","tokens":300,"sizeBytes":253},{"name":"writing-guide.md","path":"20-ml-paper-writing/ml-paper-writing/references/writing-guide.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/writing-guide.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/ml-paper-writing/references/writing-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/references/writing-guide.md)","tokens":300,"sizeBytes":265},{"name":"SKILL.md","path":"20-ml-paper-writing/ml-paper-writing/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/SKILL.md","title":"ml-paper-writing","category":"anthropic-skill","format":"markdown","content":"# ml-paper-writing\n\nPath: `20-ml-paper-writing/ml-paper-writing/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/SKILL.md)","tokens":300,"sizeBytes":227},{"name":"slide-templates.md","path":"20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/references/slide-templates.md)","tokens":300,"sizeBytes":291},{"name":"SKILL.md","path":"20-ml-paper-writing/presenting-conference-talks/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/SKILL.md","title":"presenting-conference-talks","category":"anthropic-skill","format":"markdown","content":"# presenting-conference-talks\n\nPath: `20-ml-paper-writing/presenting-conference-talks/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/SKILL.md)","tokens":300,"sizeBytes":260},{"name":"checklist.md","path":"20-ml-paper-writing/systems-paper-writing/references/checklist.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/checklist.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/checklist.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/checklist.md)","tokens":300,"sizeBytes":267},{"name":"reviewer-guidelines.md","path":"20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/reviewer-guidelines.md)","tokens":300,"sizeBytes":287},{"name":"section-blueprints.md","path":"20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/section-blueprints.md)","tokens":300,"sizeBytes":285},{"name":"systems-conferences.md","path":"20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/systems-conferences.md)","tokens":300,"sizeBytes":287},{"name":"writing-patterns.md","path":"20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/references/writing-patterns.md)","tokens":300,"sizeBytes":281},{"name":"SKILL.md","path":"20-ml-paper-writing/systems-paper-writing/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/SKILL.md","title":"systems-paper-writing","category":"anthropic-skill","format":"markdown","content":"# systems-paper-writing\n\nPath: `20-ml-paper-writing/systems-paper-writing/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/SKILL.md)","tokens":300,"sizeBytes":242},{"name":"SKILL.md","path":"21-research-ideation/brainstorming-research-ideas/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/SKILL.md","title":"brainstorming-research-ideas","category":"anthropic-skill","format":"markdown","content":"# brainstorming-research-ideas\n\nPath: `21-research-ideation/brainstorming-research-ideas/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/SKILL.md)","tokens":300,"sizeBytes":265},{"name":"SKILL.md","path":"21-research-ideation/creative-thinking-for-research/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/SKILL.md","title":"creative-thinking-for-research","category":"anthropic-skill","format":"markdown","content":"# creative-thinking-for-research\n\nPath: `21-research-ideation/creative-thinking-for-research/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/SKILL.md)","tokens":300,"sizeBytes":271},{"name":"ara-schema.md","path":"22-agent-native-research-artifact/compiler/references/ara-schema.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/ara-schema.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/ara-schema.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/ara-schema.md)","tokens":300,"sizeBytes":271},{"name":"exploration-tree-spec.md","path":"22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/exploration-tree-spec.md)","tokens":300,"sizeBytes":293},{"name":"validation-checklist.md","path":"22-agent-native-research-artifact/compiler/references/validation-checklist.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/validation-checklist.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `22-agent-native-research-artifact/compiler/references/validation-checklist.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/references/validation-checklist.md)","tokens":300,"sizeBytes":291},{"name":"SKILL.md","path":"22-agent-native-research-artifact/compiler/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/SKILL.md","title":"compiler","category":"anthropic-skill","format":"markdown","content":"# compiler\n\nPath: `22-agent-native-research-artifact/compiler/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/SKILL.md)","tokens":300,"sizeBytes":231},{"name":"event-taxonomy.md","path":"22-agent-native-research-artifact/research-manager/references/event-taxonomy.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/event-taxonomy.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/event-taxonomy.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/event-taxonomy.md)","tokens":300,"sizeBytes":295},{"name":"provenance-tags.md","path":"22-agent-native-research-artifact/research-manager/references/provenance-tags.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/provenance-tags.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/provenance-tags.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/provenance-tags.md)","tokens":300,"sizeBytes":297},{"name":"session-protocol.md","path":"22-agent-native-research-artifact/research-manager/references/session-protocol.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/session-protocol.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `22-agent-native-research-artifact/research-manager/references/session-protocol.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/references/session-protocol.md)","tokens":300,"sizeBytes":299},{"name":"SKILL.md","path":"22-agent-native-research-artifact/research-manager/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/SKILL.md","title":"research-manager","category":"anthropic-skill","format":"markdown","content":"# research-manager\n\nPath: `22-agent-native-research-artifact/research-manager/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/SKILL.md)","tokens":300,"sizeBytes":255},{"name":"review-dimensions.md","path":"22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md","title":"References Skill","category":"anthropic-skill","format":"markdown","content":"# References Skill\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/references/review-dimensions.md)","tokens":300,"sizeBytes":297},{"name":"SKILL.md","path":"22-agent-native-research-artifact/rigor-reviewer/SKILL.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/SKILL.md","title":"rigor-reviewer","category":"anthropic-skill","format":"markdown","content":"# rigor-reviewer\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/SKILL.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/SKILL.md)","tokens":300,"sizeBytes":249},{"name":"package-lock.json","path":"packages/ai-research-skills/package-lock.json","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package-lock.json","title":"Ai-research-skills Skill","category":"anthropic-skill","format":"json","content":"# Ai-research-skills Skill\n\nPath: `packages/ai-research-skills/package-lock.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package-lock.json)","tokens":300,"sizeBytes":235},{"name":"package.json","path":"packages/ai-research-skills/package.json","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package.json","title":"Ai-research-skills Skill","category":"anthropic-skill","format":"json","content":"# Ai-research-skills Skill\n\nPath: `packages/ai-research-skills/package.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/package.json)","tokens":300,"sizeBytes":225},{"name":"CLAUDE.md","path":"CLAUDE.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/CLAUDE.md","title":"Claude Agent Guidelines & System Prompt","category":"claude-rule","format":"markdown","content":"# Claude Agent Guidelines & System Prompt\n\nPath: `CLAUDE.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/CLAUDE.md)","tokens":300,"sizeBytes":178},{"name":"marketplace.json","path":".claude-plugin/marketplace.json","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/.claude-plugin/marketplace.json","title":"Claude Plugin Marketplace Catalog","category":"marketplace","format":"json","content":"# Claude Plugin Marketplace Catalog\n\nPath: `.claude-plugin/marketplace.json`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/.claude-plugin/marketplace.json)","tokens":300,"sizeBytes":216},{"name":"README.md","path":"packages/ai-research-skills/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/README.md","title":"ai-research-skills Documentation","category":"plugin-manifest","format":"markdown","content":"# ai-research-skills Documentation\n\nPath: `packages/ai-research-skills/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/packages/ai-research-skills/README.md)","tokens":300,"sizeBytes":227},{"name":"README.md","path":"0-autoresearch-skill/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `0-autoresearch-skill/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/README.md)","tokens":300,"sizeBytes":190},{"name":"README.md","path":"01-model-architecture/litgpt/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `01-model-architecture/litgpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/README.md)","tokens":300,"sizeBytes":206},{"name":"README.md","path":"01-model-architecture/mamba/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `01-model-architecture/mamba/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/README.md)","tokens":300,"sizeBytes":204},{"name":"README.md","path":"01-model-architecture/nanogpt/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `01-model-architecture/nanogpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/README.md)","tokens":300,"sizeBytes":208},{"name":"README.md","path":"01-model-architecture/rwkv/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `01-model-architecture/rwkv/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/README.md)","tokens":300,"sizeBytes":202},{"name":"README.md","path":"01-model-architecture/torchtitan/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `01-model-architecture/torchtitan/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/README.md)","tokens":300,"sizeBytes":214},{"name":"README.md","path":"02-tokenization/huggingface-tokenizers/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `02-tokenization/huggingface-tokenizers/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/README.md)","tokens":300,"sizeBytes":226},{"name":"README.md","path":"02-tokenization/sentencepiece/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `02-tokenization/sentencepiece/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/sentencepiece/README.md)","tokens":300,"sizeBytes":208},{"name":"README.md","path":"03-fine-tuning/axolotl/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `03-fine-tuning/axolotl/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/axolotl/README.md)","tokens":300,"sizeBytes":194},{"name":"README.md","path":"03-fine-tuning/llama-factory/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `03-fine-tuning/llama-factory/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/llama-factory/README.md)","tokens":300,"sizeBytes":206},{"name":"README.md","path":"03-fine-tuning/peft/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `03-fine-tuning/peft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/peft/README.md)","tokens":300,"sizeBytes":188},{"name":"README.md","path":"03-fine-tuning/unsloth/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `03-fine-tuning/unsloth/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/03-fine-tuning/unsloth/README.md)","tokens":300,"sizeBytes":194},{"name":"README.md","path":"04-mechanistic-interpretability/nnsight/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `04-mechanistic-interpretability/nnsight/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/nnsight/README.md)","tokens":300,"sizeBytes":228},{"name":"README.md","path":"04-mechanistic-interpretability/pyvene/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `04-mechanistic-interpretability/pyvene/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/pyvene/README.md)","tokens":300,"sizeBytes":226},{"name":"README.md","path":"04-mechanistic-interpretability/saelens/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `04-mechanistic-interpretability/saelens/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/saelens/README.md)","tokens":300,"sizeBytes":228},{"name":"README.md","path":"04-mechanistic-interpretability/transformer-lens/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `04-mechanistic-interpretability/transformer-lens/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/04-mechanistic-interpretability/transformer-lens/README.md)","tokens":300,"sizeBytes":246},{"name":"README.md","path":"05-data-processing/nemo-curator/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `05-data-processing/nemo-curator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/nemo-curator/README.md)","tokens":300,"sizeBytes":212},{"name":"README.md","path":"05-data-processing/ray-data/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `05-data-processing/ray-data/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/05-data-processing/ray-data/README.md)","tokens":300,"sizeBytes":204},{"name":"README.md","path":"06-post-training/grpo-rl-training/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/grpo-rl-training/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/grpo-rl-training/README.md)","tokens":300,"sizeBytes":216},{"name":"README.md","path":"06-post-training/miles/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/miles/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/miles/README.md)","tokens":300,"sizeBytes":194},{"name":"README.md","path":"06-post-training/openrlhf/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/openrlhf/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/openrlhf/README.md)","tokens":300,"sizeBytes":200},{"name":"README.md","path":"06-post-training/simpo/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/simpo/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/simpo/README.md)","tokens":300,"sizeBytes":194},{"name":"README.md","path":"06-post-training/slime/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/slime/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/slime/README.md)","tokens":300,"sizeBytes":194},{"name":"README.md","path":"06-post-training/torchforge/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/torchforge/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/torchforge/README.md)","tokens":300,"sizeBytes":204},{"name":"README.md","path":"06-post-training/trl-fine-tuning/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/trl-fine-tuning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/trl-fine-tuning/README.md)","tokens":300,"sizeBytes":214},{"name":"README.md","path":"06-post-training/verl/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `06-post-training/verl/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/06-post-training/verl/README.md)","tokens":300,"sizeBytes":192},{"name":"README.md","path":"07-safety-alignment/constitutional-ai/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `07-safety-alignment/constitutional-ai/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/constitutional-ai/README.md)","tokens":300,"sizeBytes":224},{"name":"README.md","path":"07-safety-alignment/llamaguard/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `07-safety-alignment/llamaguard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/llamaguard/README.md)","tokens":300,"sizeBytes":210},{"name":"README.md","path":"07-safety-alignment/nemo-guardrails/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `07-safety-alignment/nemo-guardrails/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/nemo-guardrails/README.md)","tokens":300,"sizeBytes":220},{"name":"README.md","path":"07-safety-alignment/prompt-guard/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `07-safety-alignment/prompt-guard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/07-safety-alignment/prompt-guard/README.md)","tokens":300,"sizeBytes":214},{"name":"README.md","path":"08-distributed-training/accelerate/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `08-distributed-training/accelerate/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/accelerate/README.md)","tokens":300,"sizeBytes":218},{"name":"README.md","path":"08-distributed-training/deepspeed/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `08-distributed-training/deepspeed/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/deepspeed/README.md)","tokens":300,"sizeBytes":216},{"name":"README.md","path":"08-distributed-training/megatron-core/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `08-distributed-training/megatron-core/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/megatron-core/README.md)","tokens":300,"sizeBytes":224},{"name":"README.md","path":"08-distributed-training/pytorch-fsdp2/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `08-distributed-training/pytorch-fsdp2/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-fsdp2/README.md)","tokens":300,"sizeBytes":224},{"name":"README.md","path":"08-distributed-training/pytorch-lightning/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `08-distributed-training/pytorch-lightning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/pytorch-lightning/README.md)","tokens":300,"sizeBytes":232},{"name":"README.md","path":"08-distributed-training/ray-train/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `08-distributed-training/ray-train/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/08-distributed-training/ray-train/README.md)","tokens":300,"sizeBytes":216},{"name":"README.md","path":"09-infrastructure/lambda-labs/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `09-infrastructure/lambda-labs/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/lambda-labs/README.md)","tokens":300,"sizeBytes":208},{"name":"README.md","path":"09-infrastructure/modal/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `09-infrastructure/modal/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/modal/README.md)","tokens":300,"sizeBytes":196},{"name":"README.md","path":"09-infrastructure/skypilot/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `09-infrastructure/skypilot/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/09-infrastructure/skypilot/README.md)","tokens":300,"sizeBytes":202},{"name":"README.md","path":"10-optimization/awq/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `10-optimization/awq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/awq/README.md)","tokens":300,"sizeBytes":188},{"name":"README.md","path":"10-optimization/bitsandbytes/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `10-optimization/bitsandbytes/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/bitsandbytes/README.md)","tokens":300,"sizeBytes":206},{"name":"README.md","path":"10-optimization/flash-attention/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `10-optimization/flash-attention/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/flash-attention/README.md)","tokens":300,"sizeBytes":212},{"name":"README.md","path":"10-optimization/gguf/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `10-optimization/gguf/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gguf/README.md)","tokens":300,"sizeBytes":190},{"name":"README.md","path":"10-optimization/gptq/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `10-optimization/gptq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/gptq/README.md)","tokens":300,"sizeBytes":190},{"name":"README.md","path":"10-optimization/hqq/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `10-optimization/hqq/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/hqq/README.md)","tokens":300,"sizeBytes":188},{"name":"README.md","path":"10-optimization/ml-training-recipes/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `10-optimization/ml-training-recipes/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/10-optimization/ml-training-recipes/README.md)","tokens":300,"sizeBytes":220},{"name":"README.md","path":"11-evaluation/bigcode-evaluation-harness/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `11-evaluation/bigcode-evaluation-harness/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/bigcode-evaluation-harness/README.md)","tokens":300,"sizeBytes":230},{"name":"README.md","path":"11-evaluation/lm-evaluation-harness/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `11-evaluation/lm-evaluation-harness/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/lm-evaluation-harness/README.md)","tokens":300,"sizeBytes":220},{"name":"README.md","path":"11-evaluation/nemo-evaluator/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `11-evaluation/nemo-evaluator/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/11-evaluation/nemo-evaluator/README.md)","tokens":300,"sizeBytes":206},{"name":"README.md","path":"12-inference-serving/llama-cpp/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `12-inference-serving/llama-cpp/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/llama-cpp/README.md)","tokens":300,"sizeBytes":210},{"name":"README.md","path":"12-inference-serving/sglang/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `12-inference-serving/sglang/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/sglang/README.md)","tokens":300,"sizeBytes":204},{"name":"README.md","path":"12-inference-serving/tensorrt-llm/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `12-inference-serving/tensorrt-llm/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/tensorrt-llm/README.md)","tokens":300,"sizeBytes":216},{"name":"README.md","path":"12-inference-serving/vllm/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `12-inference-serving/vllm/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/12-inference-serving/vllm/README.md)","tokens":300,"sizeBytes":200},{"name":"README.md","path":"13-mlops/mlflow/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `13-mlops/mlflow/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/mlflow/README.md)","tokens":300,"sizeBytes":180},{"name":"README.md","path":"13-mlops/tensorboard/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `13-mlops/tensorboard/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/tensorboard/README.md)","tokens":300,"sizeBytes":190},{"name":"README.md","path":"13-mlops/weights-and-biases/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `13-mlops/weights-and-biases/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/13-mlops/weights-and-biases/README.md)","tokens":300,"sizeBytes":204},{"name":"README.md","path":"14-agents/autogpt/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `14-agents/autogpt/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/autogpt/README.md)","tokens":300,"sizeBytes":184},{"name":"README.md","path":"14-agents/crewai/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `14-agents/crewai/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/crewai/README.md)","tokens":300,"sizeBytes":182},{"name":"README.md","path":"14-agents/langchain/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `14-agents/langchain/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/langchain/README.md)","tokens":300,"sizeBytes":188},{"name":"README.md","path":"14-agents/llamaindex/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `14-agents/llamaindex/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/14-agents/llamaindex/README.md)","tokens":300,"sizeBytes":190},{"name":"README.md","path":"15-rag/chroma/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `15-rag/chroma/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/chroma/README.md)","tokens":300,"sizeBytes":176},{"name":"README.md","path":"15-rag/faiss/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `15-rag/faiss/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/faiss/README.md)","tokens":300,"sizeBytes":174},{"name":"README.md","path":"15-rag/pinecone/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `15-rag/pinecone/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/pinecone/README.md)","tokens":300,"sizeBytes":180},{"name":"README.md","path":"15-rag/qdrant/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `15-rag/qdrant/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/qdrant/README.md)","tokens":300,"sizeBytes":176},{"name":"README.md","path":"15-rag/sentence-transformers/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `15-rag/sentence-transformers/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/15-rag/sentence-transformers/README.md)","tokens":300,"sizeBytes":206},{"name":"README.md","path":"16-prompt-engineering/dspy/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `16-prompt-engineering/dspy/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/dspy/README.md)","tokens":300,"sizeBytes":202},{"name":"README.md","path":"16-prompt-engineering/guidance/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `16-prompt-engineering/guidance/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/guidance/README.md)","tokens":300,"sizeBytes":210},{"name":"README.md","path":"16-prompt-engineering/instructor/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `16-prompt-engineering/instructor/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/instructor/README.md)","tokens":300,"sizeBytes":214},{"name":"README.md","path":"16-prompt-engineering/outlines/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `16-prompt-engineering/outlines/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/16-prompt-engineering/outlines/README.md)","tokens":300,"sizeBytes":210},{"name":"README.md","path":"17-observability/langsmith/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `17-observability/langsmith/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/langsmith/README.md)","tokens":300,"sizeBytes":202},{"name":"README.md","path":"17-observability/phoenix/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `17-observability/phoenix/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/17-observability/phoenix/README.md)","tokens":300,"sizeBytes":198},{"name":"README.md","path":"18-multimodal/audiocraft/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/audiocraft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/audiocraft/README.md)","tokens":300,"sizeBytes":198},{"name":"README.md","path":"18-multimodal/blip-2/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/blip-2/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/blip-2/README.md)","tokens":300,"sizeBytes":190},{"name":"README.md","path":"18-multimodal/clip/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/clip/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/clip/README.md)","tokens":300,"sizeBytes":186},{"name":"README.md","path":"18-multimodal/cosmos-policy/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/cosmos-policy/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/cosmos-policy/README.md)","tokens":300,"sizeBytes":204},{"name":"README.md","path":"18-multimodal/llava/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/llava/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/llava/README.md)","tokens":300,"sizeBytes":188},{"name":"README.md","path":"18-multimodal/openpi/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/openpi/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openpi/README.md)","tokens":300,"sizeBytes":190},{"name":"README.md","path":"18-multimodal/openvla-oft/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/openvla-oft/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/openvla-oft/README.md)","tokens":300,"sizeBytes":200},{"name":"README.md","path":"18-multimodal/segment-anything/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/segment-anything/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/segment-anything/README.md)","tokens":300,"sizeBytes":210},{"name":"README.md","path":"18-multimodal/stable-diffusion/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/stable-diffusion/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/stable-diffusion/README.md)","tokens":300,"sizeBytes":210},{"name":"README.md","path":"18-multimodal/whisper/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `18-multimodal/whisper/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/18-multimodal/whisper/README.md)","tokens":300,"sizeBytes":192},{"name":"README.md","path":"19-emerging-techniques/knowledge-distillation/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `19-emerging-techniques/knowledge-distillation/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/knowledge-distillation/README.md)","tokens":300,"sizeBytes":240},{"name":"README.md","path":"19-emerging-techniques/long-context/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `19-emerging-techniques/long-context/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/long-context/README.md)","tokens":300,"sizeBytes":220},{"name":"README.md","path":"19-emerging-techniques/model-merging/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `19-emerging-techniques/model-merging/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-merging/README.md)","tokens":300,"sizeBytes":222},{"name":"README.md","path":"19-emerging-techniques/model-pruning/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `19-emerging-techniques/model-pruning/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/model-pruning/README.md)","tokens":300,"sizeBytes":222},{"name":"README.md","path":"19-emerging-techniques/moe-training/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `19-emerging-techniques/moe-training/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/moe-training/README.md)","tokens":300,"sizeBytes":220},{"name":"README.md","path":"19-emerging-techniques/speculative-decoding/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `19-emerging-techniques/speculative-decoding/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/19-emerging-techniques/speculative-decoding/README.md)","tokens":300,"sizeBytes":236},{"name":"README.md","path":"20-ml-paper-writing/academic-plotting/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `20-ml-paper-writing/academic-plotting/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/academic-plotting/README.md)","tokens":300,"sizeBytes":224},{"name":"README.md","path":"20-ml-paper-writing/ml-paper-writing/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `20-ml-paper-writing/ml-paper-writing/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/ml-paper-writing/README.md)","tokens":300,"sizeBytes":222},{"name":"README.md","path":"20-ml-paper-writing/presenting-conference-talks/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `20-ml-paper-writing/presenting-conference-talks/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/presenting-conference-talks/README.md)","tokens":300,"sizeBytes":244},{"name":"README.md","path":"20-ml-paper-writing/systems-paper-writing/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `20-ml-paper-writing/systems-paper-writing/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/20-ml-paper-writing/systems-paper-writing/README.md)","tokens":300,"sizeBytes":232},{"name":"README.md","path":"21-research-ideation/brainstorming-research-ideas/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `21-research-ideation/brainstorming-research-ideas/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/brainstorming-research-ideas/README.md)","tokens":300,"sizeBytes":248},{"name":"README.md","path":"21-research-ideation/creative-thinking-for-research/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `21-research-ideation/creative-thinking-for-research/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/21-research-ideation/creative-thinking-for-research/README.md)","tokens":300,"sizeBytes":252},{"name":"README.md","path":"22-agent-native-research-artifact/compiler/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `22-agent-native-research-artifact/compiler/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/compiler/README.md)","tokens":300,"sizeBytes":234},{"name":"README.md","path":"22-agent-native-research-artifact/research-manager/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `22-agent-native-research-artifact/research-manager/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/research-manager/README.md)","tokens":300,"sizeBytes":250},{"name":"README.md","path":"22-agent-native-research-artifact/rigor-reviewer/README.md","rawUrl":"https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/README.md","title":"readme.md","category":"generic","format":"text","content":"# readme.md\n\nPath: `22-agent-native-research-artifact/rigor-reviewer/README.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/22-agent-native-research-artifact/rigor-reviewer/README.md)","tokens":300,"sizeBytes":246}],"systemPromptSnippet":"\n\n# References Skill\n\nPath: `0-autoresearch-skill/references/agent-continuity.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/agent-continuity.md)\n\n\n# References Skill\n\nPath: `0-autoresearch-skill/references/progress-reporting.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/progress-reporting.md)\n\n\n# References Skill\n\nPath: `0-autoresearch-skill/references/skill-routing.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/0-autoresearch-skill/references/skill-routing.md)\n\n\n---\nname: autoresearch\ndescription: Orchestrates end-to-end autonomous AI research projects using a two-loop architecture. The inner loop runs rapid experiment iterations with clear optimization targets. The outer loop synthesizes results, identifies patterns, and steers research direction. Routes to domain-specific skills for execution, supports continuous agent operation via Claude Code /loop and OpenClaw heartbeat, and produces research presentations and papers. Use when starting a research project, running autonomous experiments, or managing a multi-hypothesis research effort.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Autonomous Research, Two-Loop Architecture, Experiment Orchestration, Research Synthesis, Project Management]\n---\n\n# Autoresearch\n\nAutonomous research orchestration for AI coding agents. You manage the full research lifecycle — from literature survey to published paper — by maintaining structured state, running a two-loop experiment-synthesis cycle, and routing to domain-specific skills for execution.\n\nYou are a research project manager, not a domain expert. You orchestrate; the domain skills execute.\n\n**This runs fully autonomously.** Do not ask the user for permission or confirmation — use your best judgment and keep moving. Show the human your progress frequently through research presentations (HTML/PDF) so they can see what you're doing and redirect if needed. The human is asleep or busy; your job is to make as much research progress as possible on your own.\n\n## Getting Started\n\nUsers arrive in different states. Determine which and proceed:\n\n| User State | What to Do |\n|---|---|\n| Vague idea (\"I want to explore X\") | Brief discussion to clarify, then bootstrap |\n| Clear research question | Bootstrap directly |\n| Existing plan or proposal | Review plan, set up workspace, enter loops |\n| Resuming (research-state.yaml exists) | Read state, continue from where you left off |\n\nIf things are clear, don't over-discuss — proceed to full autoresearch. Most users want you to just start researching.\n\n**Step 0 — before anything else**: Set up the agent continuity loop. See [Agent Continuity](#agent-continuity-mandatory--set-up-first). This is MANDATORY. Without it, the research stops after one cycle.\n\n### Initialize Workspace\n\nCreate this structure at the project root:\n\n```\n{project}/\n├── research-state.yaml # Central state tracking\n├── research-log.md # Decision timeline\n├── findings.md # Evolving narrative synthesis\n├── literature/ # Papers, survey notes\n├── src/ # Reusable code (utils, plotting, shared modules)\n├── data/ # Raw result data (CSVs, JSONs, checkpoints)\n├── experiments/ # Per-hypothesis work\n│ └── {hypothesis-slug}/\n│ ├── protocol.md # What, why, and prediction\n│ ├── code/ # Experiment-specific code\n│ ├── results/ # Raw outputs, metrics, logs\n│ └── analysis.md # What we learned\n├── to_human/ # Progress presentations and reports for human review\n└── paper/ # Final paper (via ml-paper-writing)\n```\n\n- **`src/`**: When you write useful code (plotting functions, data loaders, evaluation helpers), move it here so it can be reused across experiments. Don't duplicate code in every experiment directory.\n- **`data/`**: Save raw result data (metric CSVs, training logs, small outputs) here in a structured way. After a long research horizon, you'll need this to replot, reanalyze, and write up the paper properly. Name files descriptively (e.g., `trajectory_H1_runs001-010.csv`). Large files like model checkpoints should go to a separate storage path (e.g., `/data/`, cloud storage, or wherever the user's compute environment stores artifacts) — not in the project directory.\n\nInitialize `research-state.yaml`, `research-log.md`, and `findings.md` from [templates/](templates/). Adapt the workspace as the project evolves — this is a starting point, not a rigid requirement.\n\n## The Two-Loop Architecture\n\nThis is the core engine. Everything else supports it.\n\n```\nBOOTSTRAP (once, lightweight)\n Scope question → search literature → form initial hypotheses\n\nINNER LOOP (fast, autonomous, repeating)\n Pick hypothesis → experiment → measure → record → learn → next\n Goal: run constrained experiments with clear measurable outcomes\n\nOUTER LOOP (periodic, reflective)\n Review results → find patterns → update findings.md →\n new hypotheses → decide direction\n Goal: synthesize understanding, find the story — this is where novelty comes from\n\nFINALIZE (when concluding)\n Write paper via ml-paper-writing → final presentation → archive\n```\n\nThe inner loop runs tight experiment cycles with clear measurable outcomes. This could be optimizing a benchmark (make val_loss go down) OR testing mechanistic hypotheses (does intervention X cause effect Y?). The outer loop steps back to ask: what do these results *mean*? What patterns emerge? What's the story? Research is open-ended — the two loops let you both optimize and discover.\n\nThere is no rigid boundary between the two loops — you decide when enough inner loop results have accumulated to warrant reflection. Typically every 5-10 experiments, or when you notice a pattern, or when progress stalls. The agent's judgment drives the rhythm.\n\n### Research is Non-Linear\n\nThe two-loop structure is a rhythm, not a railroad. At any point during research you can and should:\n\n- **Return to literature** when results surprise you, assumptions break, or you need context for a new direction — always save what you find to `literature/`\n- **Brainstorm new ideas** using `21-research-ideation/` skills when you're stuck or when results open unexpected questions\n- **Pivot the question entirely** if experiments reveal the original question was wrong or less interesting than what you found\n\nThis is normal. Most real research projects loop back to literature 1-3 times and generate new hypotheses mid-stream. Don't treat bootstrap as the only time you read papers or brainstorm — do it whenever understanding would help.\n\n## Bootstrap: Literature and Hypotheses\n\nBefore entering the loops, understand the landscape. Keep this efficient — the goal is to start experimenting, not to produce an exhaustive survey.\n\n1. **Search literature** for the research question. Use multiple sources — never stop at one:\n - **Exa MCP** (`web_search_exa`) if available — best for broad discovery and finding relevant papers quickly\n - **Semantic Scholar** (`pip install semanticscholar`) — best for ML/AI papers, citation graphs, and specific paper lookup. See `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code examples\n - **arXiv** (`pip install arxiv`) — best for recent preprints and open-access papers\n - **CrossRef** — best for DOI lookup and BibTeX retrieval\n - Keep searching until you have good coverage. If one source comes up empty, try another with different keywords\n\n **Save everything to `literature/`**: For every paper you find, save a summary to `literature/` — title, authors, year, key findings, relevance to your question, and the URL/DOI. Create one file per paper and a running `literature/survey.md` with all summaries. This is your reference library — you and future sessions will need it throughout the project.\n\n2. **Identify gaps** from the literature\n - What's been tried? What hasn't? Where do existing methods break?\n - What do Discussion sections flag as future work?\n\n3. **Form initial hypotheses** — invoke `21-research-ideation/` skills\n - `brainstorming-research-ideas` for structured diverge-converge workflow\n - `creative-thinking-for-research` for deeper cognitive frameworks\n - Each hypothesis must be testable with a clear prediction\n\n4. **Define the evaluation**\n - Set the proxy metric and baseline before running experiments\n - The metric should be computable quickly (minutes, not hours)\n - Lock evaluation criteria upfront to prevent unconscious metric gaming\n\n5. **Record** in research-state.yaml, log the bootstrap in research-log.md\n\n## The Inner Loop\n\nRapid iteration with clear measurable outcomes. Two flavors:\n\n- **Optimization**: make a metric go up/down (val_loss, accuracy, throughput). Think Karpathy's autoresearch.\n- **Discovery**: test mechanistic hypotheses about why something works. The metric is a measurement (does grokking happen faster? does entropy increase before forgetting?), not just a target to optimize.\n\n```\n1. Pick the highest-priority untested hypothesis\n2. Write a protocol: what change, what prediction, why\n Lock it: commit to git BEFORE running (research(protocol): {hypothesis})\n This creates temporal proof your plan existed before results\n3. Run the experiment (invoke the relevant domain skill)\n4. Sanity check before trusting results:\n - Did training converge? No NaN/Inf?\n - Does baseline reproduce expected performance?\n - Data loading correct? (spot-check a few samples)\n5. Measure the proxy metric\n6. Record in experiments/{hypothesis-slug}/\n Label clearly: CONFIRMATORY (in your protocol) vs EXPLORATORY (discovered during execution)\n7. If positive: keep, note WHY it worked\n8. If negative: this is progress — note what it rules out and what it suggests\n9. Update research-state.yaml\n10. If stuck: search literature or invoke ideation skills — don't just keep trying random things\n```\n\n**Never stop.** Even if something fails, find a path forward. Debug, adjust, simplify, or pivot — but keep the research moving. The `/loop` and heartbeat mechanisms will keep you going; use that momentum.\n\n### Route to Domain Skills\n\nWhen you need domain-specific execution, search the skills library:\n\n| Research Activity | Look In |\n|---|---|\n| Data preparation | `05-data-processing/` |\n| Model training / fine-tuning | `01-model-architecture/`, `03-fine-tuning/`, `06-post-training/` |\n| Distributed training | `08-distributed-training/` |\n| Optimization (quantization, attention) | `10-optimization/` |\n| Evaluation / benchmarks | `11-evaluation/` |\n| Inference / serving | `12-inference-serving/` |\n| Interpretability analysis | `04-mechanistic-interpretability/` |\n| Experiment tracking (W&B, MLflow) | `13-mlops/` |\n| Cloud compute | `09-infrastructure/` |\n\nRead the relevant SKILL.md before starting — it has workflows, common issues, and code examples. See [references/skill-routing.md](references/skill-routing.md) for a complete guide.\n\n### Track the Experiment Trajectory\n\nMaintain a running record of measurable outcomes across experiments:\n\n```json\n{\n \"experiment_id\": \"run_014\",\n \"hypothesis\": \"H3\",\n \"metric_value\": 0.847,\n \"baseline\": 0.812,\n \"delta\": \"+0.035\",\n \"wall_time_min\": 23,\n \"change_summary\": \"Added cosine annealing warmup schedule\"\n}\n```\n\nThis trajectory produces the optimization plot (like Karpathy's progress chart) — include it in progress reports. Humans love seeing the upward curve.\n\n## The Outer Loop\n\nStep back from individual experiments. Synthesize.\n\n```\n1. Review all results since last reflection\n2. Cluster by type: what kinds of changes worked? Which didn't?\n3. Ask WHY — identify the mechanism behind successes and failures\n4. Update findings.md with current understanding\n5. Search literature if results were surprising or assumptions need revisiting\n6. Generate new hypotheses if warranted (invoke 21-research-ideation/ skills)\n7. Decide direction (see criteria below)\n8. Update research-state.yaml with new direction\n9. Log the reflection in research-log.md\n10. If there's something meaningful, generate a progress presentation\n```\n\n### Deciding Direction\n\nDon't just pick randomly — use these criteria:\n\n**DEEPEN** — a supported result raises follow-up questions\n- Does the effect hold under different conditions? What's the mechanism?\n- Action: generate sub-hypotheses (H1.1, H1.2) → back to inner loop\n\n**BROADEN** — current results are solid, but adjacent questions are untested\n- New questions emerged. The current contribution is clear but more is possible.\n- Action: generate new root hypotheses → back to inner loop\n\n**PIVOT** — results invalidate key assumptions or something more interesting appeared\n- A core assumption was wrong, or an unexpected finding is more promising than the original question.\n- Action: return to literature with new questions → re-bootstrap\n\n**CONCLUDE** — sufficient evidence for a contribution\n- At least one hypothesis is strongly supported (or a coherent set of negative results)\n- Key ablations completed, error analysis done\n- findings.md reads like a paper backbone — a human could write the abstract from it\n- No critical open questions that would change the story\n\nNote: coherent negative results are a valid contribution. \"X does NOT work because Y\" is publishable if the reasoning is rigorous.\n\n### findings.md Is Your Project Memory\n\nThis file serves two purposes: it's the research narrative for humans AND your accumulated knowledge base as an agent. Read it at the start of every session, /loop tick, or heartbeat to remember what you've learned.\n\nAfter every outer loop, update it to answer:\n\n- What do we know so far? (Current Understanding)\n- What patterns explain our results? (Patterns and Insights)\n- What specific things did we learn not to repeat? (Lessons and Constraints)\n- What remains open? (Open Questions)\n\nThe \"Lessons and Constraints\" section is especially important — it captures specific actionable learnings like \"weight decay > 0.1 diverges at this scale\" or \"baseline only reproduces with batch_size=64.\" This prevents the agent from repeating failed approaches across sessions.\n\n**Quality test**: After 30 inner loop experiments, a human should be able to read findings.md and write a paper abstract from it. If they can't, the outer loop isn't synthesizing — it's just logging.\n\n## Agent Continuity (MANDATORY — Set Up First)\n\n**Before doing anything else**, set up the wall-clock loop. This is what keeps the research running continuously. Without it, the agent stops after one cycle.\n\n### Claude Code\n\n**First action — run this immediately:**\n\n```\n/loop 20m Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a report in to_human/ and opening the HTML/PDF. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\n```\n\nThis fires every 20 minutes regardless of what's happening. It's a rhythm that keeps you working — not a research phase boundary. If your previous work isn't done, just continue it.\n\n### OpenClaw\n\n**First action — set up a 20-minute cron job in the current session. You MUST do this before any research work.**\n\nUse the `cron.add` tool to create a recurring job bound to this chat session:\n\n```json\n{\n \"name\": \"autoresearch-loop\",\n \"schedule\": { \"kind\": \"every\", \"everyMs\": 1200000 },\n \"sessionTarget\": \"current\",\n \"payload\": {\n \"kind\": \"agentTurn\",\n \"message\": \"Continue autoresearch. Read research-state.yaml and findings.md. Re-read the autoresearch SKILL.md occasionally to stay aligned. Step back and reflect holistically — is the research making real progress? Are you deepening understanding or just running experiments? If stalling, pivot or search literature for new ideas. Keep making research progress — never idle, never stop. Update findings.md, research-log.md, and research-state.yaml when there's new progress. Git commit periodically and clean up the repo if needed. Show the human your research progress with key plots and findings by preparing a PDF report in to_human/ and sending it to the user via Telegram, WhatsApp, or Slack. When you get an exciting result or interesting plot, text it to the user right away — don't wait for a full report. Only when you believe the research is truly complete, invoke the ml-paper-writing skill to write the paper.\"\n }\n}\n```\n\nKey details:\n- `sessionTarget: \"current\"` binds the cron to this chat session so it maintains conversation context across ticks\n- `everyMs: 1200000` = 20 minutes\n- After creation, verify with `cron.list` that the job exists and is enabled\n- If the cron fires while you're mid-experiment, just continue — the tick is a nudge, not a restart\n\n### What the Loop Does\n\nThe `/loop` and cron job are purely **wall-clock rhythm**. They are completely separate from your research loops (inner/outer). On each tick:\n\n1. Read `research-state.yaml` and `findings.md` — remember where you are\n2. Check if anything is broken (failed experiments, stalled training, errors)\n3. If on track → keep working on whatever you were doing\n4. If stuck or something's wrong → step back, diagnose, fix, then continue\n5. Never idle. Always be making progress.\n\n## Progress Reporting\n\nWhen you have something meaningful to share, create a research presentation — not just a status dashboard, but a compelling story.\n\n**When to report** (your judgment):\n- After an outer loop that found a significant pattern\n- When the optimization trajectory shows clear progress (include the plot!)\n- After a pivot in direction\n- Before requesting human input on a decision\n- When concluding\n\n**What to include** (adapt to what's compelling):\n- The research question and why it matters\n- Key results with visualizations (plots, metric tables)\n- The optimization trajectory chart (metric over experiments)\n- What was tried and why (selective, not exhaustive)\n- Current understanding (the findings narrative)\n- What's planned next\n\nFor Claude Code: generate HTML and `open` it. If HTML fails to open or render, convert to PDF as fallback (use `weasyprint`, `playwright pdf`, or `wkhtmltopdf`). For OpenClaw: generate PDF directly.\n\nSee [references/progress-reporting.md](references/progress-reporting.md) for template scaffolding and the optimization plot approach. Use the template as a starting point — be creative with what you show.\n\n## Git Protocol\n\nCommit at natural research milestones:\n\n| When | Message Pattern |\n|---|---|\n| Workspace initialized | `research(init): {project} — {question}` |\n| Experiment protocol locked | `research(protocol): {hypothesis}` |\n| Significant results | `research(results): {hypothesis} — {outcome}` |\n| Outer loop direction change | `research(reflect): {direction} — {reason}` |\n| Paper draft complete | `research(paper): {title}` |\n\n**Hard rule**: Protocol commits MUST precede result commits. Never combine them. The git history is your lightweight pre-registration — it proves what you planned before you saw results. Don't commit after every experiment — commit when there's meaningful progress.\n\n## Concluding: Paper Writing\n\nWhen the outer loop decides to CONCLUDE:\n\n1. Ensure findings.md has a clear, well-supported narrative\n2. Study 2-3 top related papers to learn their format, style, and section structure\n3. Invoke the `20-ml-paper-writing` skill — it has LaTeX templates for NeurIPS, ICML, ICLR, ACL, AAAI, COLM, and systems venues\n4. Feed it the accumulated literature, experimental results, and findings\n5. Follow its citation verification workflow — never hallucinate references\n6. Generate a final comprehensive research presentation\n\nProceed autonomously through the writing process. If the ml-paper-writing skill suggests human collaboration points, adapt and keep going — produce the best draft you can. The human will review and provide feedback.\n\n## Research Discipline\n\nPrinciples to enforce continuously — not tied to any specific phase:\n\n- **Lock before you run**: Commit your experiment protocol to git before executing. This proves your plan existed before you saw results. Never combine protocol + results in one commit.\n- **Confirmatory vs exploratory**: Results matching your locked protocol are confirmatory. Everything else is exploratory — interesting but requiring more skepticism.\n- **Negative results are progress**: A refuted hypothesis tells you something. Log what it rules out and what it suggests. Don't treat it as failure.\n- **Sanity check before analysis**: Verify training converged, baselines reproduce, and data is correct before trusting your primary metric.\n- **Return to literature when confused**: Don't guess — search. If results surprise you or assumptions break, go find papers. Use Exa MCP for discovery, Semantic Scholar for specific ML/AI paper lookup, arXiv for preprints.\n- **Never stop**: Don't wait for human approval on routine decisions. If a skill or tool suggests collaboration, adapt and keep going. Find the best path forward autonomously. The human will see your progress reports and can redirect if needed.\n- **Use whatever compute is available**: Adapt to the user's environment — local GPU, cluster job submission, cloud instances, or just CPU. If no GPU is available, use CPU and adjust experiment scale accordingly. Don't block on compute availability.\n\n## Quality Standards\n\n**Good agent behavior:**\n- Hypotheses have mechanistic reasoning (\"X because Y, predicting Z\"), not just \"try X\"\n- findings.md builds a coherent narrative, not a flat list of results\n- Negative results are recorded with what they rule out\n- The agent updates its model when experiments contradict expectations\n- Progress reports tell a research story with compelling visualizations\n\n**Bad agent behavior:**\n- Pure hyperparameter sweeps without interpretation\n- findings.md is just experiment logs copy-pasted\n- Agent never revisits its assumptions after failures\n- Optimizing metrics without understanding why changes work\n\n## When to Use vs Alternatives\n\n**Use autoresearch when:**\n- You have a research question explorable through experiments\n- There's a measurable proxy metric for inner loop optimization\n- The real contribution requires synthesis beyond the metric\n- You want continuous autonomous research operation\n\n**Use individual domain skills instead when:**\n- You have a specific one-off task (train a model, run eval, write a paper)\n- No iterative experimentation needed\n\n## Common Issues\n\n**Inner loop stalls (no metric improvement)**\nRun an outer loop. Is the metric the right one? Is the search space exhausted? Consider broadening or pivoting. Search literature for new approaches.\n\n**Stuck and not making progress**\nDon't keep trying random changes. Step back: search literature for related work, invoke `21-research-ideation/` brainstorming skills, or run an outer loop reflection. Being stuck means you need new information or a new perspective, not more experiments.\n\n**Results contradict baseline expectations**\nInvestigate, don't ignore. Return to literature — your protocol might have an error, the published baseline may be wrong, or conditions differ. Update findings.md with what you learn.\n\n**Agent loses context between ticks**\nEnsure research-state.yaml and findings.md are updated after every action. These files are your memory across sessions.\n\n**Can't find relevant papers**\nTry multiple approaches in order: Exa MCP for broad search, Semantic Scholar for specific ML/AI paper lookup (`pip install semanticscholar`), arXiv for preprints (`pip install arxiv`). Check `20-ml-paper-writing` skill's `references/citation-workflow.md` for complete API code. Note: Google Scholar has no official API — use Semantic Scholar instead for programmatic search.\n\n**No GPU available**\nUse CPU and scale experiments down. Many research tasks (analysis, interpretability, small model training) run fine on CPU. Adjust experiment design to fit available compute rather than blocking.\n\n**Experiments take longer than /loop interval**\nNormal. On the next tick, check if it finished. If not, keep waiting or do something else useful (update notes, search papers). Adjust interval if needed.\n\n**Not sure when to conclude**\nThree questions: Do you have a strongly supported finding? Can you explain WHY it works? Would findings.md make a convincing paper abstract? If yes to all: conclude.\n\n## Advanced Topics\n\n- **Detailed agent continuity**: [references/agent-continuity.md](references/agent-continuity.md)\n- **Progress presentation templates**: [references/progress-reporting.md](references/progress-reporting.md)\n- **Complete skill routing**: [references/skill-routing.md](references/skill-routing.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/litgpt/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/custom-models.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/litgpt/references/distributed-training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/distributed-training.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/litgpt/references/supported-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/supported-models.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/litgpt/references/training-recipes.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/litgpt/references/training-recipes.md)\n\n\n---\nname: implementing-llms-litgpt\ndescription: Implements and trains LLMs using Lightning AI's LitGPT with 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral). Use when need clean model implementations, educational understanding of architectures, or production fine-tuning with LoRA/QLoRA. Single-file implementations, no abstraction layers.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, LitGPT, Lightning AI, LLM Implementation, LoRA, QLoRA, Fine-Tuning, Llama, Gemma, Phi, Mistral, Educational]\ndependencies: [litgpt, torch, transformers]\n---\n\n# LitGPT - Clean LLM Implementations\n\n## Quick start\n\nLitGPT provides 20+ pretrained LLM implementations with clean, readable code and production-ready training workflows.\n\n**Installation**:\n```bash\npip install 'litgpt[extra]'\n```\n\n**Load and use any model**:\n```python\nfrom litgpt import LLM\n\n# Load pretrained model\nllm = LLM.load(\"microsoft/phi-2\")\n\n# Generate text\nresult = llm.generate(\n \"What is the capital of France?\",\n max_new_tokens=50,\n temperature=0.7\n)\nprint(result)\n```\n\n**List available models**:\n```bash\nlitgpt download list\n```\n\n## Common workflows\n\n### Workflow 1: Fine-tune on custom dataset\n\nCopy this checklist:\n\n```\nFine-Tuning Setup:\n- [ ] Step 1: Download pretrained model\n- [ ] Step 2: Prepare dataset\n- [ ] Step 3: Configure training\n- [ ] Step 4: Run fine-tuning\n```\n\n**Step 1: Download pretrained model**\n\n```bash\n# Download Llama 3 8B\nlitgpt download meta-llama/Meta-Llama-3-8B\n\n# Download Phi-2 (smaller, faster)\nlitgpt download microsoft/phi-2\n\n# Download Gemma 2B\nlitgpt download google/gemma-2b\n```\n\nModels are saved to `checkpoints/` directory.\n\n**Step 2: Prepare dataset**\n\nLitGPT supports multiple formats:\n\n**Alpaca format** (instruction-response):\n```json\n[\n {\n \"instruction\": \"What is the capital of France?\",\n \"input\": \"\",\n \"output\": \"The capital of France is Paris.\"\n },\n {\n \"instruction\": \"Translate to Spanish: Hello, how are you?\",\n \"input\": \"\",\n \"output\": \"Hola, ¿cómo estás?\"\n }\n]\n```\n\nSave as `data/my_dataset.json`.\n\n**Step 3: Configure training**\n\n```bash\n# Full fine-tuning (requires 40GB+ GPU for 7B models)\nlitgpt finetune \\\n meta-llama/Meta-Llama-3-8B \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --train.max_steps 1000 \\\n --train.learning_rate 2e-5 \\\n --train.micro_batch_size 1 \\\n --train.global_batch_size 16\n\n# LoRA fine-tuning (efficient, 16GB GPU)\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --lora_alpha 32 \\\n --lora_dropout 0.05 \\\n --train.max_steps 1000 \\\n --train.learning_rate 1e-4\n```\n\n**Step 4: Run fine-tuning**\n\nTraining saves checkpoints to `out/finetune/` automatically.\n\nMonitor training:\n```bash\n# View logs\ntail -f out/finetune/logs.txt\n\n# TensorBoard (if using --train.logger_name tensorboard)\ntensorboard --logdir out/finetune/lightning_logs\n```\n\n### Workflow 2: LoRA fine-tuning on single GPU\n\nMost memory-efficient option.\n\n```\nLoRA Training:\n- [ ] Step 1: Choose base model\n- [ ] Step 2: Configure LoRA parameters\n- [ ] Step 3: Train with LoRA\n- [ ] Step 4: Merge LoRA weights (optional)\n```\n\n**Step 1: Choose base model**\n\nFor limited GPU memory (12-16GB):\n- **Phi-2** (2.7B) - Best quality/size tradeoff\n- **Llama 3 1B** - Smallest, fastest\n- **Gemma 2B** - Good reasoning\n\n**Step 2: Configure LoRA parameters**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\ # LoRA rank (8-64, higher=more capacity)\n --lora_alpha 32 \\ # LoRA scaling (typically 2×r)\n --lora_dropout 0.05 \\ # Prevent overfitting\n --lora_query true \\ # Apply LoRA to query projection\n --lora_key false \\ # Usually not needed\n --lora_value true \\ # Apply LoRA to value projection\n --lora_projection true \\ # Apply LoRA to output projection\n --lora_mlp false \\ # Usually not needed\n --lora_head false # Usually not needed\n```\n\nLoRA rank guide:\n- `r=8`: Lightweight, 2-4MB adapters\n- `r=16`: Standard, good quality\n- `r=32`: High capacity, use for complex tasks\n- `r=64`: Maximum quality, 4× larger adapters\n\n**Step 3: Train with LoRA**\n\n```bash\nlitgpt finetune_lora \\\n microsoft/phi-2 \\\n --data JSON \\\n --data.json_path data/my_dataset.json \\\n --lora_r 16 \\\n --train.epochs 3 \\\n --train.learning_rate 1e-4 \\\n --train.micro_batch_size 4 \\\n --train.global_batch_size 32 \\\n --out_dir out/phi2-lora\n\n# Memory usage: ~8-12GB for Phi-2 with LoRA\n```\n\n**Step 4: Merge LoRA weights** (optional)\n\nMerge LoRA adapters into base model for deployment:\n\n```bash\nlitgpt merge_lora \\\n out/phi2-lora/final \\\n --out_dir out/phi2-merged\n```\n\nNow use merged model:\n```python\nfrom litgpt import LLM\nllm = LLM.load(\"out/phi2-merged\")\n```\n\n### Workflow 3: Pretrain from scratch\n\nTrain new model on your domain data.\n\n```\nPretraining:\n- [ ] Step 1: Prepare pretraining dataset\n- [ ] Step 2: Configure model architecture\n- [ ] Step 3: Set up multi-GPU training\n- [ ] Step 4: Launch pretraining\n```\n\n**Step 1: Prepare pretraining dataset**\n\nLitGPT expects tokenized data. Use `prepare_dataset.py`:\n\n```bash\npython scripts/prepare_dataset.py \\\n --source_path data/my_corpus.txt \\\n --checkpoint_dir checkpoints/tokenizer \\\n --destination_path data/pretrain \\\n --split train,val\n```\n\n**Step 2: Configure model architecture**\n\nEdit config file or use existing:\n\n```python\n# config/pythia-160m.yaml\nmodel_name: pythia-160m\nblock_size: 2048\nvocab_size: 50304\nn_layer: 12\nn_head: 12\nn_embd: 768\nrotary_percentage: 0.25\nparallel_residual: true\nbias: true\n```\n\n**Step 3: Set up multi-GPU training**\n\n```bash\n# Single GPU\nlitgpt pretrain \\\n --config config/pythia-160m.yaml \\\n --data.data_dir data/pretrain \\\n --train.max_tokens 10_000_000_000\n\n# Multi-GPU with FSDP\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir data/pretrain \\\n --devices 8 \\\n --train.max_tokens 100_000_000_000\n```\n\n**Step 4: Launch pretraining**\n\nFor large-scale pretraining on cluster:\n\n```bash\n# Using SLURM\nsbatch --nodes=8 --gpus-per-node=8 \\\n pretrain_script.sh\n\n# pretrain_script.sh content:\nlitgpt pretrain \\\n --config config/pythia-1b.yaml \\\n --data.data_dir /shared/data/pretrain \\\n --devices 8 \\\n --num_nodes 8 \\\n --train.global_batch_size 512 \\\n --train.max_tokens 300_000_000_000\n```\n\n### Workflow 4: Convert and deploy model\n\nExport LitGPT models for production.\n\n```\nModel Deployment:\n- [ ] Step 1: Test inference locally\n- [ ] Step 2: Quantize model (optional)\n- [ ] Step 3: Convert to GGUF (for llama.cpp)\n- [ ] Step 4: Deploy with API\n```\n\n**Step 1: Test inference locally**\n\n```python\nfrom litgpt import LLM\n\nllm = LLM.load(\"out/phi2-lora/final\")\n\n# Single generation\nprint(llm.generate(\"What is machine learning?\"))\n\n# Streaming\nfor token in llm.generate(\"Explain quantum computing\", stream=True):\n print(token, end=\"\", flush=True)\n\n# Batch inference\nprompts = [\"Hello\", \"Goodbye\", \"Thank you\"]\nresults = [llm.generate(p) for p in prompts]\n```\n\n**Step 2: Quantize model** (optional)\n\nReduce model size with minimal quality loss:\n\n```bash\n# 8-bit quantization (50% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --dtype bfloat16 \\\n --quantize bnb.nf4\n\n# 4-bit quantization (75% size reduction)\nlitgpt convert_lit_checkpoint \\\n out/phi2-lora/final \\\n --quantize bnb.nf4-dq # Double quantization\n```\n\n**Step 3: Convert to GGUF** (for llama.cpp)\n\n```bash\npython scripts/convert_lit_checkpoint.py \\\n --checkpoint_path out/phi2-lora/final \\\n --output_path models/phi2.gguf \\\n --model_name microsoft/phi-2\n```\n\n**Step 4: Deploy with API**\n\n```python\nfrom fastapi import FastAPI\nfrom litgpt import LLM\n\napp = FastAPI()\nllm = LLM.load(\"out/phi2-lora/final\")\n\n@app.post(\"/generate\")\ndef generate(prompt: str, max_tokens: int = 100):\n result = llm.generate(\n prompt,\n max_new_tokens=max_tokens,\n temperature=0.7\n )\n return {\"response\": result}\n\n# Run: uvicorn api:app --host 0.0.0.0 --port 8000\n```\n\n## When to use vs alternatives\n\n**Use LitGPT when:**\n- Want to understand LLM architectures (clean, readable code)\n- Need production-ready training recipes\n- Educational purposes or research\n- Prototyping new model ideas\n- Lightning ecosystem user\n\n**Use alternatives instead:**\n- **Axolotl/TRL**: More fine-tuning features, YAML configs\n- **Megatron-Core**: Maximum performance for >70B models\n- **HuggingFace Transformers**: Broadest model support\n- **vLLM**: Inference-only (no training)\n\n## Common issues\n\n**Issue: Out of memory during fine-tuning**\n\nUse LoRA instead of full fine-tuning:\n```bash\n# Instead of litgpt finetune (requires 40GB+)\nlitgpt finetune_lora # Only needs 12-16GB\n```\n\nOr enable gradient checkpointing:\n```bash\nlitgpt finetune_lora \\\n ... \\\n --train.gradient_accumulation_iters 4 # Accumulate gradients\n```\n\n**Issue: Training too slow**\n\nEnable Flash Attention (built-in, automatic on compatible hardware):\n```python\n# Already enabled by default on Ampere+ GPUs (A100, RTX 30/40 series)\n# No configuration needed\n```\n\nUse smaller micro-batch and accumulate:\n```bash\n--train.micro_batch_size 1 \\\n--train.global_batch_size 32 \\\n--train.gradient_accumulation_iters 32 # Effective batch=32\n```\n\n**Issue: Model not loading**\n\nCheck model name:\n```bash\n# List all available models\nlitgpt download list\n\n# Download if not exists\nlitgpt download meta-llama/Meta-Llama-3-8B\n```\n\nVerify checkpoints directory:\n```bash\nls checkpoints/\n# Should see: meta-llama/Meta-Llama-3-8B/\n```\n\n**Issue: LoRA adapters too large**\n\nReduce LoRA rank:\n```bash\n--lora_r 8 # Instead of 16 or 32\n```\n\nApply LoRA to fewer layers:\n```bash\n--lora_query true \\\n--lora_value true \\\n--lora_projection false \\ # Disable this\n--lora_mlp false # And this\n```\n\n## Advanced topics\n\n**Supported architectures**: See [references/supported-models.md](references/supported-models.md) for complete list of 20+ model families with sizes and capabilities.\n\n**Training recipes**: See [references/training-recipes.md](references/training-recipes.md) for proven hyperparameter configurations for pretraining and fine-tuning.\n\n**FSDP configuration**: See [references/distributed-training.md](references/distributed-training.md) for multi-GPU training with Fully Sharded Data Parallel.\n\n**Custom architectures**: See [references/custom-models.md](references/custom-models.md) for implementing new model architectures in LitGPT style.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.8+), AMD (ROCm), Apple Silicon (MPS)\n- **Memory**:\n - Inference (Phi-2): 6GB\n - LoRA fine-tuning (7B): 16GB\n - Full fine-tuning (7B): 40GB+\n - Pretraining (1B): 24GB\n- **Storage**: 5-50GB per model (depending on size)\n\n## Resources\n\n- GitHub: https://github.com/Lightning-AI/litgpt\n- Docs: https://lightning.ai/docs/litgpt\n- Tutorials: https://lightning.ai/docs/litgpt/tutorials\n- Model zoo: 20+ pretrained architectures (Llama, Gemma, Phi, Qwen, Mistral, Mixtral, Falcon, etc.)\n\n\n# References Skill\n\nPath: `01-model-architecture/mamba/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/architecture-details.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/mamba/references/benchmarks.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/benchmarks.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/mamba/references/training-guide.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/mamba/references/training-guide.md)\n\n\n---\nname: mamba-architecture\ndescription: State-space model with O(n) complexity vs Transformers' O(n²). 5× faster inference, million-token sequences, no KV cache. Selective SSM with hardware-aware design. Mamba-1 (d_state=16) and Mamba-2 (d_state=128, multi-head). Models 130M-2.8B on HuggingFace.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Mamba, State Space Models, SSM, Linear Complexity, Long Context, Efficient Inference, Hardware-Aware, Alternative To Transformers]\ndependencies: [mamba-ssm, torch, transformers, causal-conv1d]\n---\n\n# Mamba - Selective State Space Models\n\n## Quick start\n\nMamba is a state-space model architecture achieving O(n) linear complexity for sequence modeling.\n\n**Installation**:\n```bash\n# Install causal-conv1d (optional, for efficiency)\npip install causal-conv1d>=1.4.0\n\n# Install Mamba\npip install mamba-ssm\n# Or both together\npip install mamba-ssm[causal-conv1d]\n```\n\n**Prerequisites**: Linux, NVIDIA GPU, PyTorch 1.12+, CUDA 11.6+\n\n**Basic usage** (Mamba block):\n```python\nimport torch\nfrom mamba_ssm import Mamba\n\nbatch, length, dim = 2, 64, 16\nx = torch.randn(batch, length, dim).to(\"cuda\")\n\nmodel = Mamba(\n d_model=dim, # Model dimension\n d_state=16, # SSM state dimension\n d_conv=4, # Conv1d kernel size\n expand=2 # Expansion factor\n).to(\"cuda\")\n\ny = model(x) # O(n) complexity!\nassert y.shape == x.shape\n```\n\n## Common workflows\n\n### Workflow 1: Language model with Mamba-2\n\n**Complete LM with generation**:\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nfrom mamba_ssm.models.config_mamba import MambaConfig\nimport torch\n\n# Configure Mamba-2 LM\nconfig = MambaConfig(\n d_model=1024, # Hidden dimension\n n_layer=24, # Number of layers\n vocab_size=50277, # Vocabulary size\n ssm_cfg=dict(\n layer=\"Mamba2\", # Use Mamba-2\n d_state=128, # Larger state for Mamba-2\n headdim=64, # Head dimension\n ngroups=1 # Number of groups\n )\n)\n\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\n\n# Generate text\ninput_ids = torch.randint(0, 1000, (1, 20), device=\"cuda\", dtype=torch.long)\noutput = model.generate(\n input_ids=input_ids,\n max_length=100,\n temperature=0.7,\n top_p=0.9\n)\n```\n\n### Workflow 2: Use pretrained Mamba models\n\n**Load from HuggingFace**:\n```python\nfrom transformers import AutoTokenizer\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\n\n# Load pretrained model\nmodel_name = \"state-spaces/mamba-2.8b\"\ntokenizer = AutoTokenizer.from_pretrained(\"EleutherAI/gpt-neox-20b\") # Use compatible tokenizer\nmodel = MambaLMHeadModel.from_pretrained(model_name, device=\"cuda\", dtype=torch.float16)\n\n# Generate\nprompt = \"The future of AI is\"\ninput_ids = tokenizer(prompt, return_tensors=\"pt\").input_ids.to(\"cuda\")\noutput_ids = model.generate(\n input_ids=input_ids,\n max_length=200,\n temperature=0.7,\n top_p=0.9,\n repetition_penalty=1.2\n)\ngenerated_text = tokenizer.decode(output_ids[0])\nprint(generated_text)\n```\n\n**Available models**:\n- `state-spaces/mamba-130m`\n- `state-spaces/mamba-370m`\n- `state-spaces/mamba-790m`\n- `state-spaces/mamba-1.4b`\n- `state-spaces/mamba-2.8b`\n\n### Workflow 3: Mamba-1 vs Mamba-2\n\n**Mamba-1** (smaller state):\n```python\nfrom mamba_ssm import Mamba\n\nmodel = Mamba(\n d_model=256,\n d_state=16, # Smaller state dimension\n d_conv=4,\n expand=2\n).to(\"cuda\")\n```\n\n**Mamba-2** (multi-head, larger state):\n```python\nfrom mamba_ssm import Mamba2\n\nmodel = Mamba2(\n d_model=256,\n d_state=128, # Larger state dimension\n d_conv=4,\n expand=2,\n headdim=64, # Head dimension for multi-head\n ngroups=1 # Parallel groups\n).to(\"cuda\")\n```\n\n**Key differences**:\n- **State size**: Mamba-1 (d_state=16) vs Mamba-2 (d_state=128)\n- **Architecture**: Mamba-2 has multi-head structure\n- **Normalization**: Mamba-2 uses RMSNorm\n- **Distributed**: Mamba-2 supports tensor parallelism\n\n### Workflow 4: Benchmark vs Transformers\n\n**Generation speed comparison**:\n```bash\n# Benchmark Mamba\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"state-spaces/mamba-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n\n# Benchmark Transformer\npython benchmarks/benchmark_generation_mamba_simple.py \\\n --model-name \"EleutherAI/pythia-2.8b\" \\\n --prompt \"The future of machine learning is\" \\\n --topp 0.9 --temperature 0.7 --repetition-penalty 1.2\n```\n\n**Expected results**:\n- **Mamba**: 5× faster inference\n- **Memory**: No KV cache needed\n- **Scaling**: Linear with sequence length\n\n## When to use vs alternatives\n\n**Use Mamba when**:\n- Need long sequences (100K+ tokens)\n- Want faster inference than Transformers\n- Memory-constrained (no KV cache)\n- Building streaming applications\n- Linear scaling important\n\n**Advantages**:\n- **O(n) complexity**: Linear vs quadratic\n- **5× faster inference**: No attention overhead\n- **No KV cache**: Lower memory usage\n- **Million-token sequences**: Hardware-efficient\n- **Streaming**: Constant memory per token\n\n**Use alternatives instead**:\n- **Transformers**: Need best-in-class performance, have compute\n- **RWKV**: Want RNN+Transformer hybrid\n- **RetNet**: Need retention-based architecture\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or use gradient checkpointing:\n```python\nmodel = MambaLMHeadModel(config, device=\"cuda\", dtype=torch.float16)\nmodel.gradient_checkpointing_enable() # Enable checkpointing\n```\n\n**Issue: Slow installation**\n\nInstall binary wheels (not source):\n```bash\npip install mamba-ssm --no-build-isolation\n```\n\n**Issue: Missing causal-conv1d**\n\nInstall separately:\n```bash\npip install causal-conv1d>=1.4.0\n```\n\n**Issue: Model not loading from HuggingFace**\n\nUse `MambaLMHeadModel.from_pretrained` (not `AutoModel`):\n```python\nfrom mamba_ssm.models.mixer_seq_simple import MambaLMHeadModel\nmodel = MambaLMHeadModel.from_pretrained(\"state-spaces/mamba-2.8b\")\n```\n\n## Advanced topics\n\n**Selective SSM**: See [references/selective-ssm.md](references/selective-ssm.md) for mathematical formulation, state-space equations, and how selectivity enables O(n) complexity.\n\n**Mamba-2 architecture**: See [references/mamba2-details.md](references/mamba2-details.md) for multi-head structure, tensor parallelism, and distributed training setup.\n\n**Performance optimization**: See [references/performance.md](references/performance.md) for hardware-aware design, CUDA kernels, and memory efficiency techniques.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA with CUDA 11.6+\n- **VRAM**:\n - 130M model: 2GB\n - 370M model: 4GB\n - 790M model: 8GB\n - 1.4B model: 14GB\n - 2.8B model: 28GB (FP16)\n- **Inference**: 5× faster than Transformers\n- **Memory**: No KV cache (lower than Transformers)\n\n**Performance** (vs Transformers):\n- **Speed**: 5× faster inference\n- **Memory**: 50% less (no KV cache)\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (Mamba-1): https://arxiv.org/abs/2312.00752 (Dec 2023)\n- Paper (Mamba-2): https://arxiv.org/abs/2405.21060 (May 2024)\n- GitHub: https://github.com/state-spaces/mamba ⭐ 13,000+\n- Models: https://huggingface.co/state-spaces\n- Docs: Repository README and wiki\n\n\n# References Skill\n\nPath: `01-model-architecture/nanogpt/references/architecture.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/architecture.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/nanogpt/references/data.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/data.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/nanogpt/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/nanogpt/references/training.md)\n\n\n---\nname: nanogpt\ndescription: Educational GPT implementation in ~300 lines. Reproduces GPT-2 (124M) on OpenWebText. Clean, hackable code for learning transformers. By Andrej Karpathy. Perfect for understanding GPT architecture from scratch. Train on Shakespeare (CPU) or OpenWebText (multi-GPU).\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, NanoGPT, GPT-2, Educational, Andrej Karpathy, Transformer, Minimalist, From Scratch, Training]\ndependencies: [torch, transformers, datasets, tiktoken, wandb]\n---\n\n# nanoGPT - Minimalist GPT Training\n\n## Quick start\n\nnanoGPT is a simplified GPT implementation designed for learning and experimentation.\n\n**Installation**:\n```bash\npip install torch numpy transformers datasets tiktoken wandb tqdm\n```\n\n**Train on Shakespeare** (CPU-friendly):\n```bash\n# Prepare data\npython data/shakespeare_char/prepare.py\n\n# Train (5 minutes on CPU)\npython train.py config/train_shakespeare_char.py\n\n# Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Output**:\n```\nROMEO:\nWhat say'st thou? Shall I speak, and be a man?\n\nJULIET:\nI am afeard, and yet I'll speak; for thou art\nOne that hath been a man, and yet I know not\nWhat thou art.\n```\n\n## Common workflows\n\n### Workflow 1: Character-level Shakespeare\n\n**Complete training pipeline**:\n```bash\n# Step 1: Prepare data (creates train.bin, val.bin)\npython data/shakespeare_char/prepare.py\n\n# Step 2: Train small model\npython train.py config/train_shakespeare_char.py\n\n# Step 3: Generate text\npython sample.py --out_dir=out-shakespeare-char\n```\n\n**Config** (`config/train_shakespeare_char.py`):\n```python\n# Model config\nn_layer = 6 # 6 transformer layers\nn_head = 6 # 6 attention heads\nn_embd = 384 # 384-dim embeddings\nblock_size = 256 # 256 char context\n\n# Training config\nbatch_size = 64\nlearning_rate = 1e-3\nmax_iters = 5000\neval_interval = 500\n\n# Hardware\ndevice = 'cpu' # Or 'cuda'\ncompile = False # Set True for PyTorch 2.0\n```\n\n**Training time**: ~5 minutes (CPU), ~1 minute (GPU)\n\n### Workflow 2: Reproduce GPT-2 (124M)\n\n**Multi-GPU training on OpenWebText**:\n```bash\n# Step 1: Prepare OpenWebText (takes ~1 hour)\npython data/openwebtext/prepare.py\n\n# Step 2: Train GPT-2 124M with DDP (8 GPUs)\ntorchrun --standalone --nproc_per_node=8 \\\n train.py config/train_gpt2.py\n\n# Step 3: Sample from trained model\npython sample.py --out_dir=out\n```\n\n**Config** (`config/train_gpt2.py`):\n```python\n# GPT-2 (124M) architecture\nn_layer = 12\nn_head = 12\nn_embd = 768\nblock_size = 1024\ndropout = 0.0\n\n# Training\nbatch_size = 12\ngradient_accumulation_steps = 5 * 8 # Total batch ~0.5M tokens\nlearning_rate = 6e-4\nmax_iters = 600000\nlr_decay_iters = 600000\n\n# System\ncompile = True # PyTorch 2.0\n```\n\n**Training time**: ~4 days (8× A100)\n\n### Workflow 3: Fine-tune pretrained GPT-2\n\n**Start from OpenAI checkpoint**:\n```python\n# In train.py or config\ninit_from = 'gpt2' # Options: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n\n# Model loads OpenAI weights automatically\npython train.py config/finetune_shakespeare.py\n```\n\n**Example config** (`config/finetune_shakespeare.py`):\n```python\n# Start from GPT-2\ninit_from = 'gpt2'\n\n# Dataset\ndataset = 'shakespeare_char'\nbatch_size = 1\nblock_size = 1024\n\n# Fine-tuning\nlearning_rate = 3e-5 # Lower LR for fine-tuning\nmax_iters = 2000\nwarmup_iters = 100\n\n# Regularization\nweight_decay = 1e-1\n```\n\n### Workflow 4: Custom dataset\n\n**Train on your own text**:\n```python\n# data/custom/prepare.py\nimport numpy as np\n\n# Load your data\nwith open('my_data.txt', 'r') as f:\n text = f.read()\n\n# Create character mappings\nchars = sorted(list(set(text)))\nstoi = {ch: i for i, ch in enumerate(chars)}\nitos = {i: ch for i, ch in enumerate(chars)}\n\n# Tokenize\ndata = np.array([stoi[ch] for ch in text], dtype=np.uint16)\n\n# Split train/val\nn = len(data)\ntrain_data = data[:int(n*0.9)]\nval_data = data[int(n*0.9):]\n\n# Save\ntrain_data.tofile('data/custom/train.bin')\nval_data.tofile('data/custom/val.bin')\n```\n\n**Train**:\n```bash\npython data/custom/prepare.py\npython train.py --dataset=custom\n```\n\n## When to use vs alternatives\n\n**Use nanoGPT when**:\n- Learning how GPT works\n- Experimenting with transformer variants\n- Teaching/education purposes\n- Quick prototyping\n- Limited compute (can run on CPU)\n\n**Simplicity advantages**:\n- **~300 lines**: Entire model in `model.py`\n- **~300 lines**: Training loop in `train.py`\n- **Hackable**: Easy to modify\n- **No abstractions**: Pure PyTorch\n\n**Use alternatives instead**:\n- **HuggingFace Transformers**: Production use, many models\n- **Megatron-LM**: Large-scale distributed training\n- **LitGPT**: More architectures, production-ready\n- **PyTorch Lightning**: Need high-level framework\n\n## Common issues\n\n**Issue: CUDA out of memory**\n\nReduce batch size or context length:\n```python\nbatch_size = 1 # Reduce from 12\nblock_size = 512 # Reduce from 1024\ngradient_accumulation_steps = 40 # Increase to maintain effective batch\n```\n\n**Issue: Training too slow**\n\nEnable compilation (PyTorch 2.0+):\n```python\ncompile = True # 2× speedup\n```\n\nUse mixed precision:\n```python\ndtype = 'bfloat16' # Or 'float16'\n```\n\n**Issue: Poor generation quality**\n\nTrain longer:\n```python\nmax_iters = 10000 # Increase from 5000\n```\n\nLower temperature:\n```python\n# In sample.py\ntemperature = 0.7 # Lower from 1.0\ntop_k = 200 # Add top-k sampling\n```\n\n**Issue: Can't load GPT-2 weights**\n\nInstall transformers:\n```bash\npip install transformers\n```\n\nCheck model name:\n```python\ninit_from = 'gpt2' # Valid: gpt2, gpt2-medium, gpt2-large, gpt2-xl\n```\n\n## Advanced topics\n\n**Model architecture**: See [references/architecture.md](references/architecture.md) for GPT block structure, multi-head attention, and MLP layers explained simply.\n\n**Training loop**: See [references/training.md](references/training.md) for learning rate schedule, gradient accumulation, and distributed data parallel setup.\n\n**Data preparation**: See [references/data.md](references/data.md) for tokenization strategies (character-level vs BPE) and binary format details.\n\n## Hardware requirements\n\n- **Shakespeare (char-level)**:\n - CPU: 5 minutes\n - GPU (T4): 1 minute\n - VRAM: <1GB\n\n- **GPT-2 (124M)**:\n - 1× A100: ~1 week\n - 8× A100: ~4 days\n - VRAM: ~16GB per GPU\n\n- **GPT-2 Medium (350M)**:\n - 8× A100: ~2 weeks\n - VRAM: ~40GB per GPU\n\n**Performance**:\n- With `compile=True`: 2× speedup\n- With `dtype=bfloat16`: 50% memory reduction\n\n## Resources\n\n- GitHub: https://github.com/karpathy/nanoGPT ⭐ 48,000+\n- Video: \"Let's build GPT\" by Andrej Karpathy\n- Paper: \"Attention is All You Need\" (Vaswani et al.)\n- OpenWebText: https://huggingface.co/datasets/Skylion007/openwebtext\n- Educational: Best for understanding transformers from scratch\n\n\n# References Skill\n\nPath: `01-model-architecture/rwkv/references/architecture-details.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/architecture-details.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/rwkv/references/rwkv7.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/rwkv7.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/rwkv/references/state-management.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/rwkv/references/state-management.md)\n\n\n---\nname: rwkv-architecture\ndescription: RNN+Transformer hybrid with O(n) inference. Linear time, infinite context, no KV cache. Train like GPT (parallel), infer like RNN (sequential). Linux Foundation AI project. Production at Windows, Office, NeMo. RWKV-7 (March 2025). Models up to 14B parameters.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [RWKV, Model Architecture, RNN, Transformer Hybrid, Linear Complexity, Infinite Context, Efficient Inference, Linux Foundation, Alternative Architecture]\ndependencies: [rwkv, torch, transformers]\n---\n\n# RWKV - Receptance Weighted Key Value\n\n## Quick start\n\nRWKV (RwaKuv) combines Transformer parallelization (training) with RNN efficiency (inference).\n\n**Installation**:\n```bash\n# Install PyTorch\npip install torch --upgrade --extra-index-url https://download.pytorch.org/whl/cu121\n\n# Install dependencies\npip install pytorch-lightning==1.9.5 deepspeed wandb ninja --upgrade\n\n# Install RWKV\npip install rwkv\n```\n\n**Basic usage** (GPT mode + RNN mode):\n```python\nimport os\nfrom rwkv.model import RWKV\n\nos.environ[\"RWKV_JIT_ON\"] = '1'\nos.environ[\"RWKV_CUDA_ON\"] = '1' # Use CUDA kernel for speed\n\n# Load model\nmodel = RWKV(\n model='/path/to/RWKV-4-Pile-1B5-20220903-8040',\n strategy='cuda fp16'\n)\n\n# GPT mode (parallel processing)\nout, state = model.forward([187, 510, 1563, 310, 247], None)\nprint(out.detach().cpu().numpy()) # Logits\n\n# RNN mode (sequential processing, same result)\nout, state = model.forward([187, 510], None) # First 2 tokens\nout, state = model.forward([1563], state) # Next token\nout, state = model.forward([310, 247], state) # Last tokens\nprint(out.detach().cpu().numpy()) # Same logits as above!\n```\n\n## Common workflows\n\n### Workflow 1: Text generation (streaming)\n\n**Efficient token-by-token generation**:\n```python\nfrom rwkv.model import RWKV\nfrom rwkv.utils import PIPELINE\n\nmodel = RWKV(model='RWKV-4-Pile-14B-20230313-ctx8192-test1050', strategy='cuda fp16')\npipeline = PIPELINE(model, \"20B_tokenizer.json\")\n\n# Initial prompt\nprompt = \"The future of AI is\"\nstate = None\n\n# Generate token by token\nfor token in prompt:\n out, state = pipeline.model.forward(pipeline.encode(token), state)\n\n# Continue generation\nfor _ in range(100):\n out, state = pipeline.model.forward(None, state)\n token = pipeline.sample_logits(out)\n print(pipeline.decode(token), end='', flush=True)\n```\n\n**Key advantage**: Constant memory per token (no growing KV cache)\n\n### Workflow 2: Long context processing (infinite context)\n\n**Process million-token sequences**:\n```python\nmodel = RWKV(model='RWKV-4-Pile-14B', strategy='cuda fp16')\n\n# Process very long document\nstate = None\nlong_document = load_document() # e.g., 1M tokens\n\n# Stream through entire document\nfor chunk in chunks(long_document, chunk_size=1024):\n out, state = model.forward(chunk, state)\n\n# State now contains information from entire 1M token document\n# Memory usage: O(1) (constant, not O(n)!)\n```\n\n### Workflow 3: Fine-tuning RWKV\n\n**Standard fine-tuning workflow**:\n```python\n# Training script\nimport pytorch_lightning as pl\nfrom rwkv.model import RWKV\nfrom rwkv.trainer import RWKVTrainer\n\n# Configure model\nconfig = {\n 'n_layer': 24,\n 'n_embd': 1024,\n 'vocab_size': 50277,\n 'ctx_len': 1024\n}\n\n# Setup trainer\ntrainer = pl.Trainer(\n accelerator='gpu',\n devices=8,\n precision='bf16',\n strategy='deepspeed_stage_2',\n max_epochs=1\n)\n\n# Train\nmodel = RWKV(config)\ntrainer.fit(model, train_dataloader)\n```\n\n### Workflow 4: RWKV vs Transformer comparison\n\n**Memory comparison** (1M token sequence):\n```python\n# Transformer (GPT)\n# Memory: O(n²) for attention\n# KV cache: 1M × hidden_dim × n_layers × 2 (keys + values)\n# Example: 1M × 4096 × 24 × 2 = ~400GB (impractical!)\n\n# RWKV\n# Memory: O(1) per token\n# State: hidden_dim × n_layers = 4096 × 24 = ~400KB\n# 1,000,000× more efficient!\n```\n\n**Speed comparison** (inference):\n```python\n# Transformer: O(n) per token (quadratic overall)\n# First token: 1 computation\n# Second token: 2 computations\n# ...\n# 1000th token: 1000 computations\n\n# RWKV: O(1) per token (linear overall)\n# Every token: 1 computation\n# 1000th token: 1 computation (same as first!)\n```\n\n## When to use vs alternatives\n\n**Use RWKV when**:\n- Need very long context (100K+ tokens)\n- Want constant memory usage\n- Building streaming applications\n- Need RNN efficiency with Transformer performance\n- Memory-constrained deployment\n\n**Key advantages**:\n- **Linear time**: O(n) vs O(n²) for Transformers\n- **No KV cache**: Constant memory per token\n- **Infinite context**: No fixed window limit\n- **Parallelizable training**: Like GPT\n- **Sequential inference**: Like RNN\n\n**Use alternatives instead**:\n- **Transformers**: Need absolute best performance, have compute\n- **Mamba**: Want state-space models\n- **RetNet**: Need retention mechanism\n- **Hyena**: Want convolution-based approach\n\n## Common issues\n\n**Issue: Out of memory during training**\n\nUse gradient checkpointing and DeepSpeed:\n```python\ntrainer = pl.Trainer(\n strategy='deepspeed_stage_3', # Full ZeRO-3\n precision='bf16'\n)\n```\n\n**Issue: Slow inference**\n\nEnable CUDA kernel:\n```python\nos.environ[\"RWKV_CUDA_ON\"] = '1'\n```\n\n**Issue: Model not loading**\n\nCheck model path and strategy:\n```python\nmodel = RWKV(\n model='/absolute/path/to/model.pth',\n strategy='cuda fp16' # Or 'cpu fp32' for CPU\n)\n```\n\n**Issue: State management in RNN mode**\n\nAlways pass state between forward calls:\n```python\n# WRONG: State lost\nout1, _ = model.forward(tokens1, None)\nout2, _ = model.forward(tokens2, None) # No context from tokens1!\n\n# CORRECT: State preserved\nout1, state = model.forward(tokens1, None)\nout2, state = model.forward(tokens2, state) # Has context from tokens1\n```\n\n## Advanced topics\n\n**Time-mixing and channel-mixing**: See [references/architecture-details.md](references/architecture-details.md) for WKV operation, time-decay mechanism, and receptance gates.\n\n**State management**: See [references/state-management.md](references/state-management.md) for att_x_prev, att_kv, ffn_x_prev states, and numerical stability considerations.\n\n**RWKV-7 improvements**: See [references/rwkv7.md](references/rwkv7.md) for latest architectural improvements (March 2025) and multimodal capabilities.\n\n## Hardware requirements\n\n- **GPU**: NVIDIA (CUDA 11.6+) or CPU\n- **VRAM** (FP16):\n - 169M model: 1GB\n - 430M model: 2GB\n - 1.5B model: 4GB\n - 3B model: 8GB\n - 7B model: 16GB\n - 14B model: 32GB\n- **Inference**: O(1) memory per token\n- **Training**: Parallelizable like GPT\n\n**Performance** (vs Transformers):\n- **Speed**: Similar training, faster inference\n- **Memory**: 1000× less for long sequences\n- **Scaling**: Linear vs quadratic\n\n## Resources\n\n- Paper (RWKV): https://arxiv.org/abs/2305.13048 (May 2023)\n- Paper (RWKV-7): https://arxiv.org/abs/2503.14456 (March 2025)\n- GitHub: https://github.com/BlinkDL/RWKV-LM ⭐ 12,000+\n- Docs: https://wiki.rwkv.com/\n- Models: https://huggingface.co/BlinkDL\n- Linux Foundation AI: Official project\n- Production: Microsoft Windows, Office integration, NeMo support\n\n\n# References Skill\n\nPath: `01-model-architecture/torchtitan/references/checkpoint.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/checkpoint.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/torchtitan/references/custom-models.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/custom-models.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/torchtitan/references/float8.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/float8.md)\n\n\n# References Skill\n\nPath: `01-model-architecture/torchtitan/references/fsdp.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/01-model-architecture/torchtitan/references/fsdp.md)\n\n\n---\nname: distributed-llm-pretraining-torchtitan\ndescription: Provides PyTorch-native distributed LLM pretraining using torchtitan with 4D parallelism (FSDP2, TP, PP, CP). Use when pretraining Llama 3.1, DeepSeek V3, or custom models at scale from 8 to 512+ GPUs with Float8, torch.compile, and distributed checkpointing.\nversion: 1.0.0\nauthor: Orchestra Research\nlicense: MIT\ntags: [Model Architecture, Distributed Training, TorchTitan, FSDP2, Tensor Parallel, Pipeline Parallel, Context Parallel, Float8, Llama, Pretraining]\ndependencies: [torch>=2.6.0, torchtitan>=0.2.0, torchao>=0.5.0]\n---\n\n# TorchTitan - PyTorch Native Distributed LLM Pretraining\n\n## Quick start\n\nTorchTitan is PyTorch's official platform for large-scale LLM pretraining with composable 4D parallelism (FSDP2, TP, PP, CP), achieving 65%+ speedups over baselines on H100 GPUs.\n\n**Installation**:\n```bash\n# From PyPI (stable)\npip install torchtitan\n\n# From source (latest features, requires PyTorch nightly)\ngit clone https://github.com/pytorch/torchtitan\ncd torchtitan\npip install -r requirements.txt\n```\n\n**Download tokenizer**:\n```bash\n# Get HF token from https://huggingface.co/settings/tokens\npython scripts/download_hf_assets.py --repo_id meta-llama/Llama-3.1-8B --assets tokenizer --hf_token=...\n```\n\n**Start training on 8 GPUs**:\n```bash\nCONFIG_FILE=\"./torchtitan/models/llama3/train_configs/llama3_8b.toml\" ./run_train.sh\n```\n\n## Common workflows\n\n### Workflow 1: Pretrain Llama 3.1 8B on single node\n\nCopy this checklist:\n\n```\nSingle Node Pretraining:\n- [ ] Step 1: Download tokenizer\n- [ ] Step 2: Configure training\n- [ ] Step 3: Launch training\n- [ ] Step 4: Monitor and checkpoint\n```\n\n**Step 1: Download tokenizer**\n\n```bash\npython scripts/download_hf_assets.py \\\n --repo_id meta-llama/Llama-3.1-8B \\\n --assets tokenizer \\\n --hf_token=YOUR_HF_TOKEN\n```\n\n**Step 2: Configure training**\n\nEdit or create a TOML config file:\n\n```toml\n# llama3_8b_custom.toml\n[job]\ndump_folder = \"./outputs\"\ndescription = \"Llama 3.1 8B training\"\n\n[model]\nname = \"llama3\"\nflavor = \"8B\"\nhf_assets_path = \"./assets/hf/Llama-3.1-8B\"\n\n[optimizer]\nname = \"AdamW\"\nlr = 3e-4\n\n[lr_scheduler]\nwarmup_steps = 200\n\n[training]\nlocal_batch_size = 2\nseq_len = 8192\nmax_norm = 1.0\nsteps = 1000\ndataset = \"c4\"\n\n[parallelism]\ndata_parallel_shard_degree = -1 # Use all GPUs for FSDP\n\n[activation_checkpoint]\nmode = \"selective\"\nselective_ac_option = \"op\"\n\n[checkpoint]\nenable = true\nfolder = \"checkpoint\"\ninterval = 500\n```\n\n**Step 3: Launch training**\n\n```bash\n# 8 GPUs on single node\nCONFIG_FILE=\"./llama3_8b_custom.toml\" ./run_train.sh\n\n# Or explicitly with torchrun\ntorchrun --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_8b_custom.toml\n```\n\n**Step 4: Monitor and checkpoint**\n\nTensorBoard logs are saved to `./outputs/tb/`:\n```bash\ntensorboard --logdir ./outputs/tb\n```\n\n### Workflow 2: Multi-node training with SLURM\n\n```\nMulti-Node Training:\n- [ ] Step 1: Configure parallelism for scale\n- [ ] Step 2: Set up SLURM script\n- [ ] Step 3: Submit job\n- [ ] Step 4: Resume from checkpoint\n```\n\n**Step 1: Configure parallelism for scale**\n\nFor 70B model on 256 GPUs (32 nodes):\n```toml\n[parallelism]\ndata_parallel_shard_degree = 32 # FSDP across 32 ranks\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 1 # No PP for 70B\ncontext_parallel_degree = 1 # Increase for long sequences\n```\n\n**Step 2: Set up SLURM script**\n\n```bash\n#!/bin/bash\n#SBATCH --job-name=llama70b\n#SBATCH --nodes=32\n#SBATCH --ntasks-per-node=8\n#SBATCH --gpus-per-node=8\n\nsrun torchrun \\\n --nnodes=32 \\\n --nproc_per_node=8 \\\n --rdzv_backend=c10d \\\n --rdzv_endpoint=$MASTER_ADDR:$MASTER_PORT \\\n -m torchtitan.train \\\n --job.config_file ./llama3_70b.toml\n```\n\n**Step 3: Submit job**\n\n```bash\nsbatch multinode_trainer.slurm\n```\n\n**Step 4: Resume from checkpoint**\n\nTraining auto-resumes if checkpoint exists in configured folder.\n\n### Workflow 3: Enable Float8 training for H100s\n\nFloat8 provides 30-50% speedup on H100 GPUs.\n\n```\nFloat8 Training:\n- [ ] Step 1: Install torchao\n- [ ] Step 2: Configure Float8\n- [ ] Step 3: Launch with compile\n```\n\n**Step 1: Install torchao**\n\n```bash\nUSE_CPP=0 pip install git+https://github.com/pytorch/ao.git\n```\n\n**Step 2: Configure Float8**\n\nAdd to your TOML config:\n```toml\n[model]\nconverters = [\"quantize.linear.float8\"]\n\n[quantize.linear.float8]\nenable_fsdp_float8_all_gather = true\nprecompute_float8_dynamic_scale_for_fsdp = true\nfilter_fqns = [\"output\"] # Exclude output layer\n\n[compile]\nenable = true\ncomponents = [\"model\", \"loss\"]\n```\n\n**Step 3: Launch with compile**\n\n```bash\nCONFIG_FILE=\"./llama3_8b.toml\" ./run_train.sh \\\n --model.converters=\"quantize.linear.float8\" \\\n --quantize.linear.float8.enable_fsdp_float8_all_gather \\\n --compile.enable\n```\n\n### Workflow 4: 4D parallelism for 405B models\n\n```\n4D Parallelism (FSDP + TP + PP + CP):\n- [ ] Step 1: Create seed checkpoint\n- [ ] Step 2: Configure 4D parallelism\n- [ ] Step 3: Launch on 512 GPUs\n```\n\n**Step 1: Create seed checkpoint**\n\nRequired for consistent initialization across PP stages:\n```bash\nNGPU=1 CONFIG_FILE=./llama3_405b.toml ./run_train.sh \\\n --checkpoint.enable \\\n --checkpoint.create_seed_checkpoint \\\n --parallelism.data_parallel_shard_degree 1 \\\n --parallelism.tensor_parallel_degree 1 \\\n --parallelism.pipeline_parallel_degree 1\n```\n\n**Step 2: Configure 4D parallelism**\n\n```toml\n[parallelism]\ndata_parallel_shard_degree = 8 # FSDP\ntensor_parallel_degree = 8 # TP within node\npipeline_parallel_degree = 8 # PP across nodes\ncontext_parallel_degree = 1 # CP for long sequences\n\n[training]\nlocal_batch_size = 32\nseq_len = 8192\n```\n\n**Step 3: Launch on 512 GPUs**\n\n```bash\n# 64 nodes x 8 GPUs = 512 GPUs\nsrun torchrun --nnodes=64 --nproc_per_node=8 \\\n -m torchtitan.train \\\n --job.config_file ./llama3_405b.toml\n```\n\n## When to use vs alternatives\n\n**Use TorchTitan when:**\n- Pretraining LLMs from scratch (8B to 405B+)\n- Need PyTorch-native solution without third-party dependencies\n- Require composable 4D parallelism (FSDP2, TP, PP, CP)\n- Training on H100s with Float8 support\n- Want interoperable checkpoints with torchtune/HuggingFace\n\n**Use alternatives instead:**\n- **Megatron-LM**: Maximum performance for NVIDIA-only deployments\n- **DeepSpeed**: Broader ZeRO optimization ecosystem, inference support\n- **Axolotl/TRL**: Fine-tuning rather than pretraining\n- **LitGPT**: Educational, smaller-scale training\n\n## Common issues\n\n**Issue: Out of memory on large models**\n\nEnable activation checkpointing and reduce batch size:\n```toml\n[activation_checkpoint]\nmode = \"full\" # Instead of \"selective\"\n\n[training]\nlocal_batch_size = 1\n```\n\nOr use gradient accumulation:\n```toml\n[training]\nlocal_batch_size = 1\nglobal_batch_size = 32 # Accumulates gradients\n```\n\n**Issue: TP causes high memory with async collectives**\n\nSet environment variable:\n```bash\nexport TORCH_NCCL_AVOID_RECORD_STREAMS=1\n```\n\n**Issue: Float8 training not faster**\n\nFloat8 only benefits large GEMMs. Filter small layers:\n```toml\n[quantize.linear.float8]\nfilter_fqns = [\"attention.wk\", \"attention.wv\", \"output\", \"auto_filter_small_kn\"]\n```\n\n**Issue: Checkpoint loading fails after parallelism change**\n\nUse DCP's resharding capability:\n```bash\n# Convert sharded checkpoint to single file\npython -m torch.distributed.checkpoint.format_utils \\\n dcp_to_torch checkpoint/step-1000 checkpoint.pt\n```\n\n**Issue: Pipeline parallelism initialization**\n\nCreate seed checkpoint first (see Workflow 4, Step 1).\n\n## Supported models\n\n| Model | Sizes | Status |\n|-------|-------|--------|\n| Llama 3.1 | 8B, 70B, 405B | Production |\n| Llama 4 | Various | Experimental |\n| DeepSeek V3 | 16B, 236B, 671B (MoE) | Experimental |\n| GPT-OSS | 20B, 120B (MoE) | Experimental |\n| Qwen 3 | Various | Experimental |\n| Flux | Diffusion | Experimental |\n\n## Performance benchmarks (H100)\n\n| Model | GPUs | Parallelism | TPS/GPU | Techniques |\n|-------|------|-------------|---------|------------|\n| Llama 8B | 8 | FSDP | 5,762 | Baseline |\n| Llama 8B | 8 | FSDP+compile+FP8 | 8,532 | +48% |\n| Llama 70B | 256 | FSDP+TP+AsyncTP | 876 | 2D parallel |\n| Llama 405B | 512 | FSDP+TP+PP | 128 | 3D parallel |\n\n## Advanced topics\n\n**FSDP2 configuration**: See [references/fsdp.md](references/fsdp.md) for detailed FSDP2 vs FSDP1 comparison and ZeRO equivalents.\n\n**Float8 training**: See [references/float8.md](references/float8.md) for tensorwise vs rowwise scaling recipes.\n\n**Checkpointing**: See [references/checkpoint.md](references/checkpoint.md) for HuggingFace conversion and async checkpointing.\n\n**Adding custom models**: See [references/custom-models.md](references/custom-models.md) for TrainSpec protocol.\n\n## Resources\n\n- GitHub: https://github.com/pytorch/torchtitan\n- Paper: https://arxiv.org/abs/2410.06511\n- ICLR 2025: https://iclr.cc/virtual/2025/poster/29620\n- PyTorch Forum: https://discuss.pytorch.org/c/distributed/torchtitan/44\n\n\n# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/algorithms.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/algorithms.md)\n\n\n# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/integration.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/integration.md)\n\n\n# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/pipeline.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/pipeline.md)\n\n\n# References Skill\n\nPath: `02-tokenization/huggingface-tokenizers/references/training.md`\n\n[View Raw Content on GitHub](https://raw.githubusercontent.com/Orchestra-Research/AI-Research-SKILLs/HEAD/02-tokenization/huggingface-tokenizers/references/training.md)\n\n\n"}