README

slime Documentation

We recommend new contributors start from writing documentation, which helps you quickly understand slime codebase.
Most documentation files are located under the docs/ folder.

Docs Workflow

Install Dependency

bash

apt-get update && apt-get install -y pandoc parallel retry
pip install -r requirements.txt

Update Documentation

You can update the documentation in the en and zh folders by adding Markdown or Jupyter Notebook files to the appropriate subdirectories. If you create new files, make sure to update index.rst (or any other relevant .rst files) accordingly.

Build and Render

bash

build english version

bash ./build.sh en
bash ./serve.sh en
build chinese version

bash ./build.sh zh
bash ./serve.sh zh

You can then visit http://localhost:8000 to view the documentation.

---

En/Get Started/Agent

Agentic RL Training Roadmap

slime is not limited to single-turn RL. Its main advantage for agentic training is the combination of high-performance training, SGLang rollout serving, and pluggable data-generation interfaces. This makes it suitable for multi-turn tool use, sandbox interaction, subagent branches, context compaction, and test-based rewards.

This page is a roadmap: use it to decide which docs and examples to read when plugging an agent workflow into slime.

Where To Start

| Goal | Recommended entry point |
| :--- | :--- |
| Run a custom agent loop, tool calls, RAG, browser/terminal/sandbox interaction for each sample | --custom-generate-function-path, writing a custom generation function |
| Implement verifier rewards, test-based rewards, environment success checks, or an external reward service | --custom-rm-path, writing a custom reward function |
| Return multiple training samples from one prompt, such as subagent, multi-agent, or context-compaction segments | fan-out return from custom generate, examples/multi_agent |
| Avoid blocking training on long-tail agent rollouts | examples/fully_async |
| Study a full end-to-end agent example with sandboxing, real code edits, and test-based grading | examples/coding_agent_rl |
| Improve SGLang serving throughput for multi-turn agents | PD Disaggregation, SGLang Config |
| Enable SGLang optimization flags, router policies, or multi-model serving | How to Use SGLang, SGLang Config, Speculative Decoding, Low Precision Training |

Recommended Integration Pattern

Most agentic RL tasks should start with --custom-generate-function-path. This function converts one agent execution into slime-trainable Sample objects: fill tokens, response_length, loss_mask, and status, then either fill reward directly or let --custom-rm-path compute it.

The agent workflow itself may speak in strings, chat messages, tool calls, environment observations, or framework-specific events. The training target, however, should stay token based. Preserve the model-sampled token ids and use loss_mask to separate trainable model output from prompt, template, tool-observation, or environment text.

If one prompt rollout corresponds to one training sample, return a single Sample. If one rollout splits into multiple trainable segments, such as subagent trajectories, main-agent continuations, or pre/post-compaction segments, return list[Sample] and set the same rollout_id on all sibling samples. slime then keeps those samples together for train-step splitting and loss aggregation instead of counting them as independent rollouts.

Reach for --rollout-function-path only when you need to replace the whole rollout orchestration. Common reasons include custom data-source scheduling, cross-rollout background queues, fully asynchronous generation, or workflows that cannot fit the default sglang_rollout prompt-by-sample structure.

Agent Runtime Adapters

slime includes protocol adapters for existing agent runtimes:

- slime.agent.adapters.AnthropicAdapter: Anthropic Messages API, used by Claude Code style agents.
- slime.agent.adapters.OpenAIAdapter: OpenAI Chat Completions and Responses APIs, used by OpenAI SDK / OpenAI Agents SDK style clients.

Adapters are a convenience layer, not a separate agent framework. Their contract is message history in, sampled tokens out: they render the chat template, call SGLang with input_ids and return_logprob=True, and export the returned token ids/logprobs as trainable trajectory segments. They avoid re-tokenizing response text to recover the training target.

Instantiate the protocol-specific adapter in your custom generate function, run its app with aiohttp, then manage each rollout through the adapter instance:

python

from slime.agent.adapters import AnthropicAdapter
adapter = AnthropicAdapter(
    tokenizer=tokenizer,
    sglang_url=sglang_url,
    tool_parser=tool_parser,
    reasoning_parser=reasoning_parser,
)
adapter.open_session(session_id, sampling_defaults=sampling_params)
Agent client sends requests to adapter.app.

segments = await adapter.finish_session(session_id)

For multi-turn agents, use a stable session_id. The adapters pass it as X-SMG-Routing-Key so SGLang can route one session to the same worker and reuse prefix cache.

Agent Serving And Performance

Agentic rollouts tend to depend more heavily on serving configuration than ordinary single-turn generation: contexts are longer, requests are multi-turn, latency has a heavier tail, and the workflow may need actor, reference, reward, or tool-side models at the same time.

- Regular SGLang server arguments are passed as --sglang-*. For example, SGLang's --context-length becomes --sglang-context-length, and --mem-fraction-static becomes --sglang-mem-fraction-static.
- Router arguments are passed as --router-*. For multi-turn agents, consider --router-policy consistent_hashing so requests for the same sample.session_id go to the same worker and improve prefix-cache hit rate. See Session-Affinity Routing for Multi-Turn Agents.
- Use --sglang-config for more complex topologies: PD disaggregation, multi-model serving, heterogeneous server groups, and per-group SGLang overrides.
- For multi-turn or agentic RL, evaluate PD disaggregation. Prefill and decode have different workload shapes, and separating them makes it easier to scale each resource independently.
- For rollout-throughput optimization, also see Speculative Decoding and Low Precision Training.

Reference Example

The full coding-agent example is examples/coding_agent_rl. It shows an end-to-end agent RL setup that is close to a real software-engineering workflow: each sample boots an isolated sandbox, the agent uses tools to edit code, the rollout captures a git diff, and a clean sandbox runs the tests to produce the reward.

This example also demonstrates agent fan-out training. Its middleware splits one trajectory into subagent, wipe (the chain frozen before compaction), and final segments. generate() returns list[Sample], and all segments share the same rollout_id.

For smaller starting points, see examples/search-r1 for multi-turn tool use, examples/retool for tool-augmented generation, and examples/multi_agent for the multi-agent pattern.

---

En/Get Started/Customization

Customization Guide

slime provides extensive customization capabilities through function path arguments. These allow you to inject custom logic at various stages of the training and rollout pipeline without modifying the core codebase.

Overview of Customization Interfaces

Below is a summary of all available customization interfaces and their purposes.

Agentic workflows through customization interfaces

Agentic workflows — multi-turn tool use, sandbox interaction, environment feedback, verifier/test-based rewards — are an important class of data generation workflows. They plug into slime through the existing customization interfaces; slime does not require a separate agent framework.

For most agentic use cases, start with --custom-generate-function-path plus --custom-rm-path, and only override the full rollout function when the default rollout loop is insufficient.

| If you need to … | Use |
| :--- | :--- |
| Run a custom agent loop, tool calls, RAG, sandbox execution, browser/terminal interaction, or multi-turn generation for each sample, while reusing slime's default rollout loop | --custom-generate-function-path |
| Compute verifier rewards, test-based rewards, environment success checks, rule-based rewards, or call an external reward service | --custom-rm-path |
| Replace the entire rollout orchestration (only when per-sample customization is not enough) | --rollout-function-path |
| Control task sampling, buffering, requeueing, or custom prompt/task sources | --data-source-path |
| Attach custom loss masks, metadata, or convert agentic outputs into training data | --rollout-data-postprocess-path, --custom-convert-samples-to-train-data-path |
| Debug long-running custom generation, verifier calls, tool calls, or sandbox steps | trace utilities in slime.utils.trace_utils |

A native example of this pattern is examples/search-r1, which adds search-augmented multi-turn generation via --custom-generate-function-path while keeping slime's default sglang_rollout outer loop. See also examples/multi_agent for a --rollout-function-path-based multi-agent pattern and examples/fully_async for long-tail agentic generation.

Detailed Interface Reference

1. Rollout Function (`--rollout-function-path`)

Default: slime.rollout.sglang_rollout.generate_rollout

Purpose: Override the entire rollout generation logic.

Signature:

python

def generate_rollout(args, rollout_id, data_source, evaluation=False) -> RolloutFnTrainOutput | RolloutFnEvalOutput

Use Cases:
- Implementing complex multi-turn conversations
- Adding custom sampling strategies
- Integrating external tools or APIs during generation

Example: See examples/multi_agent/rollout_with_multi_agents.py

---

2. Custom Generate Function (`--custom-generate-function-path`)

Default: None (uses built-in generate function)

Purpose: Override only the generation step within the default rollout function.

Signature:

python

async def custom_generate(args, sample: Sample, sampling_params: dict) -> Sample | list[Sample]

Use Cases:
- Implementing tool-calling or function-calling capabilities
- Adding retrieval-augmented generation (RAG)
- Multi-turn conversation handling

#### Returning multiple training samples for one prompt

In agentic settings such as subagents, multi-agent execution, or context compaction, one prompt rollout can naturally split into multiple trainable segments. For example, a subagent trajectory and the main-agent continuation may both need to be trained, or the context before and after compaction may be represented as separate segments.

You do not need to replace the whole rollout function for this. A custom_generate function may return list[Sample]. The key contract is that sibling samples produced by the same rollout must share the same rollout_id, so slime keeps them together for train-step splitting and loss aggregation instead of counting them as independent rollouts.

python

import copy
from slime.utils.types import Sample

async def custom_generate(args, sample: Sample, sampling_params: dict) -> list[Sample]:
    segments = await run_agent_and_split_segments(args, sample, sampling_params)
    rollout_id = sample.rollout_id if sample.rollout_id is not None else sample.index    samples: list[Sample] = []
    for segment in segments:
        s = copy.copy(sample)
        s.tokens = segment.tokens
        s.response = segment.response
        s.response_length = segment.response_length
        s.loss_mask = segment.loss_mask
        s.reward = segment.reward
        s.status = Sample.Status.COMPLETED
        s.rollout_id = rollout_id
        samples.append(s)
    return samples

If one full trajectory has a single total reward but is split into K training segments, a common pattern is to distribute that reward across the segments, for example by assigning reward / K to each segment, so the same rollout reward is not amplified.

Example: See examples/search-r1/generate_with_search.py

---

3. Reward Model (`--custom-rm-path`)

Default: None (uses built-in reward models based on --rm-type)

Purpose: Implement custom reward computation logic.

Signature (single sample mode):

python

async def custom_rm(args, sample: Sample) -> float

Signature (batch mode, when --group-rm is enabled):

python

async def batched_custom_rm(args, samples: list[Sample]) -> list[float]

Use Cases:
- Custom rule-based rewards
- Integration with external reward model services
- Multi-dimensional reward signals

Built-in Options (--rm-type):
- math: Mathematical answer verification
- dapo: DAPO-style scoring
- deepscaler: DeepScaler rule-based reward
- f1: F1 score computation
- gpqa: GPQA reward computation
- ifbench: IFBench reward computation
- remote_rm: Remote reward model service (requires --rm-url)

---

4. Dynamic Sampling Filter (`--dynamic-sampling-filter-path`)

Default: None

Purpose: Filter samples during dynamic sampling (e.g., DAPO-style filtering).

Signature:

python

def filter_function(args, samples: list[Sample], kwargs) -> DynamicFilterOutput

Return Type:

python

@dataclass
class DynamicFilterOutput:
    keep: bool  # Whether to keep this sample group
    reason: str | None  # Reason for filtering (for logging)

Use Cases:
- Filtering out samples where all responses have the same reward
- Implementing curriculum learning strategies
- Quality-based sample selection

Example: slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std

---

5. Buffer Filter (`--buffer-filter-path`)

Default: None

Purpose: Filter samples in the rollout buffer before training.

Signature:

python

def buffer_filter(args, rollout_id, buffer: list[list[Sample]], num_samples: int) -> list[list[Sample]]

Use Cases:
- Removing low-quality samples before training
- Implementing priority-based sample selection
- Balancing sample distributions

---

6. Rollout Sample Filter (`--rollout-sample-filter-path`)

Default: None

Purpose: Determine whether individual samples participate in loss calculation.

Signature:

python

def filter_function(args, samples: list[Sample]) -> None

Note: This function should directly modify the remove_sample attribute of each Sample object.

Use Cases:
- Filtering samples based on response quality
- Implementing selective training strategies

---

7. Rollout All Samples Process (`--rollout-all-samples-process-path`)

Default: None

Purpose: Process all samples (including filtered ones) after rollout.

Signature:

python

def process_function(args, samples: list[list[Sample]], data_source) -> None

Use Cases:
- Logging and analysis of all generated samples
- Computing statistics across filtered and kept samples

---

8. Rollout Data Postprocess (`--rollout-data-postprocess-path`)

Default: None

Purpose: Post-process rollout data after log probabilities are computed.

Signature:

python

def postprocess_function(args, samples: list[list[Sample]]) -> None

Use Cases:
- Updating loss masks based on computed values
- Adding additional metadata to samples

---

9. Custom Loss Function (`--custom-loss-function-path`)

Default: None (requires --loss-type custom_loss)

Purpose: Implement custom training loss computation.

Use Cases:
- Novel RL objectives
- Multi-objective optimization
- Custom regularization terms

---

10. Custom TIS/RS Function (`--custom-tis-function-path`)

Default: None

Purpose: Implement custom importance sampling for off-policy correction.

Use Cases:
- Custom importance sampling ratio computation
- Advanced off-policy correction methods

Example: examples/train_infer_mismatch_helper/mis.py:compute_mis_weights_with_cp

---

11. Custom pg_loss Reducer (`--custom-pg-loss-reducer-function-path`)

Default: None

Purpose: Customize the reduction of pg_loss while other metrics (pg_clipfrac, ppo_kl, entropy_loss, etc.) still use the default sum_of_sample_mean.

Signature:

python

def get_pg_loss_reducer(
    total_lengths: list[int],
    response_lengths: list[int],
    loss_masks: list[torch.Tensor],
    calculate_per_token_loss: bool = False,
) -> Callable[[torch.Tensor], torch.Tensor]

Use Cases:
- Dr.GRPO: Divide by a constant instead of effective token count
- Custom loss normalization strategies

---

12. Reward Post-Processing (`--custom-reward-post-process-path`)

Default: None (uses default GRPO normalization)

Purpose: Custom post-processing of rewards before advantage computation.

Use Cases:
- Custom reward normalization strategies
- Reward shaping

---

13. Samples to Train Data Conversion (`--custom-convert-samples-to-train-data-path`)

Default: None (uses built-in conversion logic)

Purpose: Override the conversion of samples to training data format.

Signature:

python

def convert_samples_to_train_data(
    args,
    samples: list[Sample] | list[list[Sample]],
) -> dict

Return Type:

python

dict: {
    "tokens": list[list[int]],           # Token IDs for each sample
    "response_lengths": list[int],        # Response lengths
    "rewards": list[float],               # Normalized rewards
    "raw_reward": list[float],            # Raw rewards
    "truncated": list[int],               # Truncation flags (0 or 1)
    "sample_indices": list[int],          # Sample indices
    "loss_masks": list[list[int]],        # Loss masks for each sample
    # Optional fields:
    "round_number": list[int],            # Round numbers (for rollout buffer)
    "rollout_log_probs": list,            # Log probs (for off-policy correction)
    "rollout_routed_experts": list,       # Routed experts (for MoE)
    "metadata": list,                     # Train metadata
    "multimodal_train_inputs": list,      # Multimodal tensors (for VLM)
    "teacher_log_probs": list,            # Teacher log probs (for distillation)
}

Use Cases:
- Handling list[list[Sample]] inputs
- Custom data format requirements for training

---

14. Logging Functions

#### Training Rollout Logging (--custom-rollout-log-function-path)

Signature:

python

def log_rollout_data(rollout_id, args, samples, rollout_extra_metrics, rollout_time) -> bool

Return: True to skip default logging, False to continue with default logging.

#### Evaluation Rollout Logging (--custom-eval-rollout-log-function-path)

Signature:

python

def log_eval_rollout_data(rollout_id, args, data, extra_metrics) -> bool

Return: True to skip default logging, False to continue with default logging.

---

15. Data Source (`--data-source-path`)

Default: slime.rollout.data_source.RolloutDataSourceWithBuffer

Purpose: Override the data source for rollout prompts.

Base Class: slime.rollout.data_source.DataSource

Required Methods:

python

class CustomDataSource(DataSource):
    def get_samples(self, num_samples: int) -> list[list[Sample]]:
        """Return num_samples samples"""
        
    def add_samples(self, samples: list[list[Sample]]):
        """Add samples back to the data source"""
        
    def save(self, rollout_id):
        """Save state for checkpointing"""
        
    def load(self, rollout_id=None):
        """Load state from checkpoint"""    def __len__(self):
        """Length of the data source. May change when samples are added/fetched."""

---

16. Evaluation Function (`--eval-function-path`)

Default: Same as --rollout-function-path

Purpose: Override the rollout function specifically for evaluation.

Use Cases:
- Different sampling parameters for evaluation
- Evaluation-specific logic

---

17. Megatron Hooks

#### Megatron Initialization (--custom-megatron-init-path)

Signature:

python

def custom_init(args) -> None

Purpose: Custom initialization after Megatron setup.

#### Before Log Prob Hook (--custom-megatron-before-log-prob-hook-path)

Signature:

python

def custom_hook(args, model, store_prefix) -> None

Purpose: Custom logic before log probability computation.

#### Before Train Step Hook (--custom-megatron-before-train-step-hook-path)

Signature:

python

def custom_hook(args, rollout_id, step_id, model, optimizer, opt_param_scheduler) -> None

Purpose: Custom logic before each training step.

---

18. MoE Routing Replay

Stabilize MoE RL training by recording and replaying expert routing decisions to ensure consistency.

| Argument | Description |
| --- | --- |
| --use-routing-replay | Forward-backward routing consistency in training. (arXiv:2507.18071) |
| --use-rollout-routing-replay | R3: Replay routing from rollout during training. Supported by slime's default sglang_router path. (arXiv:2510.11370) |

---

19. Disk Weight-Sync Post-Write Hook (`--custom-update-weight-post-write-path`)

Signature:

python

def hook(args, version_dir: str, rollout_engines) -> None

Purpose: Called on each trainer rank after a disk weight sync's files are written
(--update-weight-transport disk, full or delta mode), before the engines read them. Use it to
publish the writes on a non-POSIX shared filesystem — e.g. upload pending writes to the
backing object store — where another host cannot see the files without an explicit sync. The hook is called
on every rank and must gate itself (e.g. once per container).

The read-side counterpart runs inside the inference engine, on every host it spans, and is
therefore an sglang server argument rather than a slime hook: pass
--sglang-custom-pull-weights-pre-read-hook <import.path> with signature
hook(source_dir: str, target_version: int) — called before /pull_weights reads the
published weights (e.g. refresh the mount's view). See
Delta Weight Sync for the full mechanism.

Testing Custom Function Paths

slime also provides CPU-only contract tests for customization interfaces. These tests resolve components through import-path strings, so they can validate both built-in hooks and user-defined implementations passed through the same CLI arguments used by training.

The tests live under tests/plugin_contracts/ and are grouped by hook shape:

- tests/plugin_contracts/test_plugin_rollout_contracts.py
Covers --rollout-function-path
- tests/plugin_contracts/test_plugin_generate_contracts.py
Covers --custom-generate-function-path
- tests/plugin_contracts/test_plugin_path_loading_contracts.py
Covers --eval-function-path, --custom-rm-path, --dynamic-sampling-filter-path, --buffer-filter-path, --data-source-path, --rollout-sample-filter-path, and --rollout-all-samples-process-path
- tests/plugin_contracts/test_plugin_runtime_hook_contracts.py
Covers --custom-rollout-log-function-path, --custom-eval-rollout-log-function-path, --custom-reward-post-process-path, --custom-convert-samples-to-train-data-path, and --rollout-data-postprocess-path

Run all customization contract tests locally:

bash

python -m pytest \
  tests/plugin_contracts/test_plugin_rollout_contracts.py \
  tests/plugin_contracts/test_plugin_generate_contracts.py \
  tests/plugin_contracts/test_plugin_path_loading_contracts.py \
  tests/plugin_contracts/test_plugin_runtime_hook_contracts.py

Each test file can also be executed directly with python tests/plugin_contracts/<file>.py, which keeps them compatible with run-ci-changed.

A dedicated run-ci-cpu-unittest CI label is also available. Adding it to a PR triggers the CPU-only unit-test job, which runs the contract tests plus other lightweight unit tests in parallel (no GPU required).

For user-defined implementations, you can either export environment variables such as SLIME_CONTRACT_ROLLOUT_FUNCTION_PATH and SLIME_CONTRACT_CUSTOM_RM_PATH, or pass overrides directly when running a test file, for example:

bash

python tests/plugin_contracts/test_plugin_rollout_contracts.py \
  --rollout-function-path my_project.custom_rollout.generate_rollout

To validate your own custom implementation, replace the plugin paths used in these tests with your module path and keep the same assertions on signatures, return structure, and side effects.

---

En/Get Started/Qa

FAQ

1. Why do I see garbled text during training?

This situation generally occurs because Megatron is not loaded correctly. Please check if there is a corresponding checkpoint in the directory specified by --load or --ref-load. Note that Megatron can only load a directory that contains a latest_checkpointed_iteration.txt file.

If you need to specify a particular iteration, you can refer to the current Megatron usage instructions. Generally, you can specify the step number using --ckpt-step.

2. Why is my task stuck on the Ray submission page?

Please check whether your task is set up for co-located training and inference or decoupled training and inference.

If it's co-located (training and inference share the same GPUs), please check:

* Whether the --colocate parameter is set to enable co-located mode.
Whether the total number of GPUs for the current task is greater than or equal to actor_num_nodes actor_num_gpus_per_node.

If it's decoupled, please check:

Whether the total number of GPUs for the current task is greater than or equal to actor_num_nodes actor_num_gpus_per_node + rollout_num_gpus.

3. Why did I encounter an Out-of-Memory (OOM) error during training? What is max_tokens_per_gpu for?

OOM errors often happen because max_tokens_per_gpu is set too high. This parameter defines the maximum number of tokens that can be processed on each GPU during training. If you are concerned about OOM, you can initially set this value to rollout_max_response_len / cp_size and then increase it later to improve training efficiency. Note that --max-tokens-per-gpu is only active when --use-dynamic-batch-size is enabled.

If you still experience OOM with a small max_tokens_per_gpu, check if the data generated in a single pass is too long. You may need to enable context parallelism (CP) with --context-parallel-size. If you are using custom data generation, check if the total length of multi-turn generations is much longer than expected.

4. During multi-node training, what should I do if the transformers library reports it cannot find a model?

This usually happens when multiple processes try to read local files simultaneously using methods like AutoConfig.from_pretrained or AutoModelForCausalLM.from_pretrained, causing file system write conflicts. You can mitigate this issue by setting the --model-name argument.

5. How do I resume training?

Simply set the --load directory to your --save directory.

6. How is the batch size calculated?

A single rollout uses rollout_batch_size prompts. For each prompt, n_samples_per_prompt samples are generated. Therefore, one rollout contains a total of rollout_batch_size * n_samples_per_prompt data entries.

You can use --num-steps-per-rollout to determine how many steps to run per rollout. This is equivalent to setting the global_batch_size to rollout_batch_size * n_samples_per_prompt // num_steps_per_rollout.

7. Does slime perform data packing / variable-length (varlen) processing?

Yes. Data packing refers to the process of concatenating samples of varying lengths during training to improve GPU utilization. slime performs this operation by default.

8. What should I do if the sglang component shows a Max retries exceeded with url: /get_model_info (Caused by NewConnectionError) error?

This issue primarily stems from port conflicts caused by multiple sglang servers running on a single machine. We are currently working with the sglang team to resolve this. A temporary workaround is to minimize the number of sglang servers on a single machine, for example, by setting tp=8.

9. My gradient norm is very high and the training crashes. What should I do?

First, ensure that your data and model are compatible. For example, if your data already uses a chat template, check if this template matches the one used by the original model. If the data is correct, please refer to our Debug Guide for a more in-depth analysis.

10. My sglang generation takes an extremely long time, GPU power is maxed out, and there's no output for a long while. Why?

Please verify that the model corresponding to --hf-checkpoint has its stop tokens configured correctly. If not, you can set them using the --rollout-stop or --rollout-stop-token-ids arguments.

11. Sglang shows an an illegal memory access was encountered error.

According to SGLang documentation, this could be an OOM error. Consider reducing the value of --sglang-mem-fraction-static.

12. A JSONDecodeError occurs related to torch compile/inductor.

This is generally an issue with the torch compiler's cache read/write operations. You can try adding "TORCHINDUCTOR_FORCE_DISABLE_CACHES": "1" to the env_vars in your Ray configuration.

13. Gradient becomes NaN or Inf during training.

You can try setting the --no-check-for-nan-in-loss-and-grad flag to skip the corresponding training steps.

---

En/Get Started/Quick Start

Quick Start

This document will guide you through setting up the environment and getting started with slime within one hour, covering environment configuration, data preparation, training startup, and key code analysis and modifications.

Basic Environment Setup

Since slime may contain temporary patches for sglang/megatron, to avoid potential environment configuration issues, we strongly recommend users to use our latest Docker image, which comes pre-configured with all dependencies.

Hardware Support

slime supports multiple NVIDIA GPU hardware platforms:

- B200 Series: Fully supported with identical setup steps as H-series GPUs
- H-Series (H100/H200): Official support with comprehensive CI testing and stable performance

Important Notes:
- Latest Docker images are compatible with both B-series and H-series GPUs without additional configuration
- Megatron backend on H-series GPUs has CI protection, thoroughly validated, recommended for production environments
- B-series basic functionality is stable and suitable for development/testing, but currently lacks CI protection
- Both hardware platforms use identical installation and startup procedures

- For scenarios where Docker is not convenient, please refer to build_conda.sh;
- For AMD support, please refer to AMD Usage Tutorial.

Pull and Start Docker Container

Please execute the following commands to pull the latest image and start an interactive container:

shell

Pull the latest image

docker pull slimerl/slime:latest
Start the container

docker run --rm --gpus all --ipc=host --shm-size=16g \
  --ulimit memlock=-1 --ulimit stack=67108864 \
  -it slimerl/slime:latest /bin/bash

Install slime

slime is already installed in the docker image. To update to the latest version, please execute the following command:

bash

Path can be adjusted according to actual situation

cd /root/slime
git pull
pip install -e . --no-deps

Model and Dataset Download

You can download required models and datasets from platforms like Hugging Face, ModelScope, etc. Here are the commands to download example resources using huggingface_hub:

bash

Download model weights (GLM-Z1-9B)

hf download zai-org/GLM-Z1-9B-0414 --local-dir /root/GLM-Z1-9B-0414
Download training dataset (dapo-math-17k)

hf download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k
Download evaluation dataset (aime-2024)

hf download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024

Model Weight Conversion

Convert from Hugging Face Format to Megatron Format

When using Megatron as the training backend, you need to first convert Hugging Face format model weights to Megatron torch_dist format.

First, load the configuration file of the target model. The slime/scripts/models directory contains configuration files for supported models. You need to source the corresponding model script to load the configuration parameters into the current environment. Here we use GLM4-9B model as an example, and it's similar for Qwen3-4B, Qwen3.5, Qwen3.6, GLM-4.7-Flash, Qwen3-30B-A3B, etc.

bash

cd /root/slime
source scripts/models/glm4-9B.sh

Next, run the conversion script. Please note the following parameters:
- --hf-checkpoint: Specify the path of the downloaded Hugging Face model weights.
- --save: Specify the save path for the converted torch_dist format weights.

bash

PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
    ${MODEL_ARGS[@]} \
    --hf-checkpoint /root/GLM-Z1-9B-0414 \
    --save /root/GLM-Z1-9B-0414_torch_dist

For larger models, you can use torchrun to start the conversion script to convert with multi-gpus or even multi-nodes.
Note: When converting the kimi-k2 model weights, you need to open config.json in the model path and change "model_type": "kimi_k2" to "model_type": "deepseek_v3".

Convert from Megatron Format to Hugging Face Format

You can use the following script to convert the saved Megatron checkpoints back to Hugging Face format:

bash

PYTHONPATH=/root/Megatron-LM python tools/convert_torch_dist_to_hf.py \
  --input-dir /path/to/torch_dist_ckpt/iter_xxx/ \
  --output-dir /root/GLM-Z1-9B-0414-iter_xxx \
  --origin-hf-dir /root/GLM-Z1-9B-0414

Note that as Megatron will do padding to embedding for better performance, it may happen that the converted embedding is not correct. In that case, please manually set --vocab-size during convertion.

Training Script and Parameter Overview

After completing the above preparation work, you can run the training script.

bash

cd /root/slime
bash scripts/run-glm4-9B.sh

We still use the run-glm4-9B.sh script as an example to briefly analyze the main parameters.

MODEL_ARGS: Model Configuration Parameters

bash

SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
source "${SCRIPT_DIR}/models/glm4-9B.sh"

This part loads model configuration from the scripts/models/glm4-9B.sh file through the source command. These configurations are all hyperparameters required by Megatron. Since Megatron cannot directly read model configuration from checkpoints, it needs to be manually specified. We provide configuration examples for some commonly used models in the scripts/models/ directory.

⚠️ Note:

Please make sure to check whether the parameters in the model configuration file (such as --rotary-base) completely match the model you are currently using. Different versions of the same model structure may use different configuration values. If you need to modify, you can directly override after source, for example:

``bash


source "${SCRIPT_DIR}/models/glm4-9B.sh"

MODEL_ARGS+=(--rotary-base 10000)

`

`CKPT_ARGS: Checkpoint and Path Parameters`

bash

CKPT_ARGS=(
   # To load tokenizer and other information, won't actually use model weight parameters from hf path
   --hf-checkpoint /root/GLM-Z1-9B-0414
   # Reference Model's Megatron format checkpoint
   --ref-load /root/GLM-Z1-9B-0414_torch_dist
   # Actor model loading path. Should typically match --save for checkpoint resumption
   # If empty or doesn't contain a valid checkpoint, loads from --ref-load instead
   --load /root/GLM-Z1-9B-0414_slime/
   # Model save path during training
   --save /root/GLM-Z1-9B-0414_slime/
   # Model save interval (steps)
   --save-interval 20
)

`ROLLOUT_ARGS: Data Generation (Rollout) Parameters`

The entire training process can be viewed as a closed loop of "Data Sampling → Weight Update".

Phase One: Data Sampling (Rollout) ---rollout-batch-size: Defines the number of Prompts for each round of sampling ---n-samples-per-prompt: Defines the number of responses generated for each Prompt (used for GRPO-like algorithms)

The product of the two determines the total number of samples generated in a single round of sampling.

Phase Two: Model Training (Training) ---global-batch-size: Defines the sample size required to execute one parameter update (optimizer.step) ---num-steps-per-rollout: Defines how many parameter updates to execute using the current sampled data (we default to 1, using on-policy training)

The product of the two determines the total number of samples consumed in a single round of training.

⚠️ The parameter update here refers to the optimizer.step() in the training phase, which is different from the weight synchronization (Weight Sync) initiated by the training engine to the inference engine.

In this process, the "output" and "consumption" of each round must be equal, following this constraint:(rollout-batch-size × n-samples-per-prompt) = (global-batch-size × num-steps-per-rollout)

- In slime, if --num-steps-per-rollout is set, --global-batch-size will be automatically set if not set, and if set, it will be validated using the above formula.

Training Process Count Control ---num-rollout: Controls the total number of execution rounds of the entire "sampling→training" loop.

bash

ROLLOUT_ARGS=(
   # Prompt dataset, JSONL format
   --prompt-data /root/dapo-math-17k/dapo-math-17k.jsonl
   --input-key prompt
   --label-key label
   # If the input_key of Prompt is in OpenAI message format, apply Chat Template
   --apply-chat-template
   # Whether to shuffle data in Rollout phase
   --rollout-shuffle
   # Reward Model type. slime has built-in multiple types, also supports custom through --custom-rm-path
   --rm-type deepscaler
   # These five parameters control the relationship between rollout and train
   --num-rollout 3000
   --rollout-batch-size 16
   --n-samples-per-prompt 8
   --num-steps-per-rollout 1
   --global-batch-size 128
   # Rollout sampling parameters
   --rollout-max-response-len 8192
   --rollout-temperature 1   # Load balancing for data collected in rollout phase. It ensures that the computational workload allocated to each training process (DP rank) is roughly equal, which may be beneficial for training speed
   --balance-data
)

`EVAL_ARGS: Evaluation Parameters`

The evaluation process inherits most of the Rollout parameters, but you can override them with the following parameters to implement evaluation strategies different from training.

bash

EVAL_ARGS=(
   # Evaluation interval (number of Rollouts)
   --eval-interval 5
   # Prompt dataset for evaluation
   --eval-prompt-data aime /root/aime-2024/aime-2024.jsonl
   # Number of samples per evaluation Prompt
   --n-samples-per-eval-prompt 16
   # Maximum response length during evaluation
   --eval-max-response-len 16384
   # Sampling parameters during evaluation
   --eval-top-p 1
)

`PERF_ARGS: Performance and Parallelism Parameters`

This part mainly contains Megatron's parallel configuration. --use-dynamic-batch-size and --max-tokens-per-gpu are slime-specific optimizations.

- --max-tokens-per-gpu: Maximum number of tokens processed per GPU. After enabling dynamic batching (use_dynamic_batch_size), the system will intelligently pack samples of varying lengths so that the total token count of each micro-batch approaches this limit, thereby improving training efficiency. If a single sample length exceeds this value, it will form an independent batch. In context parallel (CP) mode, N CP cards share the total length of N * max_tokens_per_gpu. ---use-dynamic-batch-size: Enable dynamic batching. At this time, --micro-batch-size will be ignored.

💡 Tip:


slime always trains models through data packing methods and strictly ensures that per sample loss or per token loss is correct. Therefore, enabling dynamic batch size will not affect loss calculation, and it is strongly recommended to enable it.
bash
PERF_ARGS=(
   --tensor-model-parallel-size 2
   --sequence-parallel
   --pipeline-model-parallel-size 1
   --context-parallel-size 2
   --expert-model-parallel-size 1
   --expert-tensor-parallel-size 1
   --recompute-granularity full
   --recompute-method uniform
   --recompute-num-layers 1   # --micro-batch-size 1 # This item is ignored when dynamic batching is enabled
   --use-dynamic-batch-size
   --max-tokens-per-gpu 4608
)
GRPO_ARGS: GRPO Algorithm Parameters

- --use-kl-loss: Enabling this option will load a reference model and calculate the KL divergence between the current model and the reference model as a monitoring metric. Whether KL divergence is included in the final training loss depends on the --kl-loss-coef parameter. If this parameter is set to 0, KL divergence will only be displayed as an observation metric and will not participate in loss calculation.

bash

GRPO_ARGS=(
   --advantage-estimator grpo
   --use-kl-loss
   --kl-loss-coef 0.00
   --kl-loss-type low_var_kl
   --entropy-coef 0.00
   --eps-clip 0.2
   --eps-clip-high 0.28
)

- --advantage-estimator: In addition to GRPO, slime also supports several other training algorithms, such as GSPO, Reinforce++ and Reinforce++ Baseline, and PPO. ---calculate-per-token-loss: By default, slime calculates the loss on a per-sample basis, i.e., mean(sum(sample_i) / len(sample_i)). To calculate the loss on a per-token basis, i.e., sum(sum(sample_i)) / sum(len(sample_i)), you can enable this flag. ---use-tis: Enable this setting to use TIS (Truncated Importance Sampling), which is introduced by this blog.

`OPTIMIZER_ARGS: Optimizer Parameters`

bash

OPTIMIZER_ARGS=(
   --optimizer adam
   --lr 1e-6
   --lr-decay-style constant
   --weight-decay 0.1
   --adam-beta1 0.9
   --adam-beta2 0.98
)

`SGLANG_ARGS: SGLang Service Parameters`

This part of parameters is used to configure SGLang inference service. ---rollout-num-gpus-per-engine: Basically equivalent to SGLang's tp_size. - Other SGLang parameters can be passed to slime by adding the--sglang- prefix, and slime will automatically forward them to SGLang. For example, to set SGLang's --log-level INFO parameter, just use --sglang-log-level INFO.

⚠️ Note:

slime uses sgl-router to schedule multiple SGLang Servers. Without enabling DP Attention, dp_size will be calculated through rollout-num-gpus/rollout-num-gpus-per-engine.

bash

SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 2
)

`Feature Introduction`

`Colocated Actor and Rollout`

Under the default configuration, training (Actor) and inference (Rollout) resources are specified separately. Ray allocates actor_num_nodes * actor_num_gpus_per_node GPUs to the training part and rollout_num_gpus GPUs to inference, that is, training and inference are separated. When --rollout-num-gpus is explicitly set to 0, slime still parses SGLang arguments and launches the router, but does not launch local SGLang servers.

Standard (Disaggregated) Configuration:

bash

ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 4 \
   --rollout-num-gpus 4 \
   ...


In the above configuration, Actor uses 4 cards, and Rollout also uses 4 cards, running in parallel.

Training-Inference Integration (Colocated) Configuration: To deploy training and inference on the same group of GPUs, please add the--colocate parameter. By default, this makes the number of cards for training and inference equal. You can explicitly set a different positive --rollout-num-gpus, for example to use more rollout GPUs than actor GPUs; the extra GPUs are used as rollout-only resources. If --rollout-num-gpus 0 is set explicitly, slime launches only the router and no local SGLang servers.

bash

ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 8 \
   --colocate \
   ...


At this time, training and inference will share all 8 GPUs.
⚠️ Note:

In training-inference integration mode, Megatron will occupy a certain amount of GPU memory before it can be offloaded after initialization. You need to adjust the --sglang-mem-fraction-static parameter to reduce SGLang's GPU memory usage ratio to avoid insufficient GPU memory. We usually recommend 0.8.

`Dynamic Sampling`

slime supports more complex sampling strategies, such as dynamic sampling used in DAPO. To enable this feature, you need to configure the following parameters:

bash

--over-sampling-batch-size 64 \
   --dynamic-sampling-filter-path \
     slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std

Here over_sampling_batch_size needs to be greater than rollout_batch_size, for example, configured as:

bash

--rollout-batch-size 32 \
   --n-samples-per-prompt 8 \
   --over-sampling-batch-size 64 \

Then each sampling will directly sample 64 prompts, and each prompt will be sampled 8 times. Because slime performs asynchronous sampling internally, we will successively obtain 8 responses for each prompt. When receiving responses, the function corresponding to dynamic_sampling_filter_path will be used for filtering. If it passes, these 8 pieces of data will be kept; otherwise, they will be discarded.

The filtering function check_reward_nonzero_std in the example will check whether the standard deviation of rewards for a group of samples is greater than zero, ensuring that the reward scores of each group of samples left have differences, thereby avoiding overly homogeneous data and improving data diversity.

python

def check_reward_nonzero_std(args, samples: list[Sample], kwargs):
    rewards = [sample.get_reward_value(args) for sample in samples]
    keep = torch.tensor(rewards, dtype=torch.float).std() > 0.0
    return DynamicFilterOutput(
        keep=keep,
        reason=None if keep else f"zero_std_{round(rewards[0], 1)}",
    )

If the filtering function is very strict, causing a large number of prompt groups to be discarded, the system will monitor the number of pending tasks in remaining_batch_size. Once the number of pending tasks drops below the target number (32) due to too many being discarded, the system will automatically trigger a new round of oversampling, requesting over_sampling_batch_size (64) new prompts again to repeat the above process.

`Partial Rollout`

During dynamic sampling, a large number of requests may be aborted early, causing waste of computational resources. By enabling the --partial-rollout parameter, these half-generated samples can be cached and continued to be generated in the next Rollout phase, thereby improving performance.

You can also customize the strategy for extracting data from the cache through --buffer-filter-path. The default strategy is pop_first, which extracts the required number of samples in first-in-first-out order.

python

def pop_first(args, rollout_id, buffer: list[list[Sample]], num_samples: int) -> list[list[Sample]]:
    num_to_pop = min(len(buffer), num_samples)
    samples = buffer[:num_to_pop]
    del buffer[:num_to_pop]
    return samples

That is, take out the first num_samples prompts corresponding to num_samples * n_samples_per_prompt pieces of data each time.

💡 Tip:

The sample.metadata of each partial rollout sample stores the rollout id of the first generation, which can be used for data filtering.

`bf16 Training fp8 Inference`

slime directly supports bf16 training and fp8 inference. For Qwen3-4B model, you only need to download the following model:

bash

hf download Qwen/Qwen3-4B-FP8 --local-dir /root/Qwen3-4B-FP8

And replace --hf-checkpoint with:

bash

# Used to load tokenizer and other information, actually won't use model weight parameters from hf path
   --hf-checkpoint /root/Qwen3-4B-FP8   # The megatron checkpoint still needs to be the dist weights converted from bf16 huggingface at the beginning, not modified because of FP8 rollout.
   --ref-load /root/Qwen3-4B_torch_dist

This will trigger fp8 inference. Currently, we will directly cast bf16 weights to fp8, and we will gradually add quantization schemes with less impact on accuracy in the future.

For long-context rollout, you can also enable FP8 KV cache in SGLang to increase effective KV cache capacity:

bash

--sglang-kv-cache-dtype fp8_e4m3

⚠️ The training megatron checkpoint still needs to be the one converted from bf16 huggingface at the beginning.

`Multiturn Adaptation`

The slime framework is highly extensible and supports complex Agent scenarios (such as multi-turn interaction and tool calling). Its core mechanism is to rewrite the default data generation (Rollout) and reward calculation (Reward) logic through custom functions.

This section uses an implementation based on Search-R1 as an example to illustrate how to adapt slime to support multi-turn interaction.

`Adaptation Strategy Summary`

Adapting slime to support multi-turn interaction mainly includes three steps:

1. Data Preparation: Adapt the multi-turn interaction dataset to slime's Sample objects. Map conversation history, real labels, etc. to prompt and label fields, and store additional information such as tool definitions and intermediate states in the metadata field for subsequent function calls.

2. Implement Custom Generation Function: Write functions to simulate the interaction loop of "model generates action → executes tool → concatenates observation results", and correctly handle Loss Masking.

3. Implement Custom Reward Function: Write functions to evaluate complete interaction trajectories and return final reward scores.

`Data Preparation and Mapping`

To pass complex contextual information to custom functions, you need to aggregate all relevant additional fields during the data preprocessing stage.

Core Idea: Merge all additional information in the dataset except prompt and label (such as session_id, user_profile, tool_code, etc.) to construct a single, structured field (for example, a column named metadata with JSON string content).

`Step One: Construct` metadata `Field in Dataset`

Before training starts, you need to process the original dataset. For example, your original data might be as follows:

| question | final_answer | session_id | tool_code | | :--- | :--- | :--- | :--- | | "..." | "..." | "sess_123" | "code_A" |

You need to convert it to:

| question | final_answer | metadata | | :--- | :--- | :--- | | "..." | "..." |{"session_id": "sess_123", "tool_code": "code_A"} |

`Step Two: Specify Mapping in Training Script`

After completing data preparation, in the training script, map this preprocessed metadata column to slime's Sample.metadata field through ROLLOUT_ARGS.

bash

ROLLOUT_ARGS=(
   # 1. Specify the preprocessed dataset file
   --prompt-data /root/nq_search/train_processed.json
   # 2. Map "question" column to input prompt
   --input-key question
   # 3. Map "final_answer" column to evaluation label
   --label-key final_answer   # 4. Load the pre-constructed "metadata" column into Sample.metadata
   #    slime will automatically parse it as a Python dictionary
   --metadata-key metadata
)

Through this approach, you can easily access all pre-prepared structured information through methods like sample.metadata['session_id'] in custom generate or reward functions.

`Writing Custom Generation Function`

First, specify a custom asynchronous Python function through the --custom-generate-function-path parameter.

Function Signature: async def generate(args, sample: Sample, sampling_params) -> Sample:

Core Implementation Points:

1. Build Interaction Loop: Create a loop to control maximum interaction rounds (such as for _ in range(max_turns):). 2. Call Model to Generate Action: In each round of the loop, call SGLang service to let the model generate the next action (such as<search>query</search>) based on the current conversation history. 3. Parse and Execute Action: Parse model output, identify actions and parameters, and call external tools or APIs (such as Google search). 4. Build Observation Results: Format the results returned by tools and append them to the conversation history as input for the next round. 5. Handle Loss Masking: This is the key to Agent training. - Note:loss_mask should be the same length as response, where tokens that need to calculate loss are 1, and masked ones are 0 - Model-generated tokens (such as thinking, action instructions) → setloss_mask to 1, participate in loss calculation. - Tool or environment returned tokens (such as API results) → setloss_mask to 0, do not participate in loss calculation. 6. Termination Conditions: End the loop when the model generates termination tags (such as<answer>...) or reaches maximum rounds. 7. Encapsulate Return: Fill the complete interaction history, token IDs, andloss_masks into the Sample object and return.

Code Example (Pseudocode):

python

async def generate(args, sample: Sample, sampling_params) -> Sample:
    # ... initialization ...
    prompt, full_response, loss_masks = sample.prompt, "", []
    for _ in range(max_turns):
        # 1. Model generates action
        model_output = await call_sglang(prompt + full_response, ...)
        # ... tokenization and appending ...
        loss_masks += [1] * len(model_tokens) # loss_mask = 1
        full_response += model_output
        # 2. Parse and execute action
        action, content = parse_action(model_output)
        if action == "search":
            # 3 & 4. Get and append observation results
            tool_output = await google_search(content)
            # ... tokenization and appending ...
            loss_masks += [0] * len(tool_tokens) # loss_mask = 0
            full_response += tool_output
        elif action == "answer":
            break # end loop    # 7. Fill and return Sample object
    sample.response = full_response
    sample.tokens = ...
    sample.loss_mask = loss_masks
    return sample

`Writing Custom Reward Function`

Similarly, specify a custom reward function through --custom-rm-path.

Function Signature: async def reward_func(args, sample: Sample, kwargs) -> float:

This function receives a complete Sample object and calculates scores based on the final interaction results. You can implement custom scoring logic here or call external Reward Model services.

`Configure in Training Script`

Finally, in the training script, enable the above custom functions through the following parameters:

bash

CUSTOM_ARGS=(
   # Specify the path of custom generation function (format: path.to.your.file:function_name)
   --custom-generate-function-path your_module.multiturn_logic.generate   # Specify the path of custom reward function
   --custom-rm-path your_module.multiturn_logic.reward_func
)

`Multi-Node Training for Large-Scale MOE Models`

To start a multi-node task, you need to first start a Ray cluster. On node 0, run:

bash

Node0 (HEAD)

ray start --head --node-ip-address ${MASTER_ADDR} \
  --num-gpus 8 --disable-usage-stats
Other Nodes

ray start --address=${MASTER_ADDR}:6379 --num-gpus 8

After the Ray cluster has started, you can submit a job from node 0, for example:

bash

ray job submit --address="http://127.0.0.1:8265" \
   --runtime-env-json='{
     "env_vars": {
        "PYTHONPATH": "/root/Megatron-LM/",
        ... # e.g., no_proxy, API variables, etc.
     }
   }' \
   -- python3 train.py \
   --... # Other Megatron/SGLang/slime arguments

Optionally, the following environment variables may be needed based on your environment. For example, when there are multiple IPs and the wrong one is chosen in a Docker or SLURM envionment. We provide an example used in a SLURM + enroot multi-node system as follows:

text

export SLIME_HOST_IP=$(hostname -I | awk '{print $1}')
export GLOO_SOCKET_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\\./ {print $2}')
export NCCL_SOCKET_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\\./ {print $2}')
export NVSHMEM_BOOTSTRAP_UID_SOCK_IFNAME=$(ip -o -4 addr show | awk '$4 ~ /^10\./ {print $2}')

slime has been deeply optimized for distributed training of large-scale Mixture of Experts (MoE) models. We provide some end-to-end training cases for reference:

- Example: 8xH100 Training GLM-4.7-Flash - Example: 32xH100 Training GLM-5.2 - Example: 64xH100 Training GLM-4.7 - Example: 128xH100 Training DeepSeek-R1 - Scripts such asscripts/run_qwen3_30b_a3b.py and scripts/run_glm45_355b_a32b.py also support multi-node training. Their documentation is still being expanded.

---

`En/Get Started/Usage`

`Usage Guide`

`Introduction to slime Parameters`

When using slime, parameters are primarily passed for the following purposes:

1. To allocate a portion of the GPUs in the cluster for training and another portion for inference. 2. To load Megatron for the training portion. 3. To load SGLang for the inference portion. 4. To configure the hyperparameters required for RL training.

Following this order, we need to configure these parameters:

`Cluster Resource Allocation`

There are four main parameters for cluster resource allocation:

- --actor-num-nodes: The number of nodes required for RL actor training. ---actor-num-gpus-per-node: The number of GPUs per node for RL actor training. ---rollout-num-gpus: The total number of GPUs required for rollout (inference). Set it to 0to still parse SGLang arguments and launch the router without launching local SGLang servers. ---rollout-num-gpus-per-engine: The number of GPUs per inference engine. This parameter is similar to SGLang's tp_size. When performing multi-node serving, this value should be the total number of GPUs. For example, if serving one model with 2 nodes and 16 GPUs, this value should be 16. The reason for not using a parameter like--sglang-tp-size is that we might consider supporting SGLang's dp_size parameter in the future, which means an engine could contain multiple SGLang servers (currently, only --sglang-dp-size under the --sglang-enable-dp-attention condition is supported).

With the default configuration, we use these parameters to allocate actor_num_nodes * actor_num_gpus_per_node GPUs for training and rollout_num_gpus GPUs for inference via Ray, thus achieving a separation of training and inference resources.

For co-located training and inference, you also need to configure:

- --colocate: Enables co-located training and inference. By default, this makes the number of GPUs for training and inference equal. You can explicitly set a different positive --rollout-num-gpus, for example to use more rollout GPUs than actor GPUs; the extra GPUs are used as rollout-only resources. If --rollout-num-gpus 0 is set explicitly, slime launches only the router and no local SGLang servers.

Additionally, slime supports Prefill and Decode disaggregation (PD Disaggregation). You can set the number of servers used for Prefill by setting the --prefill-num-servers argument.

`Choosing Training Backend`

slime supports multiple training backends, which can be selected via the --train-backend parameter:

- megatron (default): Uses Megatron-LM as the training backend, supporting efficient training of large-scale models.

`Loading Megatron`

Unlike tools such as SGLang, vLLM, or Hugging Face Trainer, Megatron cannot directly read Hugging Face checkpoints. Instead, the user must configure the parameters for the model to be trained and load Megatron's own checkpoint format.

Generally, we need to perform three preparatory steps:

- Configure model parameters. - Configure parallelism and other optimizations. - Configure the checkpoint to be loaded.

For details on some of Megatron's customizations and the principles behind how slime incorporates Megatron, please see the "How to Use Megatron" section.

#### Configuring Model Parameters

Taking qwen3 4B as an example, we need these parameters:

bash

MODEL_ARGS=(
   --num-layers 36
   --hidden-size 2560
   --ffn-hidden-size 9728
   --swiglu
   --vocab-size 151936
   --disable-bias-linear
   # attn head
   --num-attention-heads 32
   --group-query-attention
   --num-query-groups 8
   --kv-channels 128
   --qk-layernorm
   # norm
   --normalization "RMSNorm"
   --norm-epsilon 1e-6
   # rope
   --use-rotary-position-embeddings
   --rotary-base 1000000
)

We provide configurations for common models in scripts/models, which you can reuse directly. If you are also using Megatron for pre-training/SFT, you can directly reuse the model configurations from your pre-training/SFT setup.

Note:

- slime will load all parameters of Megatron found in the PYTHONPATH, so you can find parameters and their descriptions within the Megatron in your environment. - slime uses data packing (also known as varlen or thd) for training. There is no need to configure--seq-length or --max-positional-embedding, as these parameters do not affect the maximum context length of the trained model.

#### Setting Up Parallelism and Recomputation

Megatron is currently the most comprehensively optimized training framework. A major reason for using Megatron is to pursue its excellent performance. Here is a brief introduction to configuring Megatron's parallelism and recomputation.

- Here we list Megatron's parallelism strategies. For a more detailed discussion on the trade-offs between these strategies, please refer to more specialized discussions: ---tensor-model-parallel-size: TP ---sequence-parallel: Megatron's SP is an optimization for TP. It is recommended to always enable SP when using TP. ---pipeline-model-parallel-size: PP ---context-parallel-size: Megatron's CP, also known as sequence parallelism, generally corresponds to ring attention. ---expert-model-parallel-size: EP for MoE, where each GPU has num_experts / ep_sizeexperts. ---expert-tensor-parallel-size: Megatron supports using a different tp_sizefor the MoE experts than for other parts of the model, which we generally call ETP. - For recomputation, the following flags are commonly configured in Megatron: ---recompute-granularity: This can be set to full or selective. full means complete recomputation, while selectiverecomputes less. If not configured, no recomputation is done. ---recompute-method: uniformis generally sufficient. ---recompute-num-layers: The number of layers per group for recomputation. A value of 1 is usually fine.

#### Loading Megatron Checkpoints

Megatron supports several of its custom checkpoint formats. Here are two of the more common ones:

- The once mainstream torch format (corresponding to --ckpt-format torch). - The currently recommendedtorch_dist format (corresponding to --ckpt-format torch_dist).

The torch format is Megatron's older storage format. Its structure consists of directories like mp_rank_xxx, where each directory corresponds to the checkpoint stored by each rank under a specific parallel partitioning. Because of this, when loading a torch format checkpoint, you must ensure that the checkpoint's parallelism strategy matches that of the training task.

We recommend using the torch_dist format because it supports automatic parallel sharding, meaning that training tasks with different parallelism settings can share the same checkpoint, which is much more convenient. torch_dist is also the default format in the open-source Megatron. A torch_dist format checkpoint typically contains a set of .distcp files. When using torch_dist, you can convert from Hugging Face to torch_dist and vice versa using the checkpoint conversion method described in the README.

In terms of storage structure, a Megatron checkpoint typically looks like this, assuming the storage path is /ckpt/:

bash

--/ckpt/
    |-- latest_checkpointed_iteration.txt
    |-- iter_0000100/
         |-- _0_0.distcp
         |-- _0_1.distcp
         |-- ...
    |-- iter_0000200/
    |-- iter_0000300/
    |-- ...

The latest_checkpointed_iteration.txt file records the latest training step. When loading a model, you should not directly pass /ckpt/iter_xxxxxxx, but rather pass /ckpt/ and use --ckpt-step to select the corresponding training step (if --ckpt-step is not used, the step will be read from latest_checkpointed_iteration.txt).

When using slime, there are three parameters for loading and saving checkpoints:

- --ref-load: The Megatron checkpoint for the reference model. ---load: The Megatron checkpoint for the actor. If --load is not set, or if the specified directory does not exist or does not contain latest_checkpointed_iteration.txt, the actor will be initialized from the --ref-loadcheckpoint. ---save: The path where the actor's checkpoints are saved.

Note:

- Regardless of the checkpoint storage method (i.e., however --ckpt-format is set), Megatron can load both torch and torch_dist formats.

`Loading SGLang`

Loading SGLang is very simple. You only need:

- --hf-checkpoint: The Hugging Face checkpoint used to initialize SGLang.

Note:

- Before the first training step, slime will synchronize the parameters from Megatron to SGLang. Therefore, the --hf-checkpointdoes not need to contain the latest training parameters, and you do not need to change the HF checkpoint when resuming training. - By default, SGLang reads the maximum context length from theconfig.json in the Hugging Face checkpoint. You can use the --sglang-context-lengthparameter to override this value to support longer inference. - During co-located training and inference, although Megatron and SGLang will offload sequentially, they still need to leave some memory for each other. You need to adjust SGLang's total VRAM usage by reducing--sglang-mem-fraction-static. - slime supports passing through sgl-router parameters by adding arouter prefix to the original parameter name. For example, sgl-router's --balance-abs-threshold parameter should be set as --router-balance-abs-threshold. Since sgl-router uses cache-aware routing by default, it may cause uneven request distribution. You can set --router-balance-abs-threshold 0to force balanced distribution, but this may affect prefix cache hit rate in multi-turn conversation scenarios. - If SGLang engines are pre-launched by an external system, connect to them with--rollout-external-engine-addrs host1:port host2:port. When the trainer and engines cannot form an NCCL weight-update group, use --update-weight-mode full --update-weight-transport disk --update-weight-disk-dir /shared/fs/updates; slime writes a complete HF checkpoint and asks SGLang to hot-load it through update_weights_from_disk. For large models or cross-cluster deployments, use --update-weight-mode delta --update-weight-transport disk instead. See External Rollout Engines Roadmap and Delta Weight Sync.

For details on some of SGLang's customizations and the principles behind how slime incorporates SGLang, please see the "How to Use SGLang" section.

`Data Format`

Currently, slime only supports loading files in .jsonl format, where each line of the file is a JSON object. An example of a single data entry (expanded) is as follows:

json

{
  "prompt": [
    {
      "content": "Solve the following math problem step by step. The last line of your response should be of the form Answer: \\boxed{$Answer} where $Answer is the answer to the problem.\n\nIn triangle $ABC$, $\\sin \\angle A = \\frac{4}{5}$ and $\\angle A < 90^\\circ$. Let $D$ be a point outside triangle $ABC$ such that $\\angle BAD = \\angle DAC$ and $\\angle BDC = 90^\\circ$. Suppose that $AD = 1$ and that $\\frac{BD}{CD} = \\frac{3}{2}$. If $AB + AC$ can be expressed in the form $\\frac{a\\sqrt{b}}{c}$ where $a, b, c$ are pairwise relatively prime integers, find $a + b + c$.\n\nRemember to put your answer on its own line after \"Answer:\".",
      "role": "user",
      "step_loss_mask": 1,
    }
  ],
  "label": "34"
}

This corresponds to the following configuration:

bash

--input-key prompt
  --label-key label
  --apply-chat-template

Please note that the step_loss_mask (default=1) here is for SFT phase. If it is set to 0, the turn will not contibute to the final loss; if it is set to 1, slime will use the normal loss_mask. Additionally, we provide ametadata_key, which defaults to "metadata". When read, slime will load the metadata from the data, which can be helpful for custom data generation or creating custom reward models.

If one run mixes multiple data sources, put source_name in the sample metadata:

json

{
  "prompt": "...",
  "label": "...",
  "metadata": {
    "source_name": "math"
  }
}

The recommended contract is to put the source identifier in metadata["source_name"]; slime also recognizes a dynamically set sample.source from custom data sources. When rollout samples are converted to training data, slime carries one source_names entry per sample to the training side. The source lookup order is dynamic sample.source, then metadata["source_name"]; if neither is set, the source is "unknown". This is useful for custom rewards, filters, logging, and future per-source routing such as OPD teacher selection.

`Hyperparameters for RL Training`

- --advantage-estimator: Specifies the RL algorithm for the training process. Currently supported algorithms include: -grpo(https://arxiv.org/abs/2402.03300) -gspo(https://arxiv.org/abs/2507.18071) -cispo(https://arxiv.org/abs/2506.13585) -reinforce_plus_plus and reinforce_plus_plus_baseline(https://arxiv.org/abs/2501.03262) -ppo (https://arxiv.org/abs/1707.06347)

Note: On-policy distillation (OPD) is now orthogonal to the advantage estimator. Use --use-opd and --opd-kl-coefto enable OPD on top of any estimator. ---calculate-per-token-loss: By default, slime calculates loss on a per-sample basis, i.e., mean(sum(sample_i) / len(sample_i)). Enable this flag to calculate loss on a per-token basis, i.e., sum(sum(sample_i)) / sum(len(sample_i)). ---use-tis: Enable this setting to use TIS (Truncated Importance Sampling) (https://fengyao.notion.site/off-policy-rl).

#### GRPO Algorithm

GRPO (Group Relative Policy Optimization) is an RL algorithm proposed in DeepSeek-Math. Its core idea is to compute advantage through intra-group relative comparisons, eliminating the need for a separate critic model.

To use GRPO, set:

bash

--advantage-estimator grpo

Key features of GRPO:

- No Critic Model Required: GRPO samples multiple responses for the same prompt and estimates advantage by computing relative rewards within the group, avoiding the overhead of training and maintaining a critic model. - Resource Efficient: Since no critic model is needed, GPU resources can be fully utilized for actor training and inference. - Simple to Use: Easy configuration - just set--advantage-estimator grpo.

Related parameters:

- --n-samples-per-prompt: Number of responses sampled per prompt for intra-group comparison. ---normalize-advantages: Whether to normalize advantages. ---eps-clip: PPO-style clip range.

#### PPO Algorithm

PPO (Proximal Policy Optimization) is a classic RL algorithm that uses a critic model to estimate the value function for computing advantages.

To use PPO, set:

bash

--advantage-estimator ppo

Note: In PPO, the critic and actor share the same training GPU group. You do not need to reserve a separate set of GPUs for the critic. Specifically:

- PPO creates separate actor and critic training process groups, but places them on the same train placement group. - The critic training scale follows the actor configuration, and the actor / critic Megatron parallel topology must currently stay identical. - PPO forces train-side offload so that actor and critic can wake up and release memory on the same GPUs in turn. - There are currently no separate CLI arguments for configuring critic training resources; the critic node count and GPUs per node are derived from the actor configuration.

PPO-related parameters:

- --megatron-config-path: YAML config for role-specific Megatron overrides, such as setting critic-specific load, save, lr, or warmup parameters. ---num-critic-only-steps: Number of steps to train only the critic at the beginning of training. ---eps-clip: PPO clip range. ---value-clip: Clip range for value loss. ---kl-coef: KL penalty coefficient for reward shaping.

`Advanced Megatron Configuration (--megatron-config-path)`

For PPO workflows, you can use --megatron-config-path with a YAML file to override Megatron arguments separately for actor and critic. Common use cases include setting a different critic lr, or giving actor and critic different load / save paths.

yaml

megatron:
  - name: default
    role: actor
    overrides:
      lr: 1e-6
  - name: default
    role: critic
    overrides:
      lr: 1e-5

Note: This configuration currently only supports PPO, and in current PPO the actor and critic must use the same Megatron parallel topology. The recommended pattern is to keep parallelism-related settings in the shared CLI arguments and put only role-specific differences in YAML. See Megatron Config: Role-Based Training Overrides for details.

`Custom Rollout Function`

slime supports customizing data generation (rollout) to various degrees.

- By default, it uses the generate_rollout function from slime/rollout/sglang_rollout.py for data generation. This file implements an asynchronous (asyncio) data generation flow based on SGLang and supports features like dynamic sampling and partial rollout.

- You can completely replace the generate_rollout in sglang\_example.py by using the --rollout-function-path parameter. You just need to ensure that the function signature passed via --rollout-function-path is as follows:

python

def generate_rollout(args, rollout_id, data_source, evaluation=False) -> RolloutFnTrainOutput | RolloutFnEvalOutput:
        """
        Args:
            args: the whole args
            rollout_id: int, the id of the rollout, used for deterministic data generation
            data_source: the data source to get and store samples
            evaluation: bool, whether the rollout is for evaluation or not
        
        Returns:
            RolloutFnTrainOutput | RolloutFnEvalOutput: the output of the rollout
        """
            ...
            return output

Where:

- args: The complete arguments used for the slime run.

- rollout_id: The ID of the current data generation round, used to ensure data order when resuming training.

- data_source: A globally unique data source in slime, which can be used to get initial prompts, data IDs, and store partially generated samples for later use.

- evaluation: A boolean indicating if the rollout is for evaluation. You can configure a separate evaluation function using --eval-function-path.

- The returned Sample type is defined in slime/utils/types.py. When implementing, you need to ensure the following fields are correctly set:

- tokens: The tokens for the prompt + response. -response_length: The total length of the response. For multi-turn tasks, this is the length of the tokens remaining after the first-turn prompt. -reward: The reward for this data sample. -status: The status of this data sample (e.g., Sample.Status.COMPLETED, Sample.Status.TRUNCATED, Sample.Status.ABORTED, Sample.Status.FAILED). -loss_mask should be the same length as response_length, with 1 for tokens that should be included in the loss calculation and 0 for those that should be masked out.

- In some cases, you may only need to replace the data generation logic. You can do this using --custom-generate-function-path. A simplified implementation of this function is as follows:

python

async def generate(args, sample: Sample, sampling_params) -> Sample:
        global TOKENIZER
        if TOKENIZER is None:
            TOKENIZER = AutoTokenizer.from_pretrained(args.hf_checkpoint, trust_remote_code=True)
        # send request to router
        output = await post(
            f"http://{args.sglang_router_ip}:{args.sglang_router_port}/generate",
            {
                "text": sample.prompt,
                "sampling_params": sampling_params,
            }
        )
        prompt_tokens_ids = TOKENIZER(sample.prompt, add_special_tokens=False)["input_ids"]
        response_token_ids = TOKENIZER(output["text"], add_special_tokens=False)["input_ids"]
        # set sample
        sample.tokens = prompt_tokens_ids + response_token_ids
        sample.response_length = len(response_token_ids)
        finish_reason = output["meta_info"]["finish_reason"]["type"]
        if finish_reason == "length":
            sample.status = Sample.Status.TRUNCATED
        elif finish_reason == "abort":
            sample.status = Sample.Status.ABORTED
        else:
            sample.status = Sample.Status.COMPLETED
        sample.response = output["text"]        return sample

For a more complete version, please refer to slime/rollout/sglang_rollout.py.

- Sometimes, you may also need to support a custom reward model. This can be configured by setting --custom-rm-path.

`How to Use SGLang`

slime implements a server-based engine using SGLang via the HttpServerEngineAdapter as an intermediary.

`Parameter Configuration`

slime incorporates almost all SGLang parameters by using SGLang's ServerArgs.add_cli_args. When setting an SGLang parameter, you need to add the --sglang- prefix. For example:

- In co-located training and inference, you often need to limit --mem-fraction-static. This parameter should be changed to --sglang-mem-fraction-static. - During training, if you want SGLang to infer beyond the maximum context length specified in the Hugging Face checkpoint'sconfig.json, you need to use --context-length, which becomes --sglang-context-lengthin slime. - For multi-node large EP inference, you might need--ep-size, --enable-dp-attention, --dp-size, --moe-a2a-backend deepep, etc. These can be passed as --sglang-ep-size, --sglang-enable-dp-attention, --sglang-dp-size, and --sglang-moe-a2a-backend deepep respectively.

Some parameters related to slime's resource scheduling are configured by slime itself, for example:

- --tp-size in slime is set using --rollout-num-gpus-per-engine. ---model-path in slime is set using --hf-checkpoint.

The way SGLang parameters are integrated into slime can be found in slime/backends/sglang_utils/arguments.py.

`How to Use the Router`

slime uses sglang-router to manage the SGLang servers during the training process. You can configure the address of the sglang-router using --sglang-router-ip and --sglang-router-port. If not configured, a router will be started by default within the cluster.

After starting, all SGLang servers will register with the router via the /add_worker endpoint. When actually generating data, you only need to send HTTP requests to the router, which will perform load balancing and forward the requests to the servers.

When you configure an external router using --sglang-router-ip and --sglang-router-port, slime will not start an internal router. Instead, it will register all its servers with this external router. You can then use this external router's address to implement more complex data generation workflows. Note that the router supports OpenAI-compatible APIs.

`Advanced Engine Configuration (--sglang-config)`

For advanced deployments, you can use --sglang-config with a YAML file to configure server groups, multi-model serving, and selective weight updates.

Multi-model deployment allows serving multiple models simultaneously (e.g., an actor model that receives weight updates and a frozen reference/reward model):

yaml

sglang:
  - name: actor
    update_weights: true          # receives weight updates from training (default)
    server_groups:
      - worker_type: regular
        num_gpus: 8
        num_gpus_per_engine: 4
  - name: ref
    model_path: /path/to/ref_model
    update_weights: false          # frozen, no weight updates
    server_groups:
      - worker_type: regular
        num_gpus: 4
        num_gpus_per_engine: 2

Each model gets its own router. The per-model router info is accessible via args.sglang_model_routers (a dict mapping model name to (ip, port) tuples). Custom rollout functions can use get_model_url(args, "ref") from slime.rollout.sglang_rollout to route requests to a specific model.

Server group features: -worker_type: regular, prefill, decode, or placeholder(reserves GPU slots without creating engines) -overrides: Dict of SGLang ServerArgs field overrides applied on top of --sglang-*CLI args -num_gpus_per_engine: Per-group TP size override

`How to Use Megatron`

slime supports different and lightly modified versions of Megatron by reusing common functions from the megatron.training directory, such as parse_args, save_checkpoint, and load_checkpoint. Therefore, when using it, you must ensure that Megatron is accessible in the PYTHONPATH, for example, by adding export PYTHONPATH=/root/Megatron-LM at runtime.

`Parameter Configuration`

slime directly imports all parameters of the Megatron in the current environment by using from megatron.training.arguments import parse_args. If the version of Megatron you are using has parameters defined outside of parse_args, you can configure them by passing them in, similar to how it's done in train.py, for example:

python

if __name__ == "__main__":
    try:
        from pretrain_gpt import extra_args_provider
    except:
        extra_args_provider = None
    args = parse_args(extra_args_provider)
    train(args)

`Custom Parameters`

In some customized Megatron implementations, special operations need to be performed during initialization or before/after a training step. We have added the following plugins for this purpose:

- --custom-megatron-init-path: Adds some initialization calls. ---custom-megatron-before-log-prob-hook-path: Is called before calculating the log probability. ---custom-megatron-before-train-step-hook-path: Is called before each training step. You could use this to mix in special training losses, for example.

---

`En/Examples/Deepseek R1`

`DeepSeek R1 with 128xH100`

This is an example of doing DeepSeek R1 RL training using 128xH100 GPUs.

We will use bf16 for training, and an fp8 format with 128x128 blockwise quantization for inference. The maximum response length is 32k, and dynamic sampling will be used to filter data during training.

Regarding parallelism, for sglang we will enable EP64, activate dp attention, and deepep. For the Megatron part, we will use TP8, PP4, EP32, and CP4.

⚠️ To save GPU memory, we will use CPU Adam. Each node (8xH100) will occupy 1.4\~1.5TB of host memory. If a single machine's host memory is insufficient, this can be resolved by adding more GPUs to expand the parallelism.

`Environment Setup`

For instructions on setting up the environment and downloading data, please refer to Example: Qwen3-4B.

To prepare the DeepSeek R1 checkpoint, first you will need to download DeepSeek-R1 to a directory accessible by all machines (hereinafter referred to as $BASE_DIR):

bash

hf download deepseek-ai/DeepSeek-R1 --local-dir $BASE_DIR/DeepSeek-R1

The Hugging Face checkpoint for DeepSeek-R1 is in a block-quantized fp8 format. To convert it into a torch_dist format that Megatron can load, you first need to convert it to a bf16 Hugging Face checkpoint:

bash

cd slime/
python tools/fp8_cast_bf16.py --input-fp8-hf-path $BASE_DIR/DeepSeek-R1 --output-bf16-hf-path $BASE_DIR/DeepSeek-R1-bf16/

Next, we need to convert the bf16 version of DeepSeek-R1 into the torch_dist format. Specifically, execute the following on 4 separate nodes:

bash

cd slime/
source scripts/models/deepseek-v3.sh
PYTHONPATH=/root/Megatron-LM/ torchrun \
   --nproc-per-node 8 \
   --master-addr ${MASTER_ADDR} --master-port 12345 \
   --nnodes=4 --node-rank ${NODE_RANK} \
   tools/convert_hf_to_torch_dist.py \
   ${MODEL_ARGS[@]} \
   --tensor-model-parallel-size 1 \
   --pipeline-model-parallel-size 8 \
   --expert-tensor-parallel-size 1 \
   --expert-model-parallel-size 4 \
   --decoder-first-pipeline-num-layers 7 \
   --decoder-last-pipeline-num-layers 6 \
   --hf-checkpoint $BASE_DIR/DeepSeek-R1-bf16/ \
   --save $BASE_DIR/DeepSeek-R1_torch_dist/

Here, MASTER_ADDR is the IP of node0, and NODE_RANK indicates the node's index, both configured similarly to a multi-node torchrun setup.

`Executing the Training`

On node0, run:

bash

cd slime/
bash scripts/run-deepseek-r1.sh

On other nodes, you need to join the Ray cluster with the following command:

bash

ray start --address=${MASTER_ADDR}:6379 --num-gpus 8 --node-ip-address ${WORKER_IP} --disable-usage-stats"

Alternatively, if you have a list of all node IPs, for example, an MPI hostfile (where each line is ip slot=8), you can add the following commands after the ray start --head command in scripts/run-deepseek-r1.sh. This allows you to execute the training entirely from node0:

bash

for WORKER_IP in $(awk '{print $1}' $BASE_DIR/mpi_hostfile); do
  if [[ "$WORKER_IP" == "$MASTER_ADDR" ]]; then
    continue
  fi
  echo "Starting Ray worker on ${WORKER_IP}"
  ssh root@"${WORKER_IP}" \
    "pkill -9 sglang ; ray stop --force ; pkill -9 python ; ray start --address=${MASTER_ADDR}:6379 --num-gpus 8 --node-ip-address ${WORKER_IP} --disable-usage-stats" &
done
wait

`Parameter Introduction`

bash

SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
source "${SCRIPT_DIR}/models/deepseek-v3.sh"

This reads the model's config from scripts/models/deepseek-v3.sh. These configs are all Megatron parameters. When training with Megatron, it cannot read the model config from the checkpoint, so we need to configure it ourselves. We provide some examples in scripts/models.

#### CKPT\_ARGS

bash

CKPT_ARGS=(
   # HF ckpt required by sglang, we also read the tokenizer from here
   --hf-checkpoint $BASE_DIR/DeepSeek-R1/
   #--hf-checkpoint $BASE_DIR/DeepSeek-R1-bf16/
   --ref-load $BASE_DIR/DeepSeek-R1_torch_dist/
   # Actor's load directory, if empty, it will read from ref_load
   --load $BASE_DIR/DeepSeek-R1_slime/
   --save $BASE_DIR/DeepSeek-R1_slime/
   --save-interval 20
)

slime will perform online quantization during training based on the quantization configuration in hf_checkpoint. For instance, in the current example, we are using the fp8 checkpoint of DeepSeek R1. This means that when updating parameters, we will first perform blockwise quantization on the parameters before passing them to sglang.

#### PERF\_ARGS

A set of Megatron parallelism parameters. Only --use-dynamic-batch-size and --max-tokens-per-gpu are added by slime.

For the Megatron part, we have configured TP8, PP4, CP4, and EP32. Since DeepSeek-R1 has 61 layers, which is not divisible by 4, we have specifically configured the last pipeline stage to have 13 layers.

max_tokens_per_gpu refers to the maximum number of tokens each GPU can process. When use_dynamic_batch_size is enabled, it will pack data of varying lengths within a batch as close to max_tokens_per_gpu. If a single data item exceeds max_tokens_per_gpu, it will form its own batch without truncation. When context parallelism (CP) is enabled, it allows CP GPUs to share a total length of CP * max_tokens_per_gpu tokens.

When dynamic_batch_size is enabled, the traditional micro_batch_size is ignored.

⚠️ slime always trains the model using data packing and strictly guarantees per-sample or per-token loss. This means enabling dynamic batch size will not affect the loss calculation. It is recommended to enable it.

bash

PERF_ARGS=(
   --tensor-model-parallel-size 8
   --sequence-parallel
   --pipeline-model-parallel-size 4
   --context-parallel-size 4
   --expert-model-parallel-size 32
   --expert-tensor-parallel-size 1
   --decoder-last-pipeline-num-layers 13
   --recompute-granularity full
   --recompute-method uniform
   --recompute-num-layers 1   --use-dynamic-batch-size
   --max-tokens-per-gpu 16384
)

#### GRPO\_ARGS

Currently, these are some GRPO-related parameters in slime:

bash

GRPO_ARGS=(
   --advantage-estimator grpo
   --use-kl-loss
   --kl-loss-coef 0.00
   --kl-loss-type low_var_kl
   --entropy-coef 0.00
   --eps-clip 0.2
   --eps-clip-high 0.28
)

If you wish to train without loading the reference model, you need to remove --use-kl-loss and set --kl-coef 0.00 (the default value is 0).

#### OPTIMIZER\_ARGS

We have configured CPU Adam with the following parameters to save GPU memory.

bash

OPTIMIZER_ARGS=(
   ...   --optimizer-cpu-offload
   --overlap-cpu-optimizer-d2h-h2d
   --use-precision-aware-optimizer
)

#### SGLANG\_ARGS

These are the parameters required by sglang. Here, --rollout-num-gpus-per-engine basically corresponds to sglang's tp_size. Other sglang parameters are passed to slime by adding a --sglang- prefix. To fully leverage sglang's large EP inference capabilities, we have added configurations like ep64, dp\_attention dp8, and deepep mode auto.

The final --sglang-server-concurrency is a parameter specific to slime. It is used to prevent the sglang server's concurrent requests from becoming too large and crashing the HTTP server. The default is 512. However, since we now have one server for 8 nodes, we have adjusted it to 1024 to ensure that each dp rank can have a concurrency of 128.

bash

SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 64
   --sglang-mem-fraction-static 0.7
   ----sglang-ep-size 64
   # dp attention
   --sglang-enable-dp-attention
   --sglang-dp-size 8
   --sglang-moe-dense-tp-size 1
   --sglang-enable-dp-lm-head
   # enable deepep for sglang
   --sglang-moe-a2a-backend deepep
   --sglang-deepep-mode auto   # make every dp rank have 128 concurrency
   --sglang-server-concurrency 1024
)

#### MISC\_ARGS

Some additional Megatron configurations. Note that Megatron's deepep is configured here.

bash

MISC_ARGS=(
   ...   # use deepep for megatron
   --moe-enable-deepep
   --moe-token-dispatcher-type flex
)

---

`En/Examples/Glm4.7 30B A3B`

`GLM-4.7-Flash with 8×H100`

`Environment Preparation`

The environment setup, data, and checkpoint conversion are the same as for the Qwen3-4B model. You can refer to Example: Qwen3-4B Model, replacing mentions of Qwen3-4B with GLM-4.7-Flash.

`Download Model`

bash

hf download THUDM/GLM-4.7-Flash --local-dir /root/GLM-4.7-Flash

`Convert Checkpoint`

To convert the Hugging Face checkpoint to torch_dist format:

bash

cd /root/slime
pip install -e . --no-deps
source scripts/models/glm4.7-30B-A3B.sh
PYTHONPATH=/root/Megatron-LM/ torchrun --nproc-per-node 8 \
   tools/convert_hf_to_torch_dist.py \
   ${MODEL_ARGS[@]} \
   --hf-checkpoint /root/GLM-4.7-Flash/ \
   --save /root/GLM-4.7-Flash_torch_dist/

`Run Training`

Execute the training script:

bash

cd /root/slime
bash scripts/run-glm4.7-30B-A3B-8gpus.sh

`Parameter Introduction`

Here, we will briefly introduce the key parts in the run-glm4.7-30B-A3B-8gpus.sh script.

#### MoE Configuration

GLM-4.7-Flash is a Mixture-of-Experts (MoE) model with 64 routed experts (top-4 activation) and 1 shared expert. It has 47 layers: 1 dense layer + 46 MoE layers.

1. To support running GLM-4.7-Flash on 8×H100, we need to enable Megatron's CPU Adam to save GPU memory:

bash

OPTIMIZER_ARGS=(
       ...
       --optimizer-cpu-offload
       --overlap-cpu-optimizer-d2h-h2d
       --use-precision-aware-optimizer
    )

2. Enable MoE optimization in Megatron. For single-node 8×H100, we use TP=1, EP=8:

bash

PERF_ARGS=(
       --tensor-model-parallel-size 1
       --pipeline-model-parallel-size 1
       --context-parallel-size 1
       --expert-model-parallel-size 8
       --expert-tensor-parallel-size 1
       ...
    )

3. Enable MoE optimization in SGLang with DP attention:

bash

SGLANG_ARGS=(
       --rollout-num-gpus-per-engine 8
       --sglang-mem-fraction-static 0.7
       --sglang-enable-dp-attention
       --sglang-dp-size 8
       --sglang-enable-dp-lm-head
       --sglang-moe-dense-tp-size 1
       ...
    )

#### MTP Speculative Decoding (Inference Acceleration)

GLM-4.7-Flash includes 1 MTP (Multi-Token Prediction) layer, which can be used for speculative decoding during inference to speed up rollout generation. To enable this, add the following to SGLANG_ARGS:

bash

SGLANG_ARGS=(
   ...
   # MTP speculative decoding (EAGLE)
   --sglang-speculative-algorithm EAGLE
   --sglang-speculative-num-steps 3
   --sglang-speculative-eagle-topk 1
   --sglang-speculative-num-draft-tokens 4
)

This enables SGLang to use the model's MTP layer as a draft model for EAGLE-style speculative decoding. The MTP layer predicts multiple future tokens, and SGLang verifies them in parallel, leading to faster generation.

⚠️ Note: Speculative decoding requires additional GPU memory. If you encounter OOM issues, try reducing --sglang-mem-fraction-static or disabling speculative decoding.

#### MTP Training

slime also supports training MTP layers jointly with the main model for models that have MTP weight conversion implemented (e.g., MiMo, GLM-4.7). When enabled, the relevant arguments are:

bash

Add MTP layer count to model config

MODEL_ARGS+=(--mtp-num-layers 1)
Enable MTP training

SPEC_ARGS=(
   --enable-mtp-training
   --mtp-loss-scaling-factor 0.2
)

- --mtp-num-layers 1: Tells Megatron to load the MTP layer from the checkpoint. ---enable-mtp-training: Enables gradient computation for MTP layers. Without this flag, the MTP layer is loaded but frozen. ---mtp-loss-scaling-factor 0.2: Weight of the MTP loss relative to the main policy loss. Default is 0.2.

Note: The native DeepSeek-layout loader uses the model's configured layer count when mapping MTP weights, including GLM-4.7-Flash's 47-layer architecture.

> For other models with MTP training support (e.g., MiMo), see scripts/run-mimo-7B-rl-eagle.sh as a reference.

`Multi-Node Support`

For multi-node training (e.g., 2×8 H100), use the multi-node script:

bash

cd /root/slime
export BASE_DIR=/shared/path  # accessible by all nodes
bash scripts/run-glm4.7-30B-A3B.sh

Key modifications for multi-node:

- Place the model and data on a path accessible by all nodes. - SetMASTER_ADDRto an address accessible by all nodes. - Remove CPU Adam configurations (distributed optimizer reduces per-GPU memory usage). - Adjust parallelism: e.g., TP=4, PP=2, EP=8, CP=2.

When the total number of GPUs is not a multiple or divisor of the total number of experts (64), you can use --sglang-ep-num-redundant-experts to add redundant experts. For example, in a 24-GPU scenario:

bash

SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 24
   --sglang-mem-fraction-static 0.7
   --sglang-ep-size 24
   --sglang-enable-dp-attention
   --sglang-dp-size 3
   --sglang-moe-dense-tp-size 1
   --sglang-enable-dp-lm-head
   --sglang-ep-num-redundant-experts 16
)

---

`En/Examples/Glm4.7 355B A32B`

`GLM-4.7 with 64xH100`

`Environment Preparation`

The environment setup and dataset download are the same as for the Qwen3-4B model. You can refer to Example: Qwen3-4B Model, replacing mentions of Qwen3-4B with GLM-4.7.

`Prerequisites`

GLM-4.7 follows the standard slime Docker environment. For multi-node launches, make sure all nodes can access the same $BASE_DIR path and unset proxy variables before starting Ray workers:

bash

unset http_proxy https_proxy HTTP_PROXY HTTPS_PROXY

`Download Model`

bash

hf download zai-org/GLM-4.7 --local-dir $BASE_DIR/GLM-4.7-355B-A32B

`Convert Checkpoint`

To convert the Hugging Face checkpoint to torch_dist format, use 2 nodes x 8 GPUs:

bash

cd /root/slime
pip install -e . --no-deps
source scripts/models/glm4.5-355B-A32B.sh
PYTHONPATH=/root/Megatron-LM/ torchrun \
   --nproc-per-node 8 \
   --master-addr ${MASTER_ADDR} --master-port 12345 \
   --nnodes=2 --node-rank ${NODE_RANK} \
   tools/convert_hf_to_torch_dist.py \
   ${MODEL_ARGS[@]} \
   --hf-checkpoint $BASE_DIR/GLM-4.7-355B-A32B/ \
   --save $BASE_DIR/GLM-4.7-355B-A32B_torch_dist/

Here, MASTER_ADDR is the IP of node0, and NODE_RANK is the node index, configured just like a multi-node torchrun job.

`Run Training`

Execute the training script from node0:

bash

cd /root/slime
export BASE_DIR=/shared/path  # accessible by all nodes
bash scripts/run-glm4.7-355B-A32B.sh

`Parameter Introduction`

Here, we briefly introduce the key parts in the run-glm4.7-355B-A32B.sh script.

#### MoE Configuration

GLM-4.7 is a Mixture-of-Experts (MoE) model with 160 routed experts (top-8 activation) and shared experts. It has 92 layers: 3 dense layers + 89 MoE layers.

1. To support GLM-4.7 on 64xH100, we enable Megatron's CPU Adam to save GPU memory:

bash

OPTIMIZER_ARGS=(
      ...
      --optimizer-cpu-offload
      --overlap-cpu-optimizer-d2h-h2d
      --use-precision-aware-optimizer
   )

2. Enable MoE optimization in Megatron. For the provided 64xH100 example, we use TP=8, PP=4, CP=2, and EP=16:

bash

PERF_ARGS=(
      --tensor-model-parallel-size 8
      --sequence-parallel
      --pipeline-model-parallel-size 4
      --context-parallel-size 2
      --expert-model-parallel-size 16
      --expert-tensor-parallel-size 1
      ...
      --use-dynamic-batch-size
      --max-tokens-per-gpu 16384
   )

3. Enable MoE optimization in SGLang with DP attention:

bash

SGLANG_ARGS=(
      --rollout-num-gpus-per-engine 32
      --sglang-mem-fraction-static 0.7
      --sglang-enable-dp-attention
      --sglang-dp-size 4
      --sglang-ep-size 32
      --sglang-enable-dp-lm-head
      --sglang-moe-dense-tp-size 1
      ...
   )

#### MTP Speculative Decoding (Inference Acceleration)

GLM-4.7 includes MTP (Multi-Token Prediction) layers that can be used for speculative decoding during inference to speed up rollout generation. To enable this, add the following to SGLANG_ARGS:

bash

SGLANG_ARGS=(
   ...
   # MTP speculative decoding (EAGLE)
   --sglang-speculative-algorithm EAGLE
   --sglang-speculative-num-steps 3
   --sglang-speculative-eagle-topk 1
   --sglang-speculative-num-draft-tokens 4
)

This lets SGLang use the model's MTP layer as the draft model for EAGLE-style speculative decoding.

⚠️ Note: Speculative decoding requires additional GPU memory. If you encounter OOM issues, try reducing --sglang-mem-fraction-static or disabling speculative decoding.

#### MTP Training

slime also supports training the MTP layers jointly with the main model for GLM-4.7. When enabled, the relevant arguments are:

bash

Add MTP layer count to model config

MODEL_ARGS+=(--mtp-num-layers 1)
Enable MTP training

MTP_ARGS=(
   --enable-mtp-training
   --mtp-loss-scaling-factor 0.2
)

Note: The native loader in slime/backends/megatron_utils/hf_to_megatron/glm.py maps both regular and MTP weights.

#### Multi-Node Support

This example already targets multi-node training. Before launching:

- Place the model checkpoints and datasets on a path accessible by all nodes. - SetMASTER_ADDRto an address reachable by all nodes. - Unset proxy variables before starting Ray workers. - Provide aHOSTFILE listing worker IPs (one per line) and export HOSTFILE=/path/to/hostfilebefore launching. - Adjust parallelism coherently. The default example uses TP=8, PP=4, EP=16, CP=2, while rollout uses 32 GPUs per engine with SGLang DP attention.

If your rollout GPU count does not divide the expert count cleanly, you can use --sglang-ep-num-redundant-experts to add redundant experts.

`FP8 Rollout`

The open-source FP8 checkpoint of GLM-4.7 uses per-channel quantization, which cannot currently enable DeepEP in SGLang. You can convert it to a 128x128 per-block FP8 checkpoint with the tool provided in slime:

bash

cd /root/slime
python tools/convert_hf_to_fp8.py \
    --model-dir $BASE_DIR/GLM-4.7-355B-A32B/ \
    --save-dir $BASE_DIR/GLM-4.7-355B-A32B-FP8/ \
    --strategy block --block-size 128 128 \
    --max-workers 4

Then switch --hf-checkpoint to $BASE_DIR/GLM-4.7-355B-A32B-FP8/ to enable FP8 rollout.

An example FP8 SGLANG_ARGS setup is:

bash

SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 32
   --sglang-mem-fraction-static 0.7
   --sglang-enable-dp-attention
   --sglang-dp-size 32
   --sglang-ep-size 32
   --sglang-moe-dense-tp-size 1
   --sglang-enable-dp-lm-head
   --sglang-cuda-graph-bs 1 2 4 8 $(seq 16 8 128)
   --sglang-speculative-algorithm EAGLE
   --sglang-speculative-num-steps 3
   --sglang-speculative-eagle-topk 1
   --sglang-speculative-num-draft-tokens 4   --sglang-moe-a2a-backend deepep
   --sglang-deepep-mode auto
)

---

`En/Examples/Glm4 9B`

`GLM4-9B with 8xH100`

`Environment Setup`

After pulling the slimerl/slime:latest image, initialize the image environment as follows:

bash

cd /root/
git clone https://github.com/THUDM/slime.git
cd slime/
pip install -e . --no-deps

Download the model and data:

bash

hf checkpoint

hf download zai-org/GLM-Z1-9B-0414 --local-dir /root/GLM-Z1-9B-0414
train data

hf download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k
eval data

hf download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024

Convert the Hugging Face checkpoint to a Megatron-loadable Hugging Face checkpoint:

bash

mcore checkpoint

cd /root/slime
source scripts/models/glm4-9B.sh
PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
    ${MODEL_ARGS[@]} \
    --hf-checkpoint /root/GLM-Z1-9B-0414 \
    --save /root/GLM-Z1-9B-0414_torch_dist

`Run Training`

Execute the training:

bash

cd /root/slime
bash scripts/run-glm4-9B.sh

`Parameter Introduction`

Here, we will briefly introduce the various components of the run-glm4-9B.sh script:

#### MODEL\_ARGS

bash

SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
source "${SCRIPT_DIR}/models/glm4-9B.sh"

Reads the model's config from scripts/models/glm4-9B.sh. These configs are all Megatron parameters. When training with Megatron, it cannot read the model config from the checkpoint, so we need to configure it ourselves. We provide some examples in scripts/models.

⚠️ Ensure that settings such as --rotary-base in the model configuration file match the settings of the model you are currently training. This is because different models, even with the same architecture, might use different values. If needed, you can override these parameters in your script after loading the model weights. For instance:

bash

source "${SCRIPT_DIR}/models/glm4-9B.sh"MODEL_ARGS += ( --rotary-base 10000 )

#### CKPT\_ARGS

bash

CKPT_ARGS=(
   # HF checkpoint required by sglang; we also read the tokenizer from here
   --hf-checkpoint /root/GLM-Z1-9B-0414
   # Checkpoint for the reference model
   --ref-load /root/GLM-Z1-9B-0414_torch_dist
   # Load directory for the actor; if empty, it will be loaded from ref_load
   --load /root/GLM-Z1-9B-0414_slime/
   --save /root/GLM-Z1-9B-0414_slime/
   --save-interval 20
)

#### ROLLOUT\_ARGS

bash

ROLLOUT_ARGS=(
   # Prompt dataset, each line is a JSON object
   --prompt-data /root/dapo-math-17k/dapo-math-17k.jsonl
   --input-key prompt
   --label-key label
   # If the input_key in the prompt contains an OpenAI message,
   # tokenizer.apply_chat_template(...) will be executed
   --apply-chat-template
   # Whether to shuffle the data
   --rollout-shuffle
   # Reward model type.
   # slime provides many types and --custom-rm-path for custom models
   --rm-type deepscaler
   # Total number of rollouts to train
   --num-rollout 3000
   # Number of prompts in one rollout
   --rollout-batch-size 32
   # Number of responses to sample per prompt
   # A rollout will have rollout_batch_size * n_samples_per_prompt items
   --n-samples-per-prompt 8
   # Rollout sampling parameters
   --rollout-max-response-len 8192
   --rollout-temperature 1   # Number of training steps corresponding to one rollout
   --num-steps-per-rollout 1
   # Whether to balance data during training, which might improve speed
   --balance-data
)

#### EVAL\_ARGS

During evaluation, most rollout parameters are inherited, but we provide some parameters that can override the rollout configuration, allowing for different sampling strategies for training and evaluation.

bash

EVAL_ARGS=(
   --eval-interval 5
   --eval-prompt-data /root/aime-2024/aime-2024.jsonl
   --n-samples-per-eval-prompt 16
   --eval-max-response-len 16384
   --eval-top-p 1
)

#### PERF\_ARGS

A set of Megatron's parallelism parameters. Only --use-dynamic-batch-size and --max-tokens-per-gpu are added by slime.

max_tokens_per_gpu specifies the maximum number of tokens each GPU can process. When use_dynamic_batch_size is enabled, it will try to pack data of varying lengths within a batch up to max_tokens_per_gpu, thus forming a dynamic micro-batch size. If a single data item's length exceeds max_tokens_per_gpu, it will form its own batch without being truncated. When context parallelism (CP) is enabled, it allows the CP GPUs to share data with a total length of CP * max_tokens_per_gpu tokens.

When dynamic_batch_size is enabled, the traditional micro_batch_size is ignored.

bash

PERF_ARGS=(
   --tensor-model-parallel-size 2
   --sequence-parallel
   --pipeline-model-parallel-size 1
   --context-parallel-size 2
   --expert-model-parallel-size 1
   --expert-tensor-parallel-size 1
   --recompute-granularity full
   --recompute-method uniform
   --recompute-num-layers 1   # --micro-batch-size 1
   --use-dynamic-batch-size
   --max-tokens-per-gpu 4608
)

#### GRPO\_ARGS

Here are some GRPO-related parameters:

bash

GRPO_ARGS=(
   --advantage-estimator grpo
   --use-kl-loss
   --kl-loss-coef 0.00
   --kl-loss-type low_var_kl
   --entropy-coef 0.00
   --eps-clip 0.2
   --eps-clip-high 0.28
)

#### OPTIMIZER\_ARGS

bash

OPTIMIZER_ARGS=(
   --optimizer adam
   --lr 1e-6
   --lr-decay-style constant
   --weight-decay 0.1
   --adam-beta1 0.9
   --adam-beta2 0.98
)

#### SGLANG\_ARGS

Parameters required by sglang. Here, --rollout-num-gpus-per-engine basically corresponds to sglang's tp_size. Other sglang parameters are passed to slime by adding the --sglang- prefix.

bash

SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 2
)

⚠️ slime uses sgl-router to schedule multiple sglang servers. dp_size is not supported when DP attention is disabled.

`Co-located Training and Inference`

In the original script, the resource configuration is as follows:

bash

ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 4 \
   --rollout-num-gpus 4 \
   ...

This enables decoupled training and inference, where the training part will use 1 machine with 4 GPUs, and the inference will use another 4 GPUs.

If you want to use the co-located feature, you need to add --colocate and remove --rollout-num-gpus:

bash

ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 8 \
   --colocate \
   ...

In this case, both training and inference will share these 8 GPUs.

⚠️ When using co-located training and inference, Megatron will always occupy some GPU memory. Therefore, you need to adjust --sglang-mem-fraction-static to reduce the proportion of memory occupied by sglang.

`Dynamic Sampling`

slime supports more complex sampling schemes, such as the dynamic sampling in DAPO. To enable dynamic sampling, you need to configure:

bash

--over-sampling-batch-size ${OVER_SAMPLING_BS} \
   --dynamic-sampling-filter-path \
     slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std \

Here, over_sampling_batch_size needs to be greater than rollout_batch_size. For example:

bash

--rollout-batch-size 32 \
   --n-samples-per-prompt 8 \
   --over-sampling-batch-size 64 \

The sampling will then directly sample 64 prompts, with 8 samples per prompt. Since slime performs asynchronous sampling internally, we will receive the 8 responses for each prompt sequentially. Upon receiving responses, they will be filtered using the function specified by dynamic_sampling_filter_path. If they pass, these 8 data points are kept; otherwise, they are discarded. The function in the example checks if the answers are all correct or all incorrect:

python

def check_reward_nonzero_std(args, samples: list[Sample], kwargs):
    rewards = [sample.reward for sample in samples]
    return torch.tensor(rewards, dtype=torch.float).std() > 0.0

When we have received 32 \* 8 data points, we will immediately stop sampling and will not wait for the remaining data to be sampled. If more than 32 prompts' worth of data is discarded (leaving fewer than 32 prompts' worth), we will then sample another 64 prompts.

`Partial Rollout`

During the process of dynamic sampling, a large number of requests are aborted prematurely. We can configure the --partial-rollout parameter to save these partially generated requests to a data buffer. In the next rollout, these requests can be retrieved to continue data generation, thereby further optimizing performance.

You can customize how data is retrieved from the buffer by configuring the --buffer-filter-path. The default function is:

python

def pop_first(args, rollout_id, buffer: list[list[Sample]], num_samples: int) -> list[list[Sample]]:
    num_to_pop = min(len(buffer), num_samples)
    samples = buffer[:num_to_pop]
    del buffer[:num_to_pop]
    return samples

This means that each time, the data corresponding to the first num_samples prompts is retrieved, totaling num_samples * n_samples_per_prompt items.

⚠️ The sample.metadata of each partial rollout sample stores the rollout ID from its initial generation, which can be used for data filtering.

---

`En/Examples/Glm5.2 744B A40B`

`GLM-5.2 744B-A40B with 256xH100`

This is the recommended 32-node, 256-H100 training example for GLM-5.2.

The recipe uses the GLM-5.2 BF16 checkpoint for Megatron training and the FP8 checkpoint for SGLang rollout. It assumes two Hugging Face repositories will be available:

- BF16: zai-org/GLM-5.2- FP8:zai-org/GLM-5.2-FP8

`Environment Setup`

For environment setup and dataset download, see Example: Qwen3-4B. For multi-node training, make sure every node can access the same $BASE_DIR path.

`Download Model`

bash

hf download zai-org/GLM-5.2 --local-dir $BASE_DIR/GLM-5.2
hf download zai-org/GLM-5.2-FP8 --local-dir $BASE_DIR/GLM-5.2-FP8

The open-source GLM-5.2 config uses model_type: glm_moe_dsa, which slime maps onto the native DeepSeek-V3.2 loader since the two share the same DSA weight layout.

`Convert Checkpoint`

The training side needs the BF16 Hugging Face checkpoint converted to the Megatron torch_dist format. The torch_dist format is reshardable, so the conversion parallel layout does not need to match training; we use a layout that satisfies Megatron's expert-group constraint on the conversion node count.

Run the following on 4 nodes / 32 GPUs:

bash

cd /root/slime
pip install -e . --no-deps
source scripts/models/glm5.2-744B-A40B.sh
PYTHONPATH=/root/Megatron-LM/ torchrun \
   --nproc-per-node 8 \
   --master-addr ${MASTER_ADDR} --master-port 12345 \
   --nnodes=4 --node-rank ${NODE_RANK} \
   tools/convert_hf_to_torch_dist.py \
   ${MODEL_ARGS[@]} \
   --tensor-model-parallel-size 8 \
   --pipeline-model-parallel-size 2 \
   --decoder-last-pipeline-num-layers 40 \
   --expert-model-parallel-size 16 \
   --expert-tensor-parallel-size 1 \
   --hf-checkpoint $BASE_DIR/GLM-5.2/ \
   --save $BASE_DIR/GLM-5.2_torch_dist/

Here, MASTER_ADDR is the IP of node0, and NODE_RANK is the current node index.

MODEL_ARGS includes --allgather-cp, a slime-only flag, so tools/convert_hf_to_torch_dist.py registers it too (it is a no-op for conversion). On 32 GPUs, Megatron requires expert_tp(1) expert_model_parallel pp to divide the world size, so we convert with EP=16 (1162=32). The resulting checkpoint still loads at training-time EP=32 because torch_dist is reshardable.

`Run Training`

From node0:

bash

cd /root/slime
export BASE_DIR=/shared/path
export MASTER_ADDR=<node0-ip>
export HOSTFILE=$BASE_DIR/hostfile  # one worker IP per line, all 32 nodes
bash scripts/run-glm5.2-744B-A40B.sh

If HOSTFILE is not set, join the other nodes to the Ray cluster manually.

`Parameter Introduction`

#### Model Configuration

scripts/models/glm5.2-744B-A40B.sh contains the GLM-5.2 DSA + cross-layer index sharing configuration: 256 routed experts, top-8 activation, 1 shared expert, and 78 layers total (3 dense + 75 MoE).

The DSA index sharing schedule, such as index_topk_freq=4 and index_skip_topk_offset=3, is read from the Hugging Face config. The Megatron side uses the shared slime_plugins.models.glm5.glm5:get_glm5_spec provider and enables:

bash

--allgather-cp

This makes DSA + context parallel use the allgather-CP layout, and the index-share provider gathers index K/V across the CP group.

#### Training Parallelism

The default script targets 32 nodes and 256 GPUs:

bash

PERF_ARGS=(
   --tensor-model-parallel-size 4
   --pipeline-model-parallel-size 8
   --decoder-first-pipeline-num-layers 14
   --decoder-last-pipeline-num-layers 16
   --context-parallel-size 8
   --expert-model-parallel-size 32
   --expert-tensor-parallel-size 1
   ...
)

TP=4 PP=8 CP=8 = 256 GPUs form one training group (DP=1). The expert group constraint expert_tp(1) EP(32) PP(8) = 256 divides the world size exactly (expert_dp=1).

DSA cross-layer index sharing requires every pipeline stage to start on a "computing" layer. With index_topk_freq=4 / index_skip_topk_offset=3, the computing layers are 1, 2, 3, 7, 11, ..., 75. A uniform 78/8 split would start stages on skip layers and fail the index-share assertion in get_glm5_spec. We therefore use --decoder-first-pipeline-num-layers 14 and --decoder-last-pipeline-num-layers 16, leaving 6 middle stages of (78-14-16)/6 = 8 layers each. The stage starts land on global layers 1, 15, 23, 31, 39, 47, 55, 63 — all computing layers.

#### BF16 Training + FP8 Rollout

The launcher writes the default paths directly in CKPT_ARGS and ROLLOUT_ARGS, matching the style of the other example scripts:

bash

CKPT_ARGS=(
   --hf-checkpoint $BASE_DIR/GLM-5.2-FP8
   --ref-load $BASE_DIR/GLM-5.2_torch_dist
   --load $BASE_DIR/GLM-5.2_slime
   --save $BASE_DIR/GLM-5.2_slime
   --save-interval 20
)ROLLOUT_ARGS=(
   --prompt-data $BASE_DIR/dapo-math-17k/dapo-math-17k.jsonl
   ...
)

--hf-checkpoint provides FP8 weights and the tokenizer for SGLang rollout; --ref-load is the Megatron torch_dist checkpoint converted from BF16. To debug BF16 rollout, change --hf-checkpoint in the script to $BASE_DIR/GLM-5.2.

#### SGLang Configuration

The rollout side runs with prefill/decode (PD) disaggregation: 1 prefill engine (64 GPU) + 3 decode engines (192 GPU) = 256 GPUs total (which must equal the colocated rollout_num_gpus). Each engine spans 64 GPUs with DP attention and EP=64 (DeepEP's dispatch config map supports up to 160 EP ranks, so a single 256-GPU engine would be invalid). Prefill uses the auto DeepEP path; decode uses low_latency + deep_gemm. The split is configured via the --sglang-config YAML:

yaml

sglang:
  - name: default
    server_groups:
      - worker_type: prefill
        num_gpus: 64
        num_gpus_per_engine: 64
        overrides: { deepep_mode: auto, ... }
      - worker_type: decode
        num_gpus: 192
        num_gpus_per_engine: 64
        overrides: { deepep_mode: low_latency, moe_runner_backend: deep_gemm, ... }

PD transfer runs over RDMA/IB with the mooncake backend:

bash

--sglang-disaggregation-transfer-backend mooncake
--sglang-disaggregation-ib-device mlx5_100,...,mlx5_107

The rest of the rollout uses FP8 KV cache and the NSA + DeepEP backends:

bash

SGLANG_ARGS=(
   --sglang-enable-dp-attention
   --sglang-ep-size 64
   --sglang-dp-size 64
   --sglang-kv-cache-dtype fp8_e4m3
   --sglang-nsa-decode-backend flashmla_kv
   --sglang-nsa-prefill-backend flashmla_sparse
   --sglang-attention-backend nsa
   ...
)

MTP / EAGLE speculative decoding is enabled using the model's own next-token-prediction layer (the GLM-5.2 checkpoint ships an MTP layer), so no separate draft model is needed:

bash

--sglang-speculative-algorithm EAGLE
--sglang-speculative-num-steps 4
--sglang-speculative-eagle-topk 1
--sglang-speculative-num-draft-tokens 5

SGLANG_DEEPEP_NUM_MAX_DISPATCH_TOKENS_PER_RANK must cover the largest decode batch: max cuda_graph_max_bs (decode group = 12) * speculative_num_draft_tokens (5) = 60, rounded up to 64. A value below this trips the DeepEP low-latency dispatch buffer assertion during the decode group's CUDA-graph capture.

#### Networking

DeepEP/NVSHMEM communication across nodes needs the IB-aware NCCL settings in the Ray runtime env (NCCL_SOCKET_IFNAME, NCCL_IB_*, NCCL_NET_GDR_LEVEL, NCCL_P2P_LEVEL=NVL, NCCL_NVLS_ENABLE=0, MC_IB_PCI_RELAXED_ORDERING, ...). The script defaults to SOCKET_IFNAME=eth0; set SOCKET_IFNAME before launch if your environment differs, and it will be written to GLOO_SOCKET_IFNAME, TP_SOCKET_IFNAME, and NCCL_SOCKET_IFNAME. DeepEP also requires NVSHMEM_DISABLE_NCCL=1.

---

`En/Examples/Qwen3 4B`

`Qwen3-4B with 8xH100`

`Environment Setup`

After pulling the slimerl/slime:latest image, initialize the image environment as follows:

bash

cd /root/
git clone https://github.com/THUDM/slime.git
cd slime/
pip install -e . --no-deps

Download the model and data:

bash

hf checkpoint

hf download Qwen/Qwen3-4B --local-dir /root/Qwen3-4B
train data

hf download --repo-type dataset zhuzilin/dapo-math-17k \
  --local-dir /root/dapo-math-17k
eval data

hf download --repo-type dataset zhuzilin/aime-2024 \
  --local-dir /root/aime-2024

Convert the Hugging Face checkpoint into a format that Megatron can load:

bash

mcore checkpoint

cd /root/slime
source scripts/models/qwen3-4B.sh
PYTHONPATH=/root/Megatron-LM python tools/convert_hf_to_torch_dist.py \
    ${MODEL_ARGS[@]} \
    --hf-checkpoint /root/Qwen3-4B \
    --save /root/Qwen3-4B_torch_dist

`Run Training`

Execute the training script:

bash

cd /root/slime
bash scripts/run-qwen3-4B.sh

`Parameter Introduction`

Here, we will briefly introduce the various components of the run-qwen3-4B.sh script:

#### MODEL\_ARGS

bash

SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" &>/dev/null && pwd)"
source "${SCRIPT_DIR}/models/qwen3-4B.sh"

This reads the model's configuration from scripts/models/qwen3-4B.sh. These are all Megatron parameters. When training with Megatron, it cannot read the model config from the checkpoint, so we need to configure it ourselves. We provide some examples in scripts/models.

bash

source "${SCRIPT_DIR}/models/qwen3-4B.sh"MODEL_ARGS += ( --rotary-base 10000 )

#### CKPT\_ARGS

bash

CKPT_ARGS=(
   # HF checkpoint required by sglang; we also read the tokenizer from here
   --hf-checkpoint /root/Qwen3-4B
   # Checkpoint for the reference model
   --ref-load /root/Qwen3-4B_torch_dist
   # Load directory for the actor; if empty, it will be loaded from ref_load
   --load /root/Qwen3-4B_slime/
   --save /root/Qwen3-4B_slime/
   --save-interval 20
)

#### ROLLOUT\_ARGS

bash

ROLLOUT_ARGS=(
   # Prompt dataset, each line is a JSON object
   --prompt-data /root/dapo-math-17k/dapo-math-17k.jsonl
   --input-key prompt
   --label-key label
   # If the input_key in the prompt contains an OpenAI message,
   # tokenizer.apply_chat_template(...) will be executed
   --apply-chat-template
   # Whether to shuffle the data
   --rollout-shuffle
   # Reward model type.
   # slime provides many types and --custom-rm-path for custom models
   --rm-type deepscaler
   # Total number of rollouts to train
   --num-rollout 3000
   # Number of prompts in one rollout
   --rollout-batch-size 32
   # Number of responses to sample per prompt
   # A rollout will have rollout_batch_size * n_samples_per_prompt samples
   --n-samples-per-prompt 8
   # Rollout sampling parameters
   --rollout-max-response-len 8192
   --rollout-temperature 1   # Number of training steps corresponding to one rollout
   --num-steps-per-rollout 1
   # Whether to balance data during training, which might improve speed
   --balance-data
)

#### EVAL\_ARGS

During evaluation, most rollout parameters are inherited, but we provide some parameters that can override the rollout configuration to allow for different sampling strategies for training and evaluation.

bash

EVAL_ARGS=(
   --eval-interval 5
   --eval-prompt-data /root/aime-2024/aime-2024.jsonl
   --n-samples-per-eval-prompt 16
   --eval-max-response-len 16384
   --eval-top-p 1
)

#### PERF\_ARGS

This is a set of Megatron's parallelism parameters. Only --use-dynamic-batch-size and --max-tokens-per-gpu are added by slime.

max_tokens_per_gpu specifies the maximum number of tokens each GPU can process. When use_dynamic_batch_size is enabled, it attempts to pack data of varying lengths within a batch as close to max_tokens_per_gpu as possible, thus forming a dynamic micro-batch size. If a single data item exceeds max_tokens_per_gpu, it forms its own batch without being truncated. When context parallelism (CP) is enabled, it allows the CP GPUs to share a total of CP * max_tokens_per_gpu tokens.

When dynamic_batch_size is enabled, the traditional micro_batch_size is ignored.

bash

PERF_ARGS=(
   --tensor-model-parallel-size 2
   --sequence-parallel
   --pipeline-model-parallel-size 1
   --context-parallel-size 1
   --expert-model-parallel-size 1
   --expert-tensor-parallel-size 1
   --recompute-granularity full
   --recompute-method uniform
   --recompute-num-layers 1   # --micro-batch-size 1
   --use-dynamic-batch-size
   --max-tokens-per-gpu 9216
)

#### GRPO\_ARGS

Here are some GRPO-related parameters:

bash

GRPO_ARGS=(
   --advantage-estimator grpo
   --use-kl-loss
   --kl-loss-coef 0.00
   --kl-loss-type low_var_kl
   --entropy-coef 0.00
   --eps-clip 0.2
   --eps-clip-high 0.28
)

#### OPTIMIZER\_ARGS

bash

OPTIMIZER_ARGS=(
   --optimizer adam
   --lr 1e-6
   --lr-decay-style constant
   --weight-decay 0.1
   --adam-beta1 0.9
   --adam-beta2 0.98
)

#### SGLANG\_ARGS

bash

SGLANG_ARGS=(
   --rollout-num-gpus-per-engine 2
   --sglang-mem-fraction-static 0.7
)

⚠️ slime uses sgl-router to schedule multiple sglang servers. dp_size is not supported when DP attention is disabled.

`Dynamic Sampling`

slime supports more complex sampling schemes, such as the dynamic sampling in DAPO. To enable dynamic sampling, you need to configure:

bash

--over-sampling-batch-size ${OVER_SAMPLING_BS} \
   --dynamic-sampling-filter-path \
     slime.rollout.filter_hub.dynamic_sampling_filters.check_reward_nonzero_std \

Here, over_sampling_batch_size needs to be greater than rollout_batch_size. For example, you can configure it as:

bash

--rollout-batch-size 32 \
   --n-samples-per-prompt 8 \
   --over-sampling-batch-size 64 \

In this case, the sampling process will directly sample 64 prompts, with 8 samples per prompt. Since slime performs asynchronous sampling internally, we will receive the 8 responses for each prompt sequentially. Upon receiving the responses, the function specified by dynamic_sampling_filter_path is used for filtering. If the samples pass the filter, these 8 data points are kept; otherwise, they are discarded. The function in the example checks if the rewards for the samples are not all identical (i.e., not all correct or all incorrect):

python

def check_reward_nonzero_std(args, samples: list[Sample], kwargs):
    rewards = [sample.reward for sample in samples]
    return torch.tensor(rewards, dtype=torch.float).std() > 0.0

When we have received 32 \* 8 data points, we will immediately stop the current sampling round and will not wait for the remaining data to be sampled. If more than 32 prompts' worth of data is discarded (leaving fewer than 32 prompts' worth), we will then sample another 64 prompts.

`Partial Rollout`

You can customize how data is retrieved from the buffer by configuring the --buffer-filter-path. The default function is:

python

def pop_first(args, rollout_id, buffer: list[list[Sample]], num_samples: int) -> list[list[Sample]]:
    num_to_pop = min(len(buffer), num_samples)
    samples = buffer[:num_to_pop]
    del buffer[:num_to_pop]
    return samples

This means that each time, the data corresponding to the first num_samples prompts is retrieved, totaling num_samples * n_samples_per_prompt items.

⚠️ The sample.metadata of each partial rollout sample stores the rollout ID from its initial generation, which can be used for data filtering.

`Decoupled Training and Inference`

In the original script, the resource configuration is as follows:

bash

ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 8 \
   --colocate \
   ...

This enables co-located training and inference, where the training part uses 1 machine with 8 GPUs, and inference shares these 8 GPUs with training.

If you want to use the decoupled training and inference feature, you need to remove --colocate and configure --rollout-num-gpus. For example:

bash

ray job submit ... \
   -- python3 train.py \
   --actor-num-nodes 1 \
   --actor-num-gpus-per-node 2 \
   --rollout-num-gpus 6 \
   ...

In this case, 2 GPUs will be allocated for training, and 6 GPUs will be allocated for inference.

⚠️ If the concurrency on each sglang server is too high, it may exceed sglang's default CUDA graph concurrency limit (the default maximum is 160), which will affect inference speed. You can adjust this in the following two ways:

1. Use --sglang-server-concurrency to limit the maximum number of concurrent requests sent to a single sglang server. For example:

bash

--sglang-server-concurrency 160

2. Use --sglang-cuda-graph-bs (which corresponds to sglang's native --cuda-graph-bs argument) to increase the number of CUDA graphs initialized by sglang. For example:

bash

--sglang-cuda-graph-bs 1 2 4 8 $(seq 16 8 256)

`Asynchronous Training`

When you separate training and inference, you may notice that the training and inference GPUs are always waiting for each other. To prevent these resources from being idle, we can enable asynchronous training. This can be done by changing train.py to train_async.py in the startup script. By doing this, slime will generate data for the next rollout while training on the current one.

The only difference between train.py and train_async.py lies in the synchronization logic of the training loop. We achieve this by using Ray's asynchronous features (.remote, ray.get).

---

`En/Examples/Qwen3 4b Base Openhermes`

`SFT Qwen3-4B-Base`

`Environment Preparation`

First, we need to create a mirror environment and convert the Qwen3-4B-Base model by following the Example: Qwen3-4B Model.

After that, we will process the SFT data. Here, we use the classic OpenHermes-2.5 as an example. First, we process the data into a format suitable for slime to load. You can use the following script to add a column that conforms to the OpenAI message format and save it to /root/openhermes2_5.parquet.

python

from datasets import load_dataset
ds = load_dataset("teknium/OpenHermes-2.5")["train"]
def convert(sample):
    conversations = sample["conversations"]
    def convert_role(role):
        if role == "human":
            return "user"
        elif role == "gpt":
            return "assistant"
        elif role == "system":
            return "system"
        else:
            raise ValueError(f"Unknown role: {role}")
    messages = [
        {
            "role": convert_role(turn["from"]),
            "content": turn["value"],
        }
        for turn in conversations
    ]
    return {"messages": messages}ds = ds.map(convert)
ds.to_parquet("/root/openhermes2_5.parquet")

`Execute Training`

Execute the training:

bash

cd /root/slime
bash script/run-qwen3-4B-base-sft.sh

`Parameter Introduction`

You can compare run-qwen3-4B-base-sft.sh with run-qwen3-4B.sh. You will find that besides changing the model from the instruct version to the base model, the main adjustments are as follows:

1. Removed SGLANG_ARGS and GRPO_ARGS. This is because it is not necessary to start SGLang or configure GRPO-related settings during the SFT process.

2. Renamed ROLLOUT_ARGS to SFT_ARGS and configured it as follows:

bash

SFT_ARGS=(
       --rollout-function-path slime.rollout.sft_rollout.generate_rollout
       --prompt-data /root/openhermes2_5.parquet
       --input-key messages
       --rollout-shuffle
       --num-epoch 3
       --rollout-batch-size 128
       --global-batch-size 128       --loss-type sft_loss
       --calculate-per-token-loss
       --disable-compute-advantages-and-returns
       --debug-train-only
    )

SFT actually reuses the custom rollout functionality of slime. By using --rollout-function-path, the data generation part is switched from the RL rollout that uses sglang to the SFT version that reads data from a file, which is slime.rollout.sft_rollout.generate_rollout.

For SFT, it is recommended to set rollout_batch_size and global_batch_size to the same value and not to configure n_samples_per_prompt. This is equivalent to training one batch right after reading one batch.

slime also supports different loss types, and we configure the SFT loss using --loss-type sft_loss.

As for --calculate-per-token-loss, this is because slime defaults to calculating the per-sample mean for GRPO. In general SFT training, the average is taken over all unmasked tokens in a batch, so it is recommended to configure this.

Finally, --disable-compute-advantages-and-returns indicates that there is no need to pre-calculate log probabilities during the SFT process, and --debug-train-only means that sglang does not need to be initialized.

3. Used train_async.py instead of train.py. This is to leverage the asynchronous training process to implement data prefetching.

---

`En/Examples/Qwen3 30B A3B`

`Qwen3-30B-A3B with 8xH100`

`Environment Preparation`

The environment setup, model download, data, and checkpoint conversion are the same as for the Qwen3-4B model. You can refer to Example: Qwen3-4B Model, replacing mentions of Qwen3-4B with Qwen3-30B-A3B.

To convert huggingface checkpoint to torch_dist, please try:

bash

cd slime/
pip install -e . --no-deps
source scripts/models/qwen3-30B-A3B.sh
PYTHONPATH=/root/Megatron-LM/ torchrun --nproc-per-node 8 \
   tools/convert_hf_to_torch_dist.py \
   ${MODEL_ARGS[@]} \
   --hf-checkpoint /root/Qwen3-30B-A3B/ \
   --save /root/Qwen3-30B-A3B_torch_dist/

`Run Training`

Execute the training script:

bash

cd /root/slime
bash scripts/run-qwen3-30B-A3B.sh

`Parameter Introduction`

Here, we will briefly introduce the MoE-related parts in the run-qwen3-30B-A3B.sh script.

1. To support running Qwen3-30B-A3B in an 8xH800 environment, we need to enable Megatron's CPU Adam to save GPU memory. The corresponding configuration is:

bash

OPTIMIZER_ARGS=(
       ...
       --optimizer-cpu-offload
       --overlap-cpu-optimizer-d2h-h2d
       --use-precision-aware-optimizer
    )

2. Enable MoE optimization supported by Megatron. The current configuration is tp4, ep8:

bash

PERF_ARGS=(
       --tensor-model-parallel-size 4
       --sequence-parallel
       --pipeline-model-parallel-size 1
       --context-parallel-size 1
       --expert-model-parallel-size 8
       --expert-tensor-parallel-size 1
       ...
    )

3. Enable MoE optimization supported by SGLang. The current configuration is ep8:

bash

SGLANG_ARGS=(
       --rollout-num-gpus-per-engine 8
       --sglang-mem-fraction-static 0.7
       --sglang-ep-size 8
       --sglang-cuda-graph-bs 1 2 4 8 $(seq 16 8 256)
    )

Similarly, you can also add DP attention, for example, by configuring:

bash

--sglang-enable-dp-attention
       --sglang-dp-size 8

`BF16 Training with FP8 Inference`

slime also supports BF16 training with FP8 inference. For the Qwen3-30B-A3B model, you just need to download the following model:

bash

hf download Qwen/Qwen3-30B-A3B-FP8 --local-dir /root/Qwen3-30B-A3B-FP8

And replace --hf-checkpoint with:

bash

#--hf-checkpoint /root/Qwen3-30B-A3B
--hf-checkpoint /root/Qwen3-30B-A3B-FP8

This will trigger FP8 inference. Currently, we directly cast the BF16 weights to FP8. In the future, we will gradually add more sophisticated quantization schemes that have less impact on precision.

⚠️ The Megatron checkpoint for training still needs to be the one that was originally converted from the BF16 Hugging Face model.

`Multi-Node Support`

For a multi-node environment, the following modifications are necessary:

- Place the training model and data on a path accessible by all nodes. - Set theMASTER_ADDRto an address that is accessible by all nodes. - Remove configurations related to CPU Adam. This is because a distributed optimizer is used, which significantly reduces the optimizer's video memory (VRAM) usage in a multi-node setup.

In addition, you can make the following changes:

- When the total number of GPUs is not a multiple or divisor of the total number of experts, you can use --sglang-ep-num-redundant-experts to add redundant experts. For example, in a 24-GPU scenario, you can configure it as follows:

bash

SGLANG_ARGS=(
      --rollout-num-gpus-per-engine 24
      --sglang-mem-fraction-static 0.7
      --sglang-ep-size 24
      --sglang-enable-dp-attention
      --sglang-dp-size 3      --sglang-moe-dense-tp-size 1
      --sglang-enable-dp-lm-head
      --sglang-ep-num-redundant-experts 16   
   )

---

`En/Developer Guide/Ci`

`CI (Continuous Integration)`

slime CI has two layers:

1. Always-on CPU correctness tests that run on every PR, every push to main, and manual workflow_dispatch. 2. Label-gated GPU end-to-end tests that validate real Megatron + SGLang training and rollout paths on self-hosted GPU runners.

This split is intentional. Most invariants should be checked quickly without waiting for the GPU fleet, while full training/rollout behavior is still covered by GPU e2e jobs.

`How It Works`

The workflow is defined in .github/workflows/pr-test.yml, which is auto-generated from .github/workflows/pr-test.yml.j2.

`CPU Jobs`

CPU jobs run on GitHub-hosted ubuntu-latest runners:

- cpu-unittest installs CPU PyTorch and lightweight dependencies, then runs registered unit and contract tests with python tests/<test_file>.py. -agent-adapter-test does the same for agent adapter tests, with extra dependencies such as openai, openai-agents, and anthropic.

CPU jobs do not use Docker, do not acquire GPUs, and do not call tests/ci/gpu_lock_exec.py.

`GPU E2E Jobs`

GPU jobs run on self-hosted GPU runners. Each job:

1. Starts a Docker container, usually slimerl/slime:latest; image validation uses slimerl/slime-test:latest. 2. Installs slime withpip install -e . --no-deps. 3. Acquires the requested GPUs withtests/ci/gpu_lock_exec.py --count <num_gpus>. 4. Executes the registered test file withpython tests/<test_file>.py.

GPU tests usually follow the e2e pattern: prepare() downloads models/datasets, and execute() builds CLI arguments and calls U.execute_train(...).

`Changed-Test Job`

run-ci-changed dynamically detects added or modified files under tests/test_.py and tests/plugin_contracts/test_.py relative to origin/main.

For each changed test file, it extracts a top-level NUM_GPUS = <N> constant and builds a matrix. If NUM_GPUS is missing, CI defaults to 8, so CPU-only tests should declare:

python

NUM_GPUS = 0

The changed-test job itself runs through the self-hosted Docker path. When NUM_GPUS = 0, it runs the test without acquiring GPUs.

`CI Jobs and Triggers`

| Trigger | Job | Type | Description | |---|---|---|---| | Automatic |cpu-unittest| CPU | Always-on unit and contract tests for argument validation, schedules, rewards, samples, rollout validation, checkpoint utilities, and plugin contracts. | | Automatic |agent-adapter-test| CPU | Always-on agent adapter tests with optional provider SDK dependencies. | |run-ci-sglang-config | e2e-test-sglang-config| GPU | SGLang config tests for advanced rollout engine deployment and mixed/offload scenarios. | |run-ci-megatron | e2e-test-megatron| GPU | Core Megatron training tests covering dense, MoE, PPO, MTP, OPD, async rollout, PD/Mooncake, and debug replay paths. | |run-ci-precision | e2e-test-precision| GPU | Numerical precision validation and parallel consistency checks. | |run-ci-ckpt | e2e-test-ckpt| GPU | Checkpoint save/load correctness, including CPU/GPU optimizer states and async save. | |run-ci-image | e2e-test-image | GPU | Runs the run-ci-megatron matrix on slimerl/slime-test:latest. | |run-ci-changed | e2e-test-changed | Mixed | Runs only changed tests, using each file's NUM_GPUS value. |

workflow_dispatch can be used from the Actions page for manual validation. It runs the registered jobs according to the workflow conditions.

`CPU Unit Tests`

The CPU suite is the first line of defense for correctness. It is designed to catch silent RL infrastructure bugs before a change reaches expensive GPU runs.

The registered CPU suite currently covers:

- Megatron argument and HF config validation; - DP/CP scheduling utilities and CP loss invariance; - metric reporting and distributed metric aggregation; - reward-model grading utilities for math, GPQA, F1, DeepScaler, and DAPO-style math; -Samplebehavior, rollout validation, and agent trajectory merging; - HF checkpoint saver behavior; - customization hook contracts for rollout functions, generate functions, runtime hooks, and path loading.

Agent adapter tests are kept in a separate CPU job because they need extra SDK dependencies.

Useful local commands:

bash

python tests/test_agent_trajectory.py
python -m pytest tests/test_megatron_argument_validation.py tests/plugin_contracts/test_plugin_generate_contracts.py

`GPU E2E Tests`

GPU e2e tests validate the integrated training/rollout behavior that CPU tests cannot cover:

- run-ci-sglang-config: advanced SGLang deployment paths, including config-based engine layouts. -run-ci-megatron: main Megatron backend coverage for dense/MoE recipes, async rollout, OPD, PPO-style paths, PD/Mooncake, and debug rollout-then-train replay. -run-ci-precision: numerical consistency across parallel settings. -run-ci-ckpt: checkpoint save/load combinations and async save. -run-ci-image: the same matrix as run-ci-megatron, but on the release/test image.

Use targeted labels for routine PRs. Use run-ci-image sparingly because it consumes significantly more GPU time.

`Writing a New Test`

`CPU Tests`

For CPU-only tests:

1. Add the test under tests/test_.py, tests/utils/test_.py, or tests/plugin_contracts/test_*.py, following nearby patterns. 2. Add a top-levelNUM_GPUS = 0 if the file may be run by run-ci-changed. 3. Make the file executable directly:

python

if __name__ == "__main__":
    raise SystemExit(pytest.main([__file__]))

4. If the test should run permanently, register it in the cpu-unittest or agent-adapter-test job in .github/workflows/pr-test.yml.j2, then regenerate the workflow.

`GPU E2E Tests`

For GPU e2e tests:

1. Create tests/test_<your_test_name>.py following the existing prepare() / execute()pattern. 2. Declare the required GPU count withNUM_GPUS = <N>. 3. Download required models/datasets inprepare(). 4. Build arguments and callU.execute_train(...) in execute(). 5. Register the test in the appropriate GPU job in.github/workflows/pr-test.yml.j2, then regenerate the workflow.

Example skeleton:

python

import os
import slime.utils.external_utils.command_utils as U
MODEL_NAME = "Qwen2.5-0.5B-Instruct"
MODEL_TYPE = "qwen2.5-0.5B"
NUM_GPUS = 4
def prepare():
    U.exec_command("mkdir -p /root/models /root/datasets")
    U.exec_command(f"hf download Qwen/{MODEL_NAME} --local-dir /root/models/{MODEL_NAME}")
def execute():
    # Build argument strings and call U.execute_train(...)
    ...if __name__ == "__main__":
    prepare()
    for proxy_var in ("http_proxy", "https_proxy", "HTTP_PROXY", "HTTPS_PROXY"):
        os.environ.pop(proxy_var, None)
    execute()

`Workflow Generation`

The workflow file pr-test.yml is auto-generated from the Jinja2 template pr-test.yml.j2. Do not edit pr-test.yml directly.

To change the permanent CI matrix:

1. Edit .github/workflows/pr-test.yml.j2. 2. Run:

bash

python .github/workflows/generate_github_workflows.py

3. Commit both .github/workflows/pr-test.yml.j2 and the generated .github/workflows/pr-test.yml.

`Choosing Checks for a PR`

- Pure argument parsing, reward, schedule, sample, trajectory, or hook-contract changes: rely on CPU tests first. - SGLang topology or rollout engine deployment changes: userun-ci-sglang-config. - Megatron training, loss, checkpoint conversion, or model recipe changes: userun-ci-megatron; add run-ci-precision or run-ci-ckptwhen relevant. - Docker image or dependency changes: userun-ci-image. - New or modified tests: userun-ci-changed for quick targeted validation.

---

`En/Developer Guide/Debug`

`Debugging`

`Aligning Precision`

During the development of slime, it is often necessary to check if the model's precision is correct. This can be verified in the following ways:

1. First Training Step 1. Check if the generatedrolloutis coherent. If not, there are two possible reasons: * Parameters were not loaded correctly. You need to check the logs for a confirmation that Megatron successfully loaded the checkpoint (ckpt). * There was an error in updating the parameters. You can check if all parameters were converted and mapped correctly, or if the parameter names were converted according to the parallelization strategy (e.g., whenpp_size > 1, check if the layer IDs for the parameters provided by the second stage are correct). A thorough method is to save all parameters in the load_weightsimplementation of the corresponding model in SGLang and verify that they are consistent with the loaded checkpoint. * If all parameters are updated correctly and the problem persists, it's possible that some special buffers in SGLang were released during the release process. * If you are testing with a pretrained model, you can switch to an instruct version of a model with the same architecture to see if this garbled output is specific to the pretrained model.

2. Check the printed rollout stats to see if log_probs and ref_log_probsare exactly equal (meaning KL divergence is 0 in the first step) and their values are small. * If they are not exactly equal, it is usually caused by certain non-deterministic kernels in the Transformer Engine, for example: * In some versions of Transformer Engine (TE), Megatron requires--attention-backend flashto enforce the use of Flash Attention, thereby avoiding numerical instability from the fused attention under Context Parallelism (CP). * If the values are large (e.g., > 1), there are generally two possibilities: * If the value is extremely large, there is likely a problem with the training configuration. * If the value is only slightly larger than the SFT loss, for example, if the log probability of an instruct model reaches 0.8, it might be because the data does not conform to the trained chat template or does not match the cold-start distribution.

3. When running one inference step per training step (num_steps_per_rollout == 1), check if the KL divergence is 0 and if the grad_normis small. * This is basically due to some Megatron / TE related bugs, for example: * Mixture of Experts (MoE) requires enabling--moe-permute-fusion.

2. Second Training Step 1. For integrated training and inference, check if the second step can be loaded correctly and whether it results in an Out of Memory (OOM) error.

`Separate Debugging for Training and Inference`

slime supports debugging the training and inference parts separately, which allows for the following:

* When tuning/debugging the inference part, you can start the task with only a few GPUs. * When tuning/debugging the training part, you can ensure the model input is fixed, removing the randomness of rollouts.

Specifically, slime currently provides the following parameters for separate debugging:

1. --debug-rollout-only

When enabled, slime will not load Megatron and will only initialize SGLang. You can use this method to debug the inference part.

2. --debug-train-only

When enabled, slime will not load SGLang and will only initialize Megatron. You can use this method to debug the training part.

3. --save-debug-rollout-data /your/saved/debug/data_{rollout_id}.pt

When enabled, the results of each rollout will be saved. This can be used in conjunction with --debug-rollout-only. Note that the data is saved using the format: args.save_debug_rollout_data.format(rollout_id=rollout_id).

4. --load-debug-rollout-data /your/saved/debug/data_{rollout_id}.pt

When enabled, data will be loaded from args.load_debug_rollout_data.format(rollout_id=rollout_id), and SGLang will not be initialized (automatically setting debug_train_only=True). This method allows you to fix the input for the training part to tune it, for example, by switching between different parallelization strategies.

5. --save-debug-train-data /your/saved/debug/train_{rollout_id}.pt

Saves one train-side file per rollout. Only the last Pipeline Parallel stage and Tensor Parallel rank 0 participate. They restore response-token fields such as log_probs, ref_log_probs, values, advantages, returns, kl, and entropy across Context Parallel ranks. Context Parallel rank 0 moves each restored tensor to CPU immediately, so complete tensors do not accumulate on the GPU, and then gathers the distinct Data Parallel shards to one writer.

The version-2 payload mirrors the rollout debug dump: a top-level samples list holds one dict per training sample (sample_index, data_parallel_rank, and its per-sample fields such as tokens, log_probs, advantages), sorted by sample_index so it lines up one-to-one with the rollout dump's samples (join on sample_index ↔ the rollout side's index). A parallel dp_shards key preserves the DP/micro-batch layout — each entry records rank, data_parallel_rank, that shard's sample_indices, and the DP-local schedule (micro_batch_indices, num_microbatches, global_batch_sizes) — without duplicating any per-sample tensor. Whole-batch fields such as raw_reward are stored once at the top level. If any sample lacks a sample_index (custom rollouts that build fresh Sample objects leave it None), the samples stay in DP-gather order and a warning is logged. With or without CP, response-token fields use the same full-response format. In configs that skip the separate actor log-prob recompute (can_reuse_log_probs_in_loss or --use-rollout-logprobs), the actor log_probs are snapshotted from the training forward itself (keyed by rollout position, at no extra forward), so the dump still carries them.

`INT4 / Compressed-Tensors Quantization Checkpoint Issues`

When using INT4-quantized models (e.g., compressed-tensors with W4A16), the checkpoint's config.json contains a quantization_config.ignore list that specifies which parameters should not be quantized. During online weight updates (Megatron → SGLang), slime also reads this ignore list to decide which parameters to INT4-quantize. An incorrect ignore list can cause silent errors:

1. MoE router weights (mlp.gate.weight) become all zeros

The MoE router weight (mlp.gate.weight, shape [num_experts, hidden_size]) is a plain 2D weight tensor, but it is not a Linear layer weight. If it is not in the ignore list, the online quantizer will INT4-quantize it into weight_packed, weight_scale, weight_zero_point, etc. However, SGLang does not expect quantized names for the router, so these parameters are silently skipped during load_weights, resulting in all-zero gate weights.

Fix: Ensure config.json contains "re:.mlp\\.gate\\.." in the ignore list.

2. Other non-Linear 2D weights

Similar issues can occur with any 2D .weight tensor that is not a true Linear layer, such as model.embed_tokens.weight. Always verify the ignore list covers all non-Linear weights.

Recommended ignore patterns (for GLM-style MoE models):

json

"ignore": [
     "lm_head",
     "model.embed_tokens.weight",
     "re:.self_attn.",
     "re:.mlp\\.shared_experts.",
     "re:.mlp\\.gate_up_proj.",
     "re:.mlp\\.gate_proj.",
     "re:.mlp\\.up_proj.",
     "re:.mlp\\.down_proj.",
     "re:.eh_proj.",
     "re:.mlp\\.gate\\.."
   ]

3. Missing safetensors shards

Conversion tools may occasionally produce an incomplete checkpoint (e.g., a missing model-00010-of-00093.safetensors). After conversion, always verify: - The number of.safetensorsfiles matches the expected count. - Themodel.safetensors.index.jsoncontains entries for every layer. - Spot-check that critical layers (e.g., the first MoE layer) have the expected number of keys.

4. How to diagnose

- Use --check-weight-update-equalto verify that weights after a Megatron → SGLang sync match the expected values. If a parameter shows all zeros on the SGLang side, it was likely incorrectly quantized or missing from the checkpoint. - Use--debug-rollout-only with a small number of GPUs to quickly test whether SGLang can generate coherent text from the quantized checkpoint alone.

`Debug sglang illegal memory access (IMA)`

When running large scale RL, we will occationally meet the IMA in SGLang, there are some debug suggestions based on our experience:

1. Enable CUDA_LAUNCH_BLOCKING=1

2. Enable or disable speculative decoding and cuda graph to see if anything changed

IMA always appears in the padding in cuda graph replay, or the difference between draft model and main model. We can minimize the scope by tuning them.

3. Turn off deepep

If you are using deepep during training or inference, you can try turn it off.

4. Try CUDA Core Dump to find the error kernel

We recommend reading the blog from the vLLM team: CUDA Core Dump: An Effective Tool to Debug Memory Access Issues and Beyond

`Step-by-Step Debugging with Ray Distributed Debugger`

Ray provides a distributed debugger based on debugpy that lets you set breakpoints in the driver process and step through code interactively.

1. Install debugpy:

bash

pip install debugpy==1.8.0

2. Enable RAY_DEBUG_POSTMORTEM in your launch script:

bash

export RAY_DEBUG_POSTMORTEM=1
   RUNTIME_ENV_JSON="{
     \"env_vars\": {
       ...
       \"RAY_DEBUG_POSTMORTEM\": \"${RAY_DEBUG_POSTMORTEM:-0}\"
     }
   }"   ray job submit --address="http://127.0.0.1:8265" \
      --runtime-env-json="${RUNTIME_ENV_JSON}" \
      -- python3 train.py [args...]

3. Add ray.init() before breakpoint() in train.py:

python

if __name__ == "__main__":
       ray.init()
       breakpoint()
       args = parse_args()
       train(args)

ray.init() is required because the distributed debugger depends on core_worker, which is only available after Ray initialization. Without it, breakpoint() raises AttributeError: 'Worker' object has no attribute 'core_worker'.

4. Connect via VS Code:

Install the Ray Distributed Debugger extension in VS Code. Run your launch script to submit the job. Once the job hits breakpoint(), open the Ray Dashboard panel in VS Code and click the active breakpoint to attach the debugger. You can then step through code, inspect variables, and set additional breakpoints directly in the editor.

Note: Remove ray.init() and breakpoint() after debugging. An explicit ray.init() without arguments may cause issues in multi-node training where Ray injects specific namespace and runtime environment configurations via ray job submit.

---

`En/Developer Guide/Profiling`

`Profiling`

In slime, we can perform detailed performance analysis of the rollout process using the profiling interface provided by SGLang.

`1. Sleeping the Rollout Process`

For more flexible stress testing and profiling, it is often useful to make the slime rollout process enter a waiting state after initialization, instead of starting generation immediately.

You can achieve this by replacing the rollout_function_path in your startup arguments without modifying the source code:

bash

python train.py \
    --rollout-function-path slime.rollout.sleep_rollout.sleep \
    ... (other arguments)

This function will make the rollout process enter an infinite wait loop, allowing you to manually send requests or run stress testing tools.

`2. Obtaining SGLang Engine List`

SGLang engines (workers) are registered with the router. You can retrieve the list of all active engines by accessing the /workers endpoint of the router.

The router address is typically printed in the startup logs:

text

Router launched at 127.0.0.1:3000

You can use curlto view the workers:

bash

curl http://127.0.0.1:3000/workers

`3. Using Automated Profiling Tool`

To simplify profiling across multiple engines simultaneously, we provide an automated script: tools/profile_rollout.py.

`Starting Profiling`

By default, this tool starts profiling on all workers and will automatically stop after 3 steps:

bash

python tools/profile_rollout.py --router-url http://127.0.0.1:3000 --action start --num-steps 3

Key Parameters: *--router-url: The URL of the Router. *--num-steps: Number of steps to record, defaults to 3. *--output-dir: Directory where trace files will be saved. *--activities: Activities to monitor, e.g., GPU CPU. *--profile-by-stage: Whether to profile by stage (prefill/decode).

`Stopping Profiling Manually`

If you did not set num_steps or wish to stop early:

bash

python tools/profile_rollout.py --router-url http://127.0.0.1:3000 --action stop

`4. Running Stress Tests`

While the Rollout process is in a waiting state via sleep_rollout, you can: 1. Start profiling usingtools/profile_rollout.py. 2. Use stress testing tools (such as SGLang's built-in benchmark tools) to send requests to the router or directly to the engines. 3. Wait for profiling to complete (ifnum_stepswas set) or stop it manually. 4. Collect the.json trace files from the output_dir and view them using chrome://tracing in Chrome or Perfetto.

---

`En/Developer Guide/Trace`

`Trace Viewer`

slime can attach lightweight execution traces to each rollout sample. These traces capture span-style events such as generation and reward-model calls, and they can be inspected later from a saved rollout debug dump.

`Save rollout trace data`

To inspect traces later, save rollout debug data during a run:

bash

python train.py \
    ... \
    --save-debug-rollout-data /path/to/debug/rollout_{rollout_id}.pt

Each saved .pt file contains the rollout samples together with their trace payloads. You can also replay the same dump later with --load-debug-rollout-data.

`Open the timeline viewer`

Use the trace viewer script on a saved rollout dump:

bash

python tools/trace_timeline_viewer.py /path/to/debug/rollout_0.pt

The script generates:

- rollout_0.trace_timeline_cache.json-rollout_0.trace_timeline_viewer.html

By default it also starts a local static server so you can open the generated HTML immediately. If you only want the files, use --no-serve.

`How to read the viewer`

- Each row corresponds to one sample. - Bars represent spans, while point markers represent instant events. - Span attributes recorded at the start or end oftrace_span(...)are shown in the details panel. - When SGLang returns PD disaggregation timings, the viewer adds synthetic[P] and [D]lanes to break out prefill/decode work. - When PD is not enabled, those virtual lanes are omitted automatically and the base trace still renders normally.

`Instrument custom code`

For custom rollout or reward code — including custom agent steps, tool calls, sandbox execution, and verifier calls in agentic workflows — reuse helpers from slime.utils.trace_utils:

- trace_span(target, name, attrs=...): record a duration span. -trace_event(target, name, attrs=...): record an instant event. -trace_function(name, ...): wrap a whole sync/async function in a span. -bind_trace(sample): ensure a sample already has a trace carrier before passing it across helpers or tasks.

trace_span `vs` trace_function

Use trace_span(...) when you only want to trace part of a function body, or when you need to update end-of-span attrs from inside the block.

Use trace_function(...) when the whole function should be represented as one span. Internally it resolves the trace target and then opens a trace_span(...) around the function call, so it works for both sync and async functions.

The decorator is what slime uses for the main rollout pipeline. For example, generate_and_rm(...) is traced per sample and generate_and_rm_group(...) is traced per sample group:

python

from slime.utils.trace_utils import trace_function

@trace_function("generate_and_rm", target="sample")
async def generate_and_rm(args, sample, sampling_params, evaluation=False):
    ...
@trace_function(
    "generate_and_rm_group",
    target="group",
    attrs_getter=lambda args, group, sampling_params, evaluation=False: {"group_size": len(group)},
)
async def generate_and_rm_group(args, group, sampling_params, evaluation=False):
    ...

`Choosing a target`

trace_function(...) needs a trace target, usually a Sample, TraceHandle, or a list of them.

- Prefer target="sample" or target="group"when the target is already one of the function arguments. - Usetarget_getter=...when the trace target has to be derived from arguments. - Avoid relying on automatic inference unless the function signature is simple. The implementation can infer a target from arguments or the current trace context, but explicit targets are more stable and avoid ambiguous traces.

`Recording attrs on decorated functions`

If you want to attach attributes at span start, use attrs_getter=...:

python

@trace_function(
    "custom_rollout_batch",
    target="samples",
    attrs_getter=lambda samples, _: {"batch_size": len(samples)},
)
async def custom_rollout_batch(samples, kwargs):
    ...

If you need to add attrs after part of the function has executed, use an inner trace_span(...) instead of only relying on the decorator. A common pattern is:

- trace_function(...)for the outer function-level lifecycle span - nestedtrace_span(...) for important sub-steps such as generation, RM, filtering, or post-processing

If you want to record SGLang generation metadata in a consistent way, reuse build_sglang_meta_trace_attrs:

python

from slime.utils.trace_utils import build_sglang_meta_trace_attrs, trace_spanwith trace_span(sample, "sglang_generate") as span:
    output = await post(url, payload)
    span.update(build_sglang_meta_trace_attrs(output["meta_info"]))

`Tips`

- Save a small number of rollouts first; the viewer is easiest to read when each dump contains a manageable number of samples. - The viewer is built from the saved.ptdump, so traces can be inspected offline on another machine. - For GPU/kernel-level SGLang profiling traces, see Profiling.

---

`En/Blogs/Introducing Slime`

`slime: An SGLang-Native Post-Training Framework for RL Scaling`

This article was first released in lmsys.org.

`Vision That Drives slime`

We believe in RL. We believe RL is the final piece toward AGI.

If you feel the same way, you'll share our vision:

- Every field should be end-to-end RLed and every task should become an agent environment. - Every RL run should last longer, and every model should scale larger. - RL systems should integrate seamlessly with existing infrastructure, letting us focus on new ideas instead of boilerplate engineering.

That's why we present slime, a post-training framework designed to be:

- Versatile – with a fully customizable rollout interface and flexible training setups (colocated or decoupled, synchronous or asynchronous, RL or SFT cold start). - Performant - integrating SGLang for inference and Megatron-LM for training, natively. - Maintainable - with a lightweight codebase and smooth transition from Megatron pretraining to SGLang deployment.

In short, a post-training framework for RL scaling.

Here’s how we made it happen.

`Customizability Brings Freedom`

We should stop trying to find simple ways to think about the contents of minds, such as simple ways to think about space, objects, multiple agents, or symmetries.


> 

— The Bitter Lesson

A prevailing misconception within the RL community is the need for separate frameworks for different tasks: one for plain math, one for multi-turn tool calling, one for asynchronous training, one for agentic tasks, and so on. Forking and maintaining multiple frameworks is dreadful, leading to time-wasting bugfix cherry-picking, or worse, training crashes by missing patches.
It wasn’t always like this: no one forks PyTorch just for a new dataloader. We believe the current chaos stems from the trap of dictating how people should build their applications. If we insist on defining a universal template for every rollout scenario, we’ll inevitably create an RL framework that meets only a fraction of real-world needs.
slime views the data sampling in RL differently. We manage all SGLang servers within slime with sgl-router and provide an interface for the data generation component, allowing users to inject custom logic and freely interact with SGLang servers. Unleash their creativity.
With the sgl-router, users only need to send HTTP requests to a single endpoint. By exposing this endpoint, complex agent environments can directly interact with slime through an OpenAI-compatible API — no need to modify the environment, and training-deployment consistency is preserved.

Regarding training schemes, slime uses Ray for resource management, enabling colocated (same GPUs) or decoupled (separate GPUs) setups with a single flag (--colocate).

And with Ray's asynchronous execution via .remote(), slime naturally supports asynchronous training. Changing synchronization behavior is as simple as moving the ray.get operation. And to make experimenting with different strategies easy, we didn't wrap the code with trainer classes, but simply exposed the training loop in entrypoint train.py.

`Built for Performance`

A decent RL framework must be fast and consistently fast.

Fast means leveraging the fastest inference and training frameworks.

Unlike pre-training, RL workloads involve tons of online sampling during training, which makes the inference performance crucial. Therefore, slime exclusively integrates SGLang, and deliberately delivers an SGLang-native experience.

So what does ‘SGLang-native’ mean? It means you can take full advantage of all SGLang optimizations — using SGLang inside slime feels just like using it standalone. To make that possible:

- slime internally launches SGLang servers in a server-based mode. - slime implements seamless pass-through for all SGLang parameters (with a--sglang prefix), ensuring that all optimization options can be enabled. For instance, you can pass --sglang-enable-ep-moe, --sglang-enable-dp-attention and --sglang-enable-deepep-moefor the powerful multi-node MoE inference capabilities. - slime provides an SGLang-only debug mode (--debug-rollout-only) for easy performance tuning.

Together, we can reproduce the standalone performance of SGLang within slime. Even the base image of slime is built on lmsysorg/sglang:dev.

For training, slime integrates the battle-tested Megatron-LM, aiming for a similarly native pre-training experience:

- slime also implements seamless pass-through for all Megatron parameters. - slime supports all Megatron parallelisms (TP, PP, EP, CP) and monitors training MFU. - slime offers a Megatron-only debug mode (--debug-train-only) and supports storing sampling data for reproducibility.

Megatron can be notoriously complex, so we also provide checkpoint conversion tools to simplify its use.

Consistently fast means keeping pace with the evolving inference and training frameworks.

If you ever followed the SGLang PR list, you will be astonished by its rapid evolution. Megatron, on the other hand, is often heavily customized, with every organization maintaining its own fork. slime is designed to keep pace with upstream changes in SGLang and adapt to optimizations in in-house Megatron variants. This is another reason why we pursue native support for SGLang and Megatron. The parameter pass-through makes upgrading effortless.

Beyond optimizing inference and training frameworks, we also tackled RL-specific workloads. When SGLang needs changes to support these workflows, we work closely with the SGLang team to upstream patches—so slime can stay native, even as RL logic evolves. Examples include:

Optimizing weight updates: Unlike inference tasks, RL training involves frequent updates to model weights. To address this, we’ve introduced several optimizations in SGLang: - Parameter updates for MoE models under various parallelism strategies (#6265, #6308, #6311). - Bucketed parameter update support to reduce overhead (#7292).

/abort_request for dynamic sampling: In RL algorithms that require oversampling, such as DAPO, some requests may continue running even after sufficient data has been collected. In collaboration with the AReal team, we designed an new endpoint: /abort_request. This endpoint enables: - Immediate termination of on-going requests. - Reclaiming partially generated content, which enables partial rollouts. Implemented in #6698, #6855, #6184, #5966.

`Lightweight and Extensible`

Focusing on customization and performance, slime:

1. Provides a customizable rollout interface. 2. Uses Ray for GPU management and asynchronous execution. 3. Integrates SGLang for inference and Megatron for training. 4. Provides weight updates between training and inference.

Pretty straightforward, right? slime transfers complexity from the framework to user-defined pipelines and core libraries (SGLang and Megatron), resulting in a lightweight, easily maintainable codebase.

But it doesn’t stop at RL.

Thanks to its modular design and powerful backends, slime can naturally extend to other post-training workflows with minimal extra code:

- SFT: Load Megatron and use token prediction loss. - Rejection Sampling: Use SGLang for filter, followed by Megatron SFT.

(Note that SFT feature is now in experimental state.)

Beyond that, slime's native integration seamlessly bridges pre-training to online services. We can use Megatron for pre-training, switch to slime (which integrates both Megatron and SGLang) for post-training, and finally use SGLang directly for evaluation and deployment. This eliminates the cumbersome and error-prone steps of converting checkpoint formats and aligning precision between frameworks.

The unified pipeline saves us from tedious glue code, freeing us to focus on what really matters: better RL. Hurray!

`Roadmap`

The journey of RL scaling has just begun, and slime is continuously evolving. In the next phase, we will focus on:

1. Collaborating with the SGLang team to explore optimal RL training strategies for large-scale MoE models. 2. Supporting broader post-training workflows, strengthening the pre-training-to-production bridge. 3. Adding native PyTorch training backend support to lower the entry barrier.

We hope slime accelerates your RL scaling journey and turns your innovative ideas into reality. Contributions and conversations are always welcome!

Special thanks to the AMD GenAI - Foundation Model Team for Day-1 AMD hardware support.

---

`En/Blogs/Release V0.1.0`

`v0.1.0: Redefining High-Performance RL Training Frameworks`

The origin version of this article is in Chinese and was first released in zhihu.

With the help of the community, we've finally released the first version of slime, v0.1.0, just two months after it was open-sourced.

In a nutshell, this version can be summarized as follows:

slime v0.1.0 provides all the essential performance optimizations needed for large-scale MoE RL training.

Specifically, this version brings the following improvements:

- Performance: - Provides efficient inference for MoE models, especially with fp8 rollout + deepep + mtp. - Designed a generic training framework memory offload solution to save more KV Cache space, thus increasing inference concurrency. - Faster parameter updates. - Achieves more training with fewer GPUs through CPU Adam. - Supports all of Megatron's parallel strategies as well as deepep. - Features: - Added support for GSPO for MoE model training. - Added support for TIS for fp8 rollout. - Correctness: - Implemented Dense and MoE model CI to strictly check metrics like kl.

We hope to use slime v0.1.0 to demonstrate our understanding of high-performance RL training frameworks and have it become a baseline for future performance comparisons.

Next, I'll elaborate on the design philosophy behind these features.

-----

`Performance Optimization: Pushing the Limits of RL Training Speed`

In traditional deep learning training, there's a universal solution for speedup: add more GPUs. By reducing the amount of data processed per GPU, you can significantly lower end-to-end training latency.

However, this method doesn't work for RL training because inference latency cannot be reduced by adding more GPUs. Even with more GPUs, we still have to wait for the longest sample to finish decoding. While increasing throughput can improve the amount of training data per rollout, the off-policy issues caused by an excessively large inference batch size still have some limitations.

I believe this is the biggest challenge for infrastructure under the current RL paradigm, which is:

We want to scale inference compute, but we cannot scale inference latency.

The decoding speed of a single data point determines the upper limit of RL training speed. For larger MoE models, there are currently three common optimization methods to push this limit, and we've tried all of them:

1. Reduce memory access through quantization: Considering that long calibration is not feasible in RL training, slime opts for fp8 quantization. 2. Use deepep low-latency mode to reduce all2all latency across machines: To work with deepep, slime recommends using blockwise quantization with fp8 to enable related SGLang configurations. 3. Enable Speculative Sampling: slime allows loading any draft model for the inference part (currently, it doesn't support updating the draft model during training).

By using the three optimizations mentioned above, we can increase a model like GLM4.5 355B-A32B from less than 10 tokens/s for a single data point to 60-70 tokens/s, which significantly raises the upper limit of RL training speed.

In addition to monitoring inference throughput, slime also monitors perf/longest_sample_tokens_per_sec to better understand the potential for performance optimization in the inference part.

-----

`Doing More Experiments with Fewer GPUs: Fully Offloading Megatron`

After optimizing the upper limit, we noticed another characteristic of RL training: as long as the KV Cache doesn't overflow, increasing the inference batch size doesn't significantly affect training latency.

KV Cache overflow occurs during inference when the response lengths of the data are all very long, leading to insufficient KV Cache space. This requires kicking out some half-generated data from the queue and then re-running prefill and subsequent inference steps after other data has been processed and freed up space. If a data point with a response length of 64k has to wait for 32k tokens to be decoded by other data during its inference, its total time is equivalent to decoding 96k tokens. This greatly impacts the RL training speed.

Therefore, a more suitable training configuration is to calculate the minimum number of GPUs needed to prevent KV Cache overflow based on the inference batch size, the average response length, and the available KV Cache space on a single server. A group of these GPUs is then used for training. For example, if we have 512 GPUs and the calculation shows that 256 GPUs provide enough KV Cache, we should run two experiments in parallel instead of launching one experiment with all 512 GPUs.

Based on this consideration, we noticed two points for optimization:

1. The optimal number of GPUs may not be sufficient to load the training part. Inference only needs to load the fp8 parameters, while training generally requires more than 18 times the parameter size of GPU memory (bf16 param, fp32 grad, fp32 master param, fp32 m and v). To solve this, slime uses Megatron's built-in CPU Adam to save GPU memory for the training part. This strategy allowed us to provide solutions for training GLM 4.5 355B-A32B with 8 nodes and DeepSeek R1 with 16 nodes. 2. Increase the KV Cache space available per SGLang Server, which means increasingmem_fraction. For the more common integrated training and inference tasks, the main limitation for a larger mem_fraction is the residual GPU memory after offloading the training part to the CPU. Therefore, we need to find a generic way to offload the GPU memory used by the Megatron part.

`How to Offload GPU Tensors Generically`

One crude approach is to find all the GPU Tensors allocated by Megatron and call .to("cpu") on all of them. This method has three difficulties:

- It's hard to capture all GPU Tensors allocated by Megatron. - Because Megatron's distributed optimizer reorganizes all parameters into some contiguous GPU buffers and then divides them with various slices, it's difficult to properly handle all references to correctly free the GPU Tensors. - It requires checking the source code again with every new Megatron version, which is hard to maintain.

Is there a more generic solution?

We noticed that SGLang's torch_memory_saver and VLLM's cumem_allocator provide a more general offload solution. Their principle is that CUDA 10.2 provides a series of Virtual Memory Management APIs, similar to an operating system's virtual and physical addresses (VA and PA). When allocating GPU memory, they return a handle to a memory mapping instead of the actual physical address. Therefore, when offloading, we only need to "secretly" release the memory corresponding to this mapping and reallocate it when this memory is needed. The upper-level application doesn't need to be aware of this.

A natural idea is to use this method to take over the entire training process in RL. However, this prevents the reuse of PyTorch's CUDACachingAllocator, and without the cache, memory fragmentation becomes more pronounced, easily leading to OOM during training.

To continue reusing the native, cached allocator, we cannot use CUDAPluggableAllocator. Noticing again that slime's architecture has training and inference in different processes, we only need to directly replace cudaMalloc and cudaFree used by CUDACachingAllocator in the training process with VMM APIs via LD_PRELOAD. This allows us to completely and generically offload all GPU Tensors allocated by PyTorch.

At the same time, we must also note one detail: VMM APIs and cudaIPC APIs (such as cudaIpcGetMemHandle) are incompatible. Therefore, for integrated training and inference tasks and DeepEP, we need to disable the LD_PRELOAD replacement and switch back to cudaMalloc.

With help from the SGLang community, we updated torch_memory_saver for slime's needs, implementing this offload solution.

`How to Offload NCCL`

After thoroughly offloading the GPU Tensors in Megatron, we found that a large amount of GPU memory still remained, which was caused by NCCL. In PyTorch, each NCCL group involved in communication allocates a substantial buffer. This issue is particularly noticeable for larger MoE models due to the various parallel strategies, potentially taking up more than 10GB.

The LD_PRELOAD solution mentioned above doesn't handle the NCCL issue well, and we don't want to modify the NCCL source code to avoid having to maintain a separate NCCL fork in addition to slime. So, slime's approach is to use destroy_process_group to destroy the NCCL group when offloading Megatron and then recreate it before loading Megatron. To do this, we mimicked the VMM API and monkey patched dist.new_group to add a layer of ReloadableProcessGroup.

In this way, we achieved a generic NCCL offload. However, because we need to rebuild the NCCL group, this operation has a slight impact on the speed of the first communication in each training iteration. But we believe this approach offers a significant advantage in terms of maintainability and the GPU memory it saves.

Combining these two optimizations, we reduced Megatron's residual GPU memory from around 15-18GB to 3-5GB, which allows us to increase the mem_fraction for MoE models to 0.7-0.8. This significantly boosts the available KV Cache, increases the concurrency each server can support, and allows us to launch more training tasks with fewer GPUs.

-----

`Parameter Update Optimization`

Parameter update is another special step in RL training. For this, slime v0.1.0 provides the best optimization solution for scenarios where training and inference are in different processes. This work was heavily optimized by Biao He. I recommend reading his blog post:

- Efficient Reinforcement Learning Training - Optimizing Weight Synchronization in slime

Currently, slime can complete weight synchronization for a GLM4.5 355B-A32B model with bf16 weights in 48s and complete fp8 blockwise quantization + parameter update in 100s (the fp8 branch is still being optimized).

-----

`Training Optimization`

For the pure training part of slime, we believe Megatron already provides ample optimizations, so our main focus was to ensure compatibility with all of Megatron's parallel strategies.

During this adaptation, we found an interesting bugfix: we discovered that when SGLang enabled mtp, the Megatron part couldn't start DeepEP. It turned out that when mtp is enabled, SGLang disables the overlap schedule, which causes a certain metadata communication to use nccl instead of gloo after being offloaded to the CPU, and this conflicts with DeepEP.

-----

`Performance Optimization Check List`

Since the release of slime, I've often been asked about its performance comparison with other frameworks.

My understanding of benchmarks is that they should not be used as a weapon for frameworks to attack each other, but rather as a tool for identifying gaps. To that end, we will gradually release performance benchmarks that slime focuses on to improve ourselves.

I also believe that before running benchmarks, you can analyze a framework's focus on performance from a qualitative perspective. Here's a basic feature check list for optimizations:

- Does it support MoE training? (Currently, large-scale experiments are focused on MoE) - Can the internal sglangmem_fraction or vllm gpu_utilizationbe adjusted to over 0.7? (Ensures KV Cache space) - Does it support fp8 or lower precision inference? (Reduces inference memory access, boosts speed) - Does it support enabling deepep for both training and inference? (Optimizes MoE all2all communication) - Does it support speculative sampling? (Improves inference latency and throughput) - Does it have an efficient training backend, such as Megatron or torchtitan, and support all necessary parallelization strategies? (Reuses mature training optimizations)

slime v0.1.0 has made preliminary attempts at all the above optimizations, and there's still a lot of room for improvement. We hope this version can serve as a baseline for future slime versions or for performance comparisons between different frameworks. We also welcome all friends who share our pursuit of performance to try out slime and join the slime community\!

-----

`New Algorithm Support`

To better train MoE models and perform fp8 rollouts, we implemented GSPO and TIS. Additionally, community experts have helped implement algorithms like reinforce++ and reinforce++ baseline.

-----

`Correctness Verification`

slime v0.1.0 adds end-to-end CI: we run single-machine GLM4 9B and Qwen3 30B-A3B training for each PR, ensuring correctness through strict checks. For example, we explicitly require:

- The recomputed log prob of the first rollout must be exactly equal to the log prob of the reference model. - The ppo_kl of the first training step within each rollout must be exactly 0.

Such precise verification is rarely achieved in training frameworks, and it's something we are very proud of.

-----

This is a brief introduction to slime v0.1.0. I hope it sparks your curiosity about slime and that it can be helpful in your work.

Everyone is welcome to join the slime community. Let's work together to build an open RL Infra and contribute to RL scaling\!

---

`En/Advanced/Arch Support Beyond Megatron`

`Supporting Model Architectures Beyond Megatron-LM`

While the Megatron-LM framework is highly efficient for parallel training, it can lack the flexibility to support rapidly evolving model architectures like Qwen3Next. Natively supporting the unique structures of these models, such as Gated-Delta-Net, often requires invasive and time-consuming modifications to Megatron's core codebase.

To accelerate the adoption of these cutting-edge models, slime introduces a more agile approach: instead of deeply re-engineering Megatron, we directly import and wrap the model's official HuggingFace implementation, embedding it as a "black-box" module into Megatron's parallel training pipeline.

This document uses Qwen3Next 80B-A3B as an example to illustrate this concept.

`Principle and Core Components`

Megatron's model instantiation is a two-step process: first, it generates a "layer specification" (ModuleSpec) based on the configuration, and then it instantiates the actual PyTorch modules according to that spec.

slime leverages this mechanism by hijacking the spec generation stage to replace Megatron's native modules with an external implementation (in this case, from HuggingFace). This process involves the coordination of three core components:

1. Replacing the Megatron Module Spec This is the entry point for our solution. We use a custom function (e.g.,get_qwen3_next_spec) to modify the standard ModuleSpec, swapping out Megatron's native Attention layer with our custom wrapper. * Implementation: It retrieves the standard Decoder Block Spec, points itsself_attention field to our custom module, and enables model-specific configurations like qk_layernormas needed. * Corresponding File:slime_plugins/models/qwen3_next.py

2. Wrapping the HuggingFace Implementation The spec modified in the previous step now points to a wrapper layer, such asHuggingfaceAttention. This layer inherits from Megatron's MegatronModule. Its core responsibility is to act as a bridge, handling the data alignment required by parallelism strategies (like sequence parallelism), and then internally calling the native Qwen3NextAttentionmodule loaded from HuggingFace. * Corresponding File:slime_plugins/models/hf_attention.py

3. Aligning Model Weights Once the model architecture is integrated, we must ensure that the weights can be loaded correctly. slime keeps the HuggingFace-to-Megatron name mapping and tensor transforms next to its checkpoint loader. * Corresponding File:slime/backends/megatron_utils/hf_to_megatron/qwen3_next.py

Through the coordination of these three components, we can successfully run a complex model architecture not natively supported by Megatron—using its HuggingFace implementation as the vehicle—on top of Megatron's parallel framework. This is achieved while fully retaining all key capabilities like model parallelism, MoE acceleration, and pipeline scheduling.

`Current Limitations`

* This approach does not currently support Tensor Parallelism (TP) within the replaced module itself (e.g., the Attention layer in this case). * Impact: In most large-scale MoE models, the parameter count of the Attention layer is relatively small, so this limitation typically has a minimal effect on memory footprint and training throughput. * Alternative: If TP for the module is critical, the only alternative is to revert to the more invasive approach of modifying Megatron's native implementation.

---

`En/Advanced/Delta Weight Sync`

`Delta Weight Sync`

Delta weight sync keeps non-colocated rollout engines up to date by shipping only the bytes that changed between two syncs, instead of a full checkpoint each time. It targets large-model training/inference disaggregation across clusters or datacenters, where writing the whole actor every sync is the dominant cost.

It is disk-transport only. The trainer publishes each sync as a canonical HF checkpoint directory; the engine's/pull_weightsendpoint (shipped in slime's sglang patch) fans the apply out to every host the engine spans and verifies it, then the engine reloads the patched local checkpoint through the ordinaryupdate_weights_from_diskendpoint. slime only ever talks to one endpoint per engine, so multi-node serving and external rollout engines need nothing extra on the slime side.

`Configuration`

bash

--update-weight-mode delta
--update-weight-transport disk
--update-weight-disk-dir /shared/fs/delta-updates
--update-weight-local-checkpoint-dir /local/nvme/rollout-ckpt
--update-weight-delta-encoding xor          # or: overwrite
--update-weight-delta-checksum xxh3-128     # or: blake3, adler32

| Flag | Role | |---|---| |--update-weight-disk-dir| Shared filesystem directory the trainer publishes deltas to and the rollout hosts read from. | |--update-weight-local-checkpoint-dir | Host-local (e.g. NVMe) full HF checkpoint that /pull_weights keeps in sync — deltas are applied into it in place; a published full checkpoint replaces it. Each host seeds it from the engine's model path on the first /pull_weights. | |--update-weight-delta-encoding | On-disk delta encoding: xor (default) or overwrite. | |--update-weight-delta-checksum | Per-tensor integrity checksum: xxh3-128 (default), blake3, or adler32. |

Deltas are always zstd-compressed (level 1); profiling showed it dominates lz4 / gzip / snappy / brotli on both wire size and decompress speed for this data, so it is not a knob.

`How it works`

1. Seed. On the first sync the trainer captures a CPU snapshot of every parameter — seeded from--hf-checkpoint, which is exactly what each rollout host materializes its local checkpoint from. Nothing is published; this snapshot is the base the next sync diffs against. The trainer also issues/pull_weights with target_version=0so every host materializes its local base now, overlapped with the snapshot capture. 2. Publish. On every later sync the trainer diffs each gathered HF tensor against the snapshot, encodes and compresses the change, and writes a new version directoryweight_v{N:06d}/ under --update-weight-disk-dir. The directory is a canonical HF checkpoint —model-NNNNN.safetensorsfiles holding the compressed diff tensors plus amodel.safetensors.index.json(tensor name → file) carrying the apply metadata — so the artifact is portable, not tied to the trainer's parallelism layout. The snapshot is then advanced to the new values for the next diff. 3. Pull. The trainer calls/pull_weightson each engine. Inside the engine the request is broadcast to every rank on every node; each host applies the new version's delta into its local checkpoint in place (a per-host file lock collapses co-located ranks to one apply). The apply is parallelized across tensors and verified per-tensor (see Integrity); the call only reports success once every host holds a checksum-verified checkpoint.

/pull_weightsis not delta-specific: each published version is self-describing, and a version that is an ordinary full HF checkpoint (no delta metadata in its index) is pulled by copying it as-is — resetting the chain, so a fresh host joining late seeds from the newest full version instead of replaying every delta, and older deltas can be pruned. slime's full-mode disk sync uses exactly this when--update-weight-local-checkpoint-diris set. 4. Reload. The engines reload the patched local checkpoint through the vanillaupdate_weights_from_disk path — the weight-loading code never sees the delta format.

Because the snapshot is seeded from --hf-checkpoint(the engine's actual base) rather than from the current GPU weights, the scheme is correct for any model even where the Megatron→HF round-trip is not byte-exact (e.g. trimmed vocab-padding rows in the embedding / LM head).

`Encodings`

Both encodings are byte-level and dtype-blind, so the same path works for quantized checkpoints. The engine reads the choice from each version's index metadata.

- xor (default): writes new ^ old. Smallest wire and fastest to apply (sequential, cache-friendly; the unchanged bytes are zeros the compressor crushes). It is an involution, so it must be applied exactly once against the correct base — applying it twice reverts. -overwrite: writes the changed positions and their new absolute values. Larger on the wire and a less cache-friendly scattered apply, but idempotent: re-applying it (or finishing a partially-applied delta) converges to the same state regardless of how many times it runs. Use it when re-applicability matters more than wire size.

`Integrity`

The trainer stores a per-tensor checksum of each tensor's new state in the version. After applying, every host recomputes the checksum and raises on any mismatch — the failure propagates through the/pull_weightsresponse, so a corrupt delta or a wrong base fails loud instead of serving bad weights. The apply also refuses to run out of order: a version only applies on top of its declared base version.

--update-weight-delta-checksumselects the algorithm. The checksum is not the apply bottleneck (the apply is decompress + XOR bound), so this is a digest-property choice, not a speed one:xxh3-128 (default) is the widest fast non-cryptographic digest; blake3is cryptographic, for untrusted storage;adler32 is for interop with systems that expect it.

`Shared-filesystem visibility hooks`

On a POSIX shared filesystem (NFS, Lustre, …) no extra step is needed. Object-store-backed mounts that need an explicit publish/refresh to make writes visible across hosts can supply two optional hooks, loaded by import path — no vendor-specific code lives in slime or sglang:

- --custom-update-weight-post-write-path(slime, trainer side): called after a version's files are written, before the engines are told to read it (e.g. upload pending writes to the backing object store). Signature:hook(args, version_dir, rollout_engines). ---sglang-custom-pull-weights-pre-read-hook(sglang server arg, engine side): called on each host inside the engine before/pull_weightsreads the delta directory (e.g. refresh the mount's view). Signature:hook(delta_dir, target_version)`.

---

README Copy Section

slime Documentation

Docs Workflow Copy Chapter

Install Dependency Copy Section

Update Documentation Copy Section

Build and Render Copy Chapter

build english version

build chinese version

En/Get Started/Agent Copy Section

Agentic RL Training Roadmap

Where To Start Copy Chapter

Recommended Integration Pattern Copy Chapter

Agent Runtime Adapters Copy Chapter

Agent client sends requests to adapter.app.

Agent Serving And Performance Copy Chapter

Reference Example Copy Chapter

En/Get Started/Customization Copy Section

Customization Guide

Overview of Customization Interfaces Copy Chapter

Agentic workflows through customization interfaces Copy Chapter

Detailed Interface Reference Copy Chapter

1. Rollout Function (--rollout-function-path) --rollout-function-path)')" title="Copy section prompt for LLMs"> Copy Section

2. Custom Generate Function (--custom-generate-function-path) --custom-generate-function-path)')" title="Copy section prompt for LLMs"> Copy Section

3. Reward Model (--custom-rm-path) --custom-rm-path)')" title="Copy section prompt for LLMs"> Copy Section

4. Dynamic Sampling Filter (--dynamic-sampling-filter-path) --dynamic-sampling-filter-path)')" title="Copy section prompt for LLMs"> Copy Section

5. Buffer Filter (--buffer-filter-path) --buffer-filter-path)')" title="Copy section prompt for LLMs"> Copy Section

6. Rollout Sample Filter (--rollout-sample-filter-path) --rollout-sample-filter-path)')" title="Copy section prompt for LLMs"> Copy Section

7. Rollout All Samples Process (--rollout-all-samples-process-path) --rollout-all-samples-process-path)')" title="Copy section prompt for LLMs"> Copy Section

8. Rollout Data Postprocess (--rollout-data-postprocess-path) --rollout-data-postprocess-path)')" title="Copy section prompt for LLMs"> Copy Section

9. Custom Loss Function (--custom-loss-function-path) --custom-loss-function-path)')" title="Copy section prompt for LLMs"> Copy Section

10. Custom TIS/RS Function (--custom-tis-function-path) --custom-tis-function-path)')" title="Copy section prompt for LLMs"> Copy Section

11. Custom pg_loss Reducer (--custom-pg-loss-reducer-function-path) --custom-pg-loss-reducer-function-path)')" title="Copy section prompt for LLMs"> Copy Section

12. Reward Post-Processing (--custom-reward-post-process-path) --custom-reward-post-process-path)')" title="Copy section prompt for LLMs"> Copy Section

13. Samples to Train Data Conversion (--custom-convert-samples-to-train-data-path) --custom-convert-samples-to-train-data-path)')" title="Copy section prompt for LLMs"> Copy Section

14. Logging Functions Copy Section

15. Data Source (--data-source-path) --data-source-path)')" title="Copy section prompt for LLMs"> Copy Section

16. Evaluation Function (--eval-function-path) --eval-function-path)')" title="Copy section prompt for LLMs"> Copy Section

17. Megatron Hooks Copy Section

18. MoE Routing Replay Copy Section

19. Disk Weight-Sync Post-Write Hook (--custom-update-weight-post-write-path) --custom-update-weight-post-write-path)')" title="Copy section prompt for LLMs"> Copy Section

Testing Custom Function Paths Copy Chapter

En/Get Started/Qa Copy Section

FAQ

En/Get Started/Quick Start Copy Section

Quick Start

Basic Environment Setup Copy Chapter

Hardware Support Copy Section

Pull and Start Docker Container Copy Section

Pull the latest image

Start the container

Install slime Copy Section

Path can be adjusted according to actual situation

Model and Dataset Download Copy Chapter

Download model weights (GLM-Z1-9B)

Download training dataset (dapo-math-17k)

Download evaluation dataset (aime-2024)

Model Weight Conversion Copy Chapter

Convert from Hugging Face Format to Megatron Format Copy Section

Convert from Megatron Format to Hugging Face Format Copy Section

Training Script and Parameter Overview Copy Chapter

MODEL_ARGS: Model Configuration Parameters Copy Section

CKPT_ARGS: Checkpoint and Path Parameters Copy Section

ROLLOUT_ARGS: Data Generation (Rollout) Parameters Copy Section

EVAL_ARGS: Evaluation Parameters Copy Section

PERF_ARGS: Performance and Parallelism Parameters Copy Section

GRPO_ARGS: GRPO Algorithm Parameters Copy Section

OPTIMIZER_ARGS: Optimizer Parameters Copy Section

SGLANG_ARGS: SGLang Service Parameters Copy Section

Feature Introduction Copy Chapter

Colocated Actor and Rollout Copy Section

Dynamic Sampling Copy Section

Partial Rollout Copy Section

bf16 Training fp8 Inference Copy Section

Multiturn Adaptation Copy Chapter

Adaptation Strategy Summary Copy Section

Data Preparation and Mapping Copy Section

Step One: Construct metadata Field in Dataset Field in Dataset')" title="Copy section prompt for LLMs"> Copy Section

Step Two: Specify Mapping in Training Script Copy Section

Writing Custom Generation Function Copy Section

Writing Custom Reward Function Copy Section

README

Docs Workflow

Install Dependency

Update Documentation

Build and Render

En/Get Started/Agent

Where To Start

Recommended Integration Pattern

Agent Runtime Adapters

Agent Serving And Performance

Reference Example

En/Get Started/Customization

Overview of Customization Interfaces

Agentic workflows through customization interfaces

Detailed Interface Reference

1. Rollout Function (`--rollout-function-path`)

2. Custom Generate Function (`--custom-generate-function-path`)

3. Reward Model (`--custom-rm-path`)

4. Dynamic Sampling Filter (`--dynamic-sampling-filter-path`)

5. Buffer Filter (`--buffer-filter-path`)

6. Rollout Sample Filter (`--rollout-sample-filter-path`)

7. Rollout All Samples Process (`--rollout-all-samples-process-path`)

8. Rollout Data Postprocess (`--rollout-data-postprocess-path`)

9. Custom Loss Function (`--custom-loss-function-path`)

10. Custom TIS/RS Function (`--custom-tis-function-path`)

11. Custom pg_loss Reducer (`--custom-pg-loss-reducer-function-path`)

12. Reward Post-Processing (`--custom-reward-post-process-path`)

13. Samples to Train Data Conversion (`--custom-convert-samples-to-train-data-path`)

14. Logging Functions

15. Data Source (`--data-source-path`)

16. Evaluation Function (`--eval-function-path`)

17. Megatron Hooks

18. MoE Routing Replay

19. Disk Weight-Sync Post-Write Hook (`--custom-update-weight-post-write-path`)

Testing Custom Function Paths

En/Get Started/Qa

En/Get Started/Quick Start

Basic Environment Setup

Hardware Support

Pull and Start Docker Container

Install slime

Model and Dataset Download

Model Weight Conversion

Convert from Hugging Face Format to Megatron Format

Convert from Megatron Format to Hugging Face Format

Training Script and Parameter Overview

MODEL_ARGS: Model Configuration Parameters

`CKPT_ARGS: Checkpoint and Path Parameters`

`ROLLOUT_ARGS: Data Generation (Rollout) Parameters`

`EVAL_ARGS: Evaluation Parameters`

`PERF_ARGS: Performance and Parallelism Parameters`

GRPO_ARGS: GRPO Algorithm Parameters

`OPTIMIZER_ARGS: Optimizer Parameters`

`SGLANG_ARGS: SGLang Service Parameters`

`Feature Introduction`

`Colocated Actor and Rollout`

`Dynamic Sampling`

`Partial Rollout`

`bf16 Training fp8 Inference`

`Multiturn Adaptation`

`Adaptation Strategy Summary`

`Data Preparation and Mapping`

`Step One: Construct` metadata `Field in Dataset`

`Step Two: Specify Mapping in Training Script`

`Writing Custom Generation Function`

`Writing Custom Reward Function`

`Configure in Training Script`

`Multi-Node Training for Large-Scale MOE Models`

`En/Get Started/Usage`

`Usage Guide`

`Introduction to slime Parameters`

`Cluster Resource Allocation`

`Choosing Training Backend`

`Loading Megatron`

`Loading SGLang`

`Data Format`

`Hyperparameters for RL Training`

`Advanced Megatron Configuration (--megatron-config-path)`

`Custom Rollout Function`

`How to Use SGLang`