openpi Documentation

File: README.md Copy Chapter

openpi

openpi holds open-source models and packages for robotics, published by the Physical Intelligence team.

Currently, this repo contains three types of models:

• the π₀ model, a flow-based vision-language-action model (VLA).
• the π₀-FAST model, an autoregressive VLA, based on the FAST action tokenizer.
• the π₀.₅ model, an upgraded version of π₀ with better open-world generalization trained with knowledge insulation. Note that, in this repository, we currently only support the flow matching head for both $\pi_{0.5}$ training and inference.

For all models, we provide base model checkpoints, pre-trained on 10k+ hours of robot data, and examples for using them out of the box or fine-tuning them to your own datasets.

This is an experiment: $\pi_0$ was developed for our own robots, which differ from the widely used platforms such as ALOHA and DROID, and though we are optimistic that researchers and practitioners will be able to run creative new experiments adapting $\pi_0$ to their own platforms, we do not expect every such attempt to be successful. All this is to say: $\pi_0$ may or may not work for you, but you are welcome to try it and see!

Updates Copy Chapter

• [Sept 2025] We released PyTorch support in openpi.
• [Sept 2025] We released pi05, an upgraded version of pi0 with better open-world generalization.
• [Sept 2025]: We have added an improved idle filter for DROID training.
• [Jun 2025]: We have added instructions for using openpi to train VLAs on the full DROID dataset. This is an approximate open-source implementation of the training pipeline used to train pi0-FAST-DROID.

Requirements Copy Chapter

To run the models in this repository, you will need an NVIDIA GPU with at least the following specifications. These estimations assume a single GPU, but you can also use multiple GPUs with model parallelism to reduce per-GPU memory requirements by configuring fsdp_devices in the training config. Please also note that the current training script does not yet support multi-node training.

The repo has been tested with Ubuntu 22.04, we do not currently support other operating systems.

Installation Copy Chapter

When cloning this repo, make sure to update submodules:

git clone --recurse-submodules [email protected]:Physical-Intelligence/openpi.git

# Or if you already cloned the repo:
git submodule update --init --recursive

We use uv to manage Python dependencies. See the uv installation instructions to set it up. Once uv is installed, run the following to set up the environment:

GIT_LFS_SKIP_SMUDGE=1 uv sync
GIT_LFS_SKIP_SMUDGE=1 uv pip install -e .

NOTE: GIT_LFS_SKIP_SMUDGE=1 is needed to pull LeRobot as a dependency.

Docker: As an alternative to uv installation, we provide instructions for installing openpi using Docker. If you encounter issues with your system setup, consider using Docker to simplify installation. See Docker Setup for more details.

Model Checkpoints Copy Chapter

Base Models Copy Section

We provide multiple base VLA model checkpoints. These checkpoints have been pre-trained on 10k+ hours of robot data, and can be used for fine-tuning.

Fine-Tuned Models Copy Section

We also provide "expert" checkpoints for various robot platforms and tasks. These models are fine-tuned from the base models above and intended to run directly on the target robot. These may or may not work on your particular robot. Since these checkpoints were fine-tuned on relatively small datasets collected with more widely available robots, such as ALOHA and the DROID Franka setup, they might not generalize to your particular setup, though we found some of these, especially the DROID checkpoint, to generalize quite broadly in practice.

By default, checkpoints are automatically downloaded from gs://openpi-assets and are cached in ~/.cache/openpi when needed. You can overwrite the download path by setting the OPENPI_DATA_HOME environment variable.

Running Inference for a Pre-Trained Model Copy Chapter

Our pre-trained model checkpoints can be run with a few lines of code (here our $\pi_0$-FAST-DROID model):

from openpi.training import config as _config
from openpi.policies import policy_config
from openpi.shared import download

config = _config.get_config("pi05_droid")
checkpoint_dir = download.maybe_download("gs://openpi-assets/checkpoints/pi05_droid")

# Create a trained policy.
policy = policy_config.create_trained_policy(config, checkpoint_dir)

# Run inference on a dummy example.
example = {
    "observation/exterior_image_1_left": ...,
    "observation/wrist_image_left": ...,
    ...
    "prompt": "pick up the fork"
}
action_chunk = policy.infer(example)["actions"]

You can also test this out in the example notebook.

We provide detailed step-by-step examples for running inference of our pre-trained checkpoints on DROID and ALOHA robots.

Remote Inference: We provide examples and code for running inference of our models remotely: the model can run on a different server and stream actions to the robot via a websocket connection. This makes it easy to use more powerful GPUs off-robot and keep robot and policy environments separate.

Test inference without a robot: We provide a script for testing inference without a robot. This script will generate a random observation and run inference with the model. See here for more details.

Fine-Tuning Base Models on Your Own Data Copy Chapter

We will fine-tune the $\pi_{0.5}$ model on the LIBERO dataset as a running example for how to fine-tune a base model on your own data. We will explain three steps:

1. Convert your data to a LeRobot dataset (which we use for training)
2. Defining training configs and running training
3. Spinning up a policy server and running inference

1. Convert your data to a LeRobot dataset Copy Section

We provide a minimal example script for converting LIBERO data to a LeRobot dataset in examples/libero/convert_libero_data_to_lerobot.py. You can easily modify it to convert your own data! You can download the raw LIBERO dataset from here, and run the script with:

uv run examples/libero/convert_libero_data_to_lerobot.py --data_dir /path/to/your/libero/data

Note: If you just want to fine-tune on LIBERO, you can skip this step, because our LIBERO fine-tuning configs point to a pre-converted LIBERO dataset. This step is merely an example that you can adapt to your own data.

2. Defining training configs and running training Copy Section

To fine-tune a base model on your own data, you need to define configs for data processing and training. We provide example configs with detailed comments for LIBERO below, which you can modify for your own dataset:

• LiberoInputs and LiberoOutputs: Defines the data mapping from the LIBERO environment to the model and vice versa. Will be used for both, training and inference.
• LeRobotLiberoDataConfig: Defines how to process raw LIBERO data from LeRobot dataset for training.
• TrainConfig: Defines fine-tuning hyperparameters, data config, and weight loader.

We provide example fine-tuning configs for π₀, π₀-FAST, and π₀.₅ on LIBERO data.

Before we can run training, we need to compute the normalization statistics for the training data. Run the script below with the name of your training config:

uv run scripts/compute_norm_stats.py --config-name pi05_libero

Now we can kick off training with the following command (the --overwrite flag is used to overwrite existing checkpoints if you rerun fine-tuning with the same config):

XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 uv run scripts/train.py pi05_libero --exp-name=my_experiment --overwrite

The command will log training progress to the console and save checkpoints to the checkpoints directory. You can also monitor training progress on the Weights & Biases dashboard. For maximally using the GPU memory, set XLA_PYTHON_CLIENT_MEM_FRACTION=0.9 before running training -- this enables JAX to use up to 90% of the GPU memory (vs. the default of 75%).

Note: We provide functionality for reloading normalization statistics for state / action normalization from pre-training. This can be beneficial if you are fine-tuning to a new task on a robot that was part of our pre-training mixture. For more details on how to reload normalization statistics, see the norm_stats.md file.

3. Spinning up a policy server and running inference Copy Section

Once training is complete, we can run inference by spinning up a policy server and then querying it from a LIBERO evaluation script. Launching a model server is easy (we use the checkpoint for iteration 20,000 for this example, modify as needed):

uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi05_libero --policy.dir=checkpoints/pi05_libero/my_experiment/20000

This will spin up a server that listens on port 8000 and waits for observations to be sent to it. We can then run an evaluation script (or robot runtime) that queries the server.

For running the LIBERO eval in particular, we provide (and recommend using) a Dockerized workflow that handles both the policy server and the evaluation script together. See the LIBERO README for more details.

If you want to embed a policy server call in your own robot runtime, we have a minimal example of how to do so in the remote inference docs.

More Examples Copy Section

We provide more examples for how to fine-tune and run inference with our models on the ALOHA platform in the following READMEs:

• ALOHA Simulator
• ALOHA Real
• UR5

PyTorch Support Copy Chapter

openpi now provides PyTorch implementations of π₀ and π₀.₅ models alongside the original JAX versions! The PyTorch implementation has been validated on the LIBERO benchmark (both inference and finetuning). A few features are currently not supported (this may change in the future):

• The π₀-FAST model
• Mixed precision training
• FSDP (fully-sharded data parallelism) training
• LoRA (low-rank adaptation) training
• EMA (exponential moving average) weights during training

Setup Copy Section

1. Make sure that you have the latest version of all dependencies installed: uv sync

2. Double check that you have transformers 4.53.2 installed: uv pip show transformers

3. Apply the transformers library patches:

   bash Copy

   cp -r ./src/openpi/models_pytorch/transformers_replace/* .venv/lib/python3.11/site-packages/transformers/

This overwrites several files in the transformers library with necessary model changes: 1) supporting AdaRMS, 2) correctly controlling the precision of activations, and 3) allowing the KV cache to be used without being updated.

WARNING: With the default uv link mode (hardlink), this will permanently affect the transformers library in your uv cache, meaning the changes will survive reinstallations of transformers and could even propagate to other projects that use transformers. To fully undo this operation, you must run uv cache clean transformers.

Converting JAX Models to PyTorch Copy Section

To convert a JAX model checkpoint to PyTorch format:

uv run examples/convert_jax_model_to_pytorch.py \
    --checkpoint_dir /path/to/jax/checkpoint \
    --config_name <config name> \
    --output_path /path/to/converted/pytorch/checkpoint

Running Inference with PyTorch Copy Section

The PyTorch implementation uses the same API as the JAX version - you only need to change the checkpoint path to point to the converted PyTorch model:

from openpi.training import config as _config
from openpi.policies import policy_config
from openpi.shared import download

config = _config.get_config("pi05_droid")
checkpoint_dir = "/path/to/converted/pytorch/checkpoint"

# Create a trained policy (automatically detects PyTorch format)
policy = policy_config.create_trained_policy(config, checkpoint_dir)

# Run inference (same API as JAX)
action_chunk = policy.infer(example)["actions"]

Policy Server with PyTorch Copy Section

The policy server works identically with PyTorch models - just point to the converted checkpoint directory:

uv run scripts/serve_policy.py policy:checkpoint \
    --policy.config=pi05_droid \
    --policy.dir=/path/to/converted/pytorch/checkpoint

Finetuning with PyTorch Copy Section

To finetune a model in PyTorch:

1. Convert the JAX base model to PyTorch format:

   bash Copy

   uv run examples/convert_jax_model_to_pytorch.py \
       --config_name <config name> \
       --checkpoint_dir /path/to/jax/base/model \
       --output_path /path/to/pytorch/base/model

2. Specify the converted PyTorch model path in your config using pytorch_weight_path

3. Launch training using one of these modes:

# Single GPU training:
uv run scripts/train_pytorch.py <config_name> --exp_name <run_name> --save_interval 

# Example:
uv run scripts/train_pytorch.py debug --exp_name pytorch_test
uv run scripts/train_pytorch.py debug --exp_name pytorch_test --resume  # Resume from latest checkpoint

# Multi-GPU training (single node):
uv run torchrun --standalone --nnodes=1 --nproc_per_node=<num_gpus> scripts/train_pytorch.py <config_name> --exp_name <run_name>

# Example:
uv run torchrun --standalone --nnodes=1 --nproc_per_node=2 scripts/train_pytorch.py pi0_aloha_sim --exp_name pytorch_ddp_test
uv run torchrun --standalone --nnodes=1 --nproc_per_node=2 scripts/train_pytorch.py pi0_aloha_sim --exp_name pytorch_ddp_test --resume

# Multi-Node Training:
uv run torchrun \
    --nnodes=<num_nodes> \
    --nproc_per_node=<gpus_per_node> \
    --node_rank=<rank_of_node> \
    --master_addr=<master_ip> \
    --master_port= \
    scripts/train_pytorch.py <config_name> --exp_name=<run_name> --save_interval

Precision Settings Copy Section

JAX and PyTorch implementations handle precision as follows:

JAX:

1. Inference: most weights and computations in bfloat16, with a few computations in float32 for stability
2. Training: defaults to mixed precision: weights and gradients in float32, (most) activations and computations in bfloat16. You can change to full float32 training by setting dtype to float32 in the config.

PyTorch:

1. Inference: matches JAX -- most weights and computations in bfloat16, with a few weights converted to float32 for stability
2. Training: supports either full bfloat16 (default) or full float32. You can change it by setting pytorch_training_precision in the config. bfloat16 uses less memory but exhibits higher losses compared to float32. Mixed precision is not yet supported.

With torch.compile, inference speed is comparable between JAX and PyTorch.

Troubleshooting Copy Chapter

We will collect common issues and their solutions here. If you encounter an issue, please check here first. If you can't find a solution, please file an issue on the repo (see here for guidelines).

File: docs/docker.md Copy Chapter

Docker Setup Copy Section

All of the examples in this repo provide instructions for being run normally, and also using Docker. Although not required, the Docker option is recommended as this will simplify software installation, produce a more stable environment, and also allow you to avoid installing ROS and cluttering your machine, for examples which depend on ROS.

• Basic Docker installation instructions are here.
• Docker must be installed in rootless mode.
• To use your GPU you must also install the NVIDIA container toolkit.
• The version of docker installed with snap is incompatible with the NVIDIA container toolkit, preventing it from accessing libnvidia-ml.so (issue). The snap version can be uninstalled with sudo snap remove docker.
• Docker Desktop is also incompatible with the NVIDIA runtime (issue). Docker Desktop can be uninstalled with sudo apt remove docker-desktop.

If starting from scratch and your host machine is Ubuntu 22.04, you can use accomplish all of the above with the convenience scripts scripts/docker/install_docker_ubuntu22.sh and scripts/docker/install_nvidia_container_toolkit.sh.

Build the Docker image and start the container with the following command:

docker compose -f scripts/docker/compose.yml up --build

To build and run the Docker image for a specific example, use the following command:

docker compose -f examples/<example_name>/compose.yml up --build

where <example_name> is the name of the example you want to run.

During the first run of any example, Docker will build the images. Go grab a coffee while this happens. Subsequent runs will be faster since the images are cached.

File: docs/norm_stats.md Copy Chapter

Normalization statistics

Following common practice, our models normalize the proprioceptive state inputs and action targets during policy training and inference. The statistics used for normalization are computed over the training data and stored alongside the model checkpoint.

Reloading normalization statistics Copy Chapter

When you fine-tune one of our models on a new dataset, you need to decide whether to (A) reuse existing normalization statistics or (B) compute new statistics over your new training data. Which option is better for you depends on the similarity of your robot and task to the robot and task distribution in the pre-training dataset. Below, we list all the available pre-training normalization statistics for each model.

If your target robot matches one of these pre-training statistics, consider reloading the same normalization statistics. By reloading the normalization statistics, the actions in your dataset will be more "familiar" to the model, which can lead to better performance. You can reload the normalization statistics by adding an AssetsConfig to your training config that points to the corresponding checkpoint directory and normalization statistics ID, like below for the Trossen (aka ALOHA) robot statistics of the pi0_base checkpoint:

TrainConfig(
    ...
    data=LeRobotAlohaDataConfig(
        ...
        assets=AssetsConfig(
            assets_dir="gs://openpi-assets/checkpoints/pi0_base/assets",
            asset_id="trossen",
        ),
    ),
)

For an example of a full training config that reloads normalization statistics, see the pi0_aloha_pen_uncap config in the training config file.

Note: To successfully reload normalization statistics, it's important that your robot + dataset are following the action space definitions used in pre-training. We provide a detailed description of our action space definitions below.

Note #2: Whether reloading normalization statistics is beneficial depends on the similarity of your robot and task to the robot and task distribution in the pre-training dataset. We recommend to always try both, reloading and training with a fresh set of statistics computed on your new dataset (see main README for instructions on how to compute new statistics), and pick the one that works better for your task.

Provided Pre-training Normalization Statistics Copy Chapter

Below is a list of all the pre-training normalization statistics we provide. We provide them for both, the pi0_base and pi0_fast_base models. For pi0_base, set the assets_dir to gs://openpi-assets/checkpoints/pi0_base/assets and for pi0_fast_base, set the assets_dir to gs://openpi-assets/checkpoints/pi0_fast_base/assets.

Pi0 Model Action Space Definitions Copy Chapter

Out of the box, both the pi0_base and pi0_fast_base use the following action space definitions (left and right are defined looking from behind the robot towards the workspace):

"dim_0:dim_5": "left arm joint angles",
    "dim_6": "left arm gripper position",
    "dim_7:dim_12": "right arm joint angles (for bi-manual only)",
    "dim_13": "right arm gripper position (for bi-manual only)",

    # For mobile robots:
    "dim_14:dim_15": "x-y base velocity (for mobile robots only)",

The proprioceptive state uses the same definitions as the action space, except for the base x-y position (the last two dimensions) for mobile robots, which we don't include in the proprioceptive state.

For 7-DoF robots (e.g. Franka), we use the first 7 dimensions of the action space for the joint actions, and the 8th dimension for the gripper action.

General info for Pi robots:

• Joint angles are expressed in radians, with position zero corresponding to the zero position reported by each robot's interface library, except for ALOHA, where the standard ALOHA code uses a slightly different convention (see the ALOHA example code for details).
• Gripper positions are in [0.0, 1.0], with 0.0 corresponding to fully open and 1.0 corresponding to fully closed.
• Control frequencies are either 20 Hz for UR5e and Franka, and 50 Hz for ARX and Trossen (ALOHA) arms.

For DROID, we use the original DROID action configuration, with joint velocity actions in the first 7 dimensions and gripper actions in the 8th dimension + a control frequency of 15 Hz.

File: docs/remote_inference.md Copy Chapter

Running openpi models remotely

We provide utilities for running openpi models remotely. This is useful for running inference on more powerful GPUs off-robot, and also helps keep the robot and policy environments separate (and e.g. avoid dependency hell with robot software).

Starting a remote policy server Copy Chapter

To start a remote policy server, you can simply run the following command:

uv run scripts/serve_policy.py --env=[DROID | ALOHA | LIBERO]

The env argument specifies which $\pi_0$ checkpoint should be loaded. Under the hood, this script will execute a command like the following, which you can use to start a policy server, e.g. for checkpoints you trained yourself (here an example for the DROID environment):

uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi0_fast_droid --policy.dir=gs://openpi-assets/checkpoints/pi0_fast_droid

This will start a policy server that will serve the policy specified by the config and dir arguments. The policy will be served on the specified port (default: 8000).

Querying the remote policy server from your robot code Copy Chapter

We provide a client utility with minimal dependencies that you can easily embed into any robot codebase.

First, install the openpi-client package in your robot environment:

cd $OPENPI_ROOT/packages/openpi-client
pip install -e .

Then, you can use the client to query the remote policy server from your robot code. Here's an example of how to do this:

from openpi_client import image_tools
from openpi_client import websocket_client_policy

# Outside of episode loop, initialize the policy client.
# Point to the host and port of the policy server (localhost and 8000 are the defaults).
client = websocket_client_policy.WebsocketClientPolicy(host="localhost", port=8000)

for step in range(num_steps):
    # Inside the episode loop, construct the observation.
    # Resize images on the client side to minimize bandwidth / latency. Always return images in uint8 format.
    # We provide utilities for resizing images + uint8 conversion so you match the training routines.
    # The typical resize_size for pre-trained pi0 models is 224.
    # Note that the proprioceptive `state` can be passed unnormalized, normalization will be handled on the server side.
    observation = {
        "observation/image": image_tools.convert_to_uint8(
            image_tools.resize_with_pad(img, 224, 224)
        ),
        "observation/wrist_image": image_tools.convert_to_uint8(
            image_tools.resize_with_pad(wrist_img, 224, 224)
        ),
        "observation/state": state,
        "prompt": task_instruction,
    }

    # Call the policy server with the current observation.
    # This returns an action chunk of shape (action_horizon, action_dim).
    # Note that you typically only need to call the policy every N steps and execute steps
    # from the predicted action chunk open-loop in the remaining steps.
    action_chunk = client.infer(observation)["actions"]

    # Execute the actions in the environment.
    ...

Here, the host and port arguments specify the IP address and port of the remote policy server. You can also specify these as command-line arguments to your robot code, or hard-code them in your robot codebase. The observation is a dictionary of observations and the prompt, following the specification of the policy inputs for the policy you are serving. We have concrete examples of how to construct this dictionary for different environments in the simple client example.

File: examples/droid/README.md Copy Chapter

DROID Policies in openpi

We offer instructions for:

• Running inference for our best $pi_{0.5}$-DROID policy
• Running inference for other pre-trained DROID policies ($\pi_0$, $\pi_0$-FAST, ...)
• Pre-training generalist policies on the full DROID dataset
• Fine-tuning expert $\pi_{0.5}$ on your custom DROID dataset

Running DROID Inference Copy Chapter

This example shows how to run the fine-tuned $\pi_{0.5}$-DROID model on the DROID robot platform. Based on the public RoboArena benchmark, this is currently our strongest generalist DROID policy.

Step 1: Start a policy server Copy Section

Since the DROID control laptop does not have a powerful GPU, we will start a remote policy server on a different machine with a more powerful GPU and then query it from the DROID control laptop during inference.

1. On a machine with a powerful GPU (~NVIDIA 4090), clone and install the openpi repository following the instructions in the README.
2. Start the OpenPI server via the following command:

uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi05_droid --policy.dir=gs://openpi-assets/checkpoints/pi05_droid

You can also run the equivalent command below:

uv run scripts/serve_policy.py --env=DROID

Step 2: Run the DROID robot Copy Section

1. Make sure you have the most recent version of the DROID package installed on both the DROID control laptop and the NUC.
2. On the control laptop, activate your DROID conda environment.
3. Clone the openpi repo and install the openpi client, which we will use to connect to the policy server (this has very few dependencies and should be very fast to install): with the DROID conda environment activated, run cd $OPENPI_ROOT/packages/openpi-client && pip install -e ..
4. Install tyro, which we will use for command line parsing: pip install tyro.
5. Copy the main.py file from this directory to the $DROID_ROOT/scripts directory.
6. Replace the camera IDs in the main.py file with the IDs of your cameras (you can find the camera IDs by running ZED_Explorer in the command line, which will open a tool that shows you all connected cameras and their IDs -- you can also use it to make sure that the cameras are well-positioned to see the scene you want the robot to interact with).
7. Run the main.py file. Make sure to point the IP and host address to the policy server. (To make sure the server machine is reachable from the DROID laptop, you can run ping <server_ip> from the DROID laptop.) Also make sure to specify the external camera to use for the policy (we only input one external camera), choose from ["left", "right"].

python3 scripts/main.py --remote_host=<server_ip> --remote_port=<server_port> --external_camera="left"

The script will ask you to enter a free-form language instruction for the robot to follow. Make sure to point the cameras at the scene you want the robot to interact with. You do not need to carefully control camera angle, object positions, etc. The policy is fairly robust in our experience. Happy prompting!

Troubleshooting Copy Chapter

Running Other Policies Copy Chapter

We provide configs for running the baseline DROID policies from the RoboArena paper. Simply run the commands below to start inference servers for the respective policies. Then follow the instructions above to run evaluation on the DROID robot.

# Train from pi0-FAST, using FAST tokenizer
uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi0_fast_droid --policy.dir=gs://openpi-assets/checkpoints/pi0_fast_droid

# Train from pi0, using flow matching
uv run scripts/serve_policy.py policy:checkpoint --policy.config=pi0_droid --policy.dir=gs://openpi-assets/checkpoints/pi0_droid

# Trained from PaliGemma, using RT-2 / OpenVLA style binning tokenizer.
uv run scripts/serve_policy.py policy:checkpoint --policy.config=paligemma_binning_droid --policy.dir=gs://openpi-assets/checkpoints/roboarena/paligemma_binning_droid

# Trained from PaliGemma, using FAST tokenizer (using universal FAST+ tokenizer).
uv run scripts/serve_policy.py policy:checkpoint --policy.config=paligemma_fast_droid --policy.dir=gs://openpi-assets/checkpoints/roboarena/paligemma_fast_droid

# Trained from PaliGemma, using FAST tokenizer (tokenizer trained on DROID dataset).
uv run scripts/serve_policy.py policy:checkpoint --policy.config=paligemma_fast_specialist_droid --policy.dir=gs://openpi-assets/checkpoints/roboarena/paligemma_fast_specialist_droid

# Trained from PaliGemma, using FSQ tokenizer.
uv run scripts/serve_policy.py policy:checkpoint --policy.config=paligemma_vq_droid --policy.dir=gs://openpi-assets/checkpoints/roboarena/paligemma_vq_droid

# pi0-style diffusion / flow VLA, trained on DROID from PaliGemma.
uv run scripts/serve_policy.py policy:checkpoint --policy.config=paligemma_diffusion_droid --policy.dir=gs://openpi-assets/checkpoints/roboarena/paligemma_diffusion_droid

You can find the inference configs in roboarena_config.py.

File: examples/aloha_sim/README.md Copy Chapter

Run Aloha Sim

With Docker Copy Chapter

export SERVER_ARGS="--env ALOHA_SIM"
docker compose -f examples/aloha_sim/compose.yml up --build

Without Docker Copy Chapter

Terminal window 1:

# Create virtual environment
uv venv --python 3.10 examples/aloha_sim/.venv
source examples/aloha_sim/.venv/bin/activate
uv pip sync examples/aloha_sim/requirements.txt
uv pip install -e packages/openpi-client

# Run the simulation
MUJOCO_GL=egl python examples/aloha_sim/main.py

Note: If you are seeing EGL errors, you may need to install the following dependencies:

sudo apt-get install -y libegl1-mesa-dev libgles2-mesa-dev

Terminal window 2:

# Run the server
uv run scripts/serve_policy.py --env ALOHA_SIM

File: examples/aloha_real/README.md Copy Chapter

Run Aloha (Real Robot)

This example demonstrates how to run with a real robot using an ALOHA setup. See here for instructions on how to load checkpoints and run inference. We list the relevant checkpoint paths for each provided fine-tuned model below.

Prerequisites Copy Chapter

This repo uses a fork of the ALOHA repo, with very minor modifications to use Realsense cameras.

1. Follow the hardware installation instructions in the ALOHA repo.
2. Modify the third_party/aloha/aloha_scripts/realsense_publisher.py file to use serial numbers for your cameras.

With Docker Copy Chapter

export SERVER_ARGS="--env ALOHA --default_prompt='take the toast out of the toaster'"
docker compose -f examples/aloha_real/compose.yml up --build

Without Docker Copy Chapter

Terminal window 1:

# Create virtual environment
uv venv --python 3.10 examples/aloha_real/.venv
source examples/aloha_real/.venv/bin/activate
uv pip sync examples/aloha_real/requirements.txt
uv pip install -e packages/openpi-client

# Run the robot
python -m examples.aloha_real.main

Terminal window 2:

roslaunch aloha ros_nodes.launch

Terminal window 3:

uv run scripts/serve_policy.py --env ALOHA --default_prompt='take the toast out of the toaster'

**ALOHA Checkpoint Guide** Copy Chapter

The pi0_base model can be used in zero shot for a simple task on the ALOHA platform, and we additionally provide two example fine-tuned checkpoints, “fold the towel” and “open the tupperware and put the food on the plate,” which can perform more advanced tasks on the ALOHA.

While we’ve found the policies to work in unseen conditions across multiple ALOHA stations, we provide some pointers here on how best to set up scenes to maximize the chance of policy success. We cover the prompts to use for the policies, objects we’ve seen it work well on, and well-represented initial state distributions. Running these policies in zero shot is still a very experimental feature, and there is no guarantee that they will work on your robot. The recommended way to use pi0_base is by finetuning with data from the target robot.

**Toast Task** Copy Section

This task involves the robot taking two pieces of toast out of a toaster and placing them on a plate.

• Checkpoint path: gs://openpi-assets/checkpoints/pi0_base
• Prompt: "take the toast out of the toaster"
• Objects needed: Two pieces of toast, a plate, and a standard toaster.
• Object Distribution:
  • Works on both real toast and rubber fake toast
  • Compatible with standard 2-slice toasters
  • Works with plates of varying colors

**Scene Setup Guidelines** Copy Section

• The toaster should be positioned in the top-left quadrant of the workspace.
• Both pieces of toast should start inside the toaster, with at least 1 cm of bread sticking out from the top.
• The plate should be placed roughly in the lower-center of the workspace.
• Works with both natural and synthetic lighting, but avoid making the scene too dark (e.g., don't place the setup inside an enclosed space or under a curtain).

**Towel Task** Copy Section

This task involves folding a small towel (e.g., roughly the size of a hand towel) into eighths.

• Checkpoint path: gs://openpi-assets/checkpoints/pi0_aloha_towel
• Prompt: "fold the towel"
• Object Distribution:
  • Works on towels of varying solid colors
  • Performance is worse on heavily textured or striped towels

**Scene Setup Guidelines** Copy Section

• The towel should be flattened and roughly centered on the table.
• Choose a towel that does not blend in with the table surface.

**Tupperware Task** Copy Section

This task involves opening a tupperware filled with food and pouring the contents onto a plate.

• Checkpoint path: gs://openpi-assets/checkpoints/pi0_aloha_tupperware
• Prompt: "open the tupperware and put the food on the plate"
• Objects needed: Tupperware, food (or food-like items), and a plate.
• Object Distribution:
  • Works on various types of fake food (e.g., fake chicken nuggets, fries, and fried chicken).
  • Compatible with tupperware of different lid colors and shapes, with best performance on square tupperware with a corner flap (see images below).
  • The policy has seen plates of varying solid colors.

**Scene Setup Guidelines** Copy Section

• Best performance observed when both the tupperware and plate are roughly centered in the workspace.
• Positioning:
  • Tupperware should be on the left.
  • Plate should be on the right or bottom.
  • The tupperware flap should point toward the plate.

Training on your own Aloha dataset Copy Chapter

1. Convert the dataset to the LeRobot dataset v2.0 format.

   We provide a script convert_aloha_data_to_lerobot.py that converts the dataset to the LeRobot dataset v2.0 format. As an example we have converted the aloha_pen_uncap_diverse_raw dataset from the BiPlay repo and uploaded it to the HuggingFace Hub as physical-intelligence/aloha_pen_uncap_diverse.

2. Define a training config that uses the custom dataset.

   We provide the pi0_aloha_pen_uncap config as an example. You should refer to the root README for how to run training with the new config.

IMPORTANT: Our base checkpoint includes normalization stats from various common robot configurations. When fine-tuning a base checkpoint with a custom dataset from one of these configurations, we recommend using the corresponding normalization stats provided in the base checkpoint. In the example, this is done by specifying the trossen asset_id and a path to the pretrained checkpoint’s asset directory within the AssetsConfig.

File: examples/libero/README.md Copy Chapter

LIBERO Benchmark

This example runs the LIBERO benchmark: https://github.com/Lifelong-Robot-Learning/LIBERO

Note: When updating requirements.txt in this directory, there is an additional flag --extra-index-url https://download.pytorch.org/whl/cu113 that must be added to the uv pip compile command.

This example requires git submodules to be initialized. Don't forget to run:

git submodule update --init --recursive

With Docker (recommended) Copy Chapter

# Grant access to the X11 server:
sudo xhost +local:docker

# To run with the default checkpoint and task suite:
SERVER_ARGS="--env LIBERO" docker compose -f examples/libero/compose.yml up --build

# To run with glx for Mujoco instead (use this if you have egl errors):
MUJOCO_GL=glx SERVER_ARGS="--env LIBERO" docker compose -f examples/libero/compose.yml up --build

You can customize the loaded checkpoint by providing additional SERVER_ARGS (see scripts/serve_policy.py), and the LIBERO task suite by providing additional CLIENT_ARGS (see examples/libero/main.py).
For example:

# To load a custom checkpoint (located in the top-level openpi/ directory):
export SERVER_ARGS="--env LIBERO policy:checkpoint --policy.config pi05_libero --policy.dir ./my_custom_checkpoint"

# To run the libero_10 task suite:
export CLIENT_ARGS="--args.task-suite-name libero_10"

Without Docker (not recommended) Copy Chapter

Terminal window 1:

# Create virtual environment
uv venv --python 3.8 examples/libero/.venv
source examples/libero/.venv/bin/activate
uv pip sync examples/libero/requirements.txt third_party/libero/requirements.txt --extra-index-url https://download.pytorch.org/whl/cu113 --index-strategy=unsafe-best-match
uv pip install -e packages/openpi-client
uv pip install -e third_party/libero
export PYTHONPATH=$PYTHONPATH:$PWD/third_party/libero

# Run the simulation
python examples/libero/main.py

# To run with glx for Mujoco instead (use this if you have egl errors):
MUJOCO_GL=glx python examples/libero/main.py

Terminal window 2:

# Run the server
uv run scripts/serve_policy.py --env LIBERO

Results Copy Chapter

If you want to reproduce the following numbers, you can evaluate the checkpoint at gs://openpi-assets/checkpoints/pi05_libero/. This
checkpoint was trained in openpi with the pi05_libero config.

File: examples/simple_client/README.md Copy Chapter

Simple Client

A minimal client that sends observations to the server and prints the inference rate.

You can specify which runtime environment to use using the --env flag. You can see the available options by running:

uv run examples/simple_client/main.py --help

With Docker Copy Chapter

export SERVER_ARGS="--env ALOHA_SIM"
docker compose -f examples/simple_client/compose.yml up --build

Without Docker Copy Chapter

Terminal window 1:

uv run examples/simple_client/main.py --env DROID

Terminal window 2:

uv run scripts/serve_policy.py --env DROID

File: examples/ur5/README.md Copy Chapter

UR5 Example

Below we provide an outline of how to implement the key components mentioned in the "Finetune on your data" section of the README for finetuning on UR5 datasets.

First, we will define the UR5Inputs and UR5Outputs classes, which map the UR5 environment to the model and vice versa. Check the corresponding files in src/openpi/policies/libero_policy.py for comments explaining each line.

/* Detailed source-code truncated for AI context efficiency. */

Next, we will define the UR5DataConfig class, which defines how to process raw UR5 data from LeRobot dataset for training. For a full example, see the LeRobotLiberoDataConfig config in the training config file.

/* Detailed source-code truncated for AI context efficiency. */

Finally, we define the TrainConfig for our UR5 dataset. Here, we define a config for fine-tuning pi0 on our UR5 dataset. See the training config file for more examples, e.g. for pi0-FAST or for LoRA fine-tuning.

TrainConfig(
    name="pi0_ur5",
    model=pi0.Pi0Config(),
    data=LeRobotUR5DataConfig(
        repo_id="your_username/ur5_dataset",
        # This config lets us reload the UR5 normalization stats from the base model checkpoint.
        # Reloading normalization stats can help transfer pre-trained models to new environments.
        # See the [norm_stats.md](../docs/norm_stats.md) file for more details.
        assets=AssetsConfig(
            assets_dir="gs://openpi-assets/checkpoints/pi0_base/assets",
            asset_id="ur5e",
        ),
        base_config=DataConfig(
            # This flag determines whether we load the prompt (i.e. the task instruction) from the
            # ``task`` field in the LeRobot dataset. The recommended setting is True.
            prompt_from_task=True,
        ),
    ),
    # Load the pi0 base model checkpoint.
    weight_loader=weight_loaders.CheckpointWeightLoader("gs://openpi-assets/checkpoints/pi0_base/params"),
    num_train_steps=30_000,
)