Gymnasium Documentation

File: README.md Copy Chapter

Gymnasium is an open source Python library for developing and comparing reinforcement learning algorithms by providing a standard API to communicate between learning algorithms and environments, as well as a standard set of environments compliant with that API. This is a fork of OpenAI's Gym library by its maintainers (OpenAI handed over maintenance a few years ago to an outside team), and is where future maintenance will occur going forward.

The documentation website is at gymnasium.farama.org, and we have a public discord server (which we also use to coordinate development work) that you can join here: https://discord.gg/bnJ6kubTg6

Environments Copy Chapter

Gymnasium includes the following families of environments along with a wide variety of third-party environments

• Classic Control - These are classic reinforcement learning based on real-world problems and physics.
• Box2D - These environments all involve toy games based around physics control, using box2d based physics and PyGame-based rendering
• Toy Text - These environments are designed to be extremely simple, with small discrete state and action spaces, and hence easy to learn. As a result, they are suitable for debugging implementations of reinforcement learning algorithms.
• MuJoCo - A physics engine based environments with multi-joint control which are more complex than the Box2D environments.
• Atari - Emulator of Atari 2600 ROMs simulated that have a high range of complexity for agents to learn.
• Third-party - A number of environments have been created that are compatible with the Gymnasium API. Be aware of the version that the software was created for and use the apply_env_compatibility in gymnasium.make if necessary.

Installation Copy Chapter

To install the base Gymnasium library, use pip install gymnasium

This does not include dependencies for all families of environments (there's a massive number, and some can be problematic to install on certain systems). You can install these dependencies for one family like pip install "gymnasium[atari]" or use pip install "gymnasium[all]" to install all dependencies.

We support and test for Python 3.10, 3.11, 3.12, 3.13 and 3.14 on Linux and macOS. We will accept PRs related to Windows, but do not officially support it.

API Copy Chapter

The Gymnasium API models environments as simple Python env classes. Creating environment instances and interacting with them is very simple- here's an example using the "CartPole-v1" environment:

import gymnasium as gym
env = gym.make("CartPole-v1")

observation, info = env.reset(seed=42)
for _ in range(1000):
    action = env.action_space.sample()
    observation, reward, terminated, truncated, info = env.step(action)

    if terminated or truncated:
        observation, info = env.reset()
env.close()

Notable Related Libraries Copy Chapter

Please note that this is an incomplete list, and just includes libraries that the maintainers most commonly point newcomers to when asked for recommendations.

• CleanRL is a learning library based on the Gymnasium API. It is designed to cater to newer people in the field and provides very good reference implementations.
• PettingZoo is a multi-agent version of Gymnasium with a number of implemented environments, for example, multi-agent Atari environments.
• The Farama Foundation also has a collection of many other environments that are maintained by the same team as Gymnasium and use the Gymnasium API.

Environment Versioning Copy Chapter

Gymnasium keeps strict versioning for reproducibility reasons. All environments end in a suffix like "-v0". When changes are made to environments that might impact learning results, the number is increased by one to prevent potential confusion. These were inherited from Gym.

Contributing Copy Chapter

We welcome contributions from the community!
Please see our CONTRIBUTING.md for details on how to get started.

Support Gymnasium's Development Copy Chapter

If you are financially able to do so and would like to support the development of Gymnasium, please join others in the community in donating to us.

Citation Copy Chapter

You can cite Gymnasium using our related paper (https://arxiv.org/abs/2407.17032) as:

@article{towers2024gymnasium,
  title={Gymnasium: A Standard Interface for Reinforcement Learning Environments},
  author={Towers, Mark and Kwiatkowski, Ariel and Terry, Jordan and Balis, John U and De Cola, Gianluca and Deleu, Tristan and Goul{\~a}o, Manuel and Kallinteris, Andreas and Krimmel, Markus and KG, Arjun and others},
  journal={arXiv preprint arXiv:2407.17032},
  year={2024}
}

File: docs/index.md Copy Chapter

hide-toc: true firstpage: lastpage: Copy Chapter

:alt: Gymnasium Logo

An API standard for reinforcement learning with a diverse collection of reference environments

:alt: Lunar Lander
   :width: 500

Gymnasium is a maintained fork of OpenAI’s Gym library. The Gymnasium interface is simple, pythonic, and capable of representing general RL problems, and has a migration guide for old Gym environments:

import gymnasium as gym

# Initialise the environment
env = gym.make("LunarLander-v3", render_mode="human")

# Reset the environment to generate the first observation
observation, info = env.reset(seed=42)
for _ in range(1000):
    # this is where you would insert your policy
    action = env.action_space.sample()

    # step (transition) through the environment with the action
    # receiving the next observation, reward and if the episode has terminated or truncated
    observation, reward, terminated, truncated, info = env.step(action)

    # If the episode has ended then we can reset to start a new episode
    if terminated or truncated:
        observation, info = env.reset()

env.close()

:hidden:
:caption: Introduction

introduction/basic_usage
introduction/train_agent
introduction/create_custom_env
introduction/record_agent
introduction/speed_up_env
introduction/migration_guide

:hidden:
:caption: API

api/env
api/registry
api/spaces
api/wrappers
api/vector
api/utils
api/functional

:hidden:
:caption: Environments

environments/classic_control
environments/box2d
environments/toy_text
environments/mujoco
environments/atari
environments/third_party_environments

:hidden:
:glob:
:caption: Tutorials

tutorials/**/index
tutorials/third-party-tutorials

:hidden:
:caption: Development

Github <https://github.com/Farama-Foundation/Gymnasium>
Paper <https://arxiv.org/abs/2407.17032>
gymnasium_release_notes/index
gym_release_notes/index
Contribute to the Docs <https://github.com/Farama-Foundation/Gymnasium/blob/main/docs/README.md>

File: docs/README.md Copy Chapter

Gymnasium-docs

This folder contains the documentation for Gymnasium.

Instructions for modifying environment pages Copy Chapter

Editing an environment page Copy Section

Fork Gymnasium and edit the docstring in the environment's Python file. Then, pip install your Gymnasium fork and run docs/_scripts/gen_mds.py in this repo. This will automatically generate a Markdown documentation file for the environment.

Adding a new environment Copy Section

Ensure the environment is in Gymnasium (or your fork). Ensure that the environment's Python file has a properly formatted markdown docstring. Install using pip install -e . and then run docs/_scripts/gen_mds.py. This will automatically generate a md page for the environment. Then complete the other steps.

Other steps

• Add the corresponding gif into the docs/_static/videos/{ENV_TYPE} folder, where ENV_TYPE is the category of your new environment (e.g. mujoco). Follow snake_case naming convention. Alternatively, run docs/_scripts/gen_gifs.py.
• Edit docs/environments/{ENV_TYPE}/index.md, and add the name of the file corresponding to your new environment to the toctree.

Build the Documentation Copy Chapter

Install the required packages and Gymnasium (or your fork):

pip install gymnasium
pip install -r docs/requirements.txt

To build the documentation once:

cd docs
make dirhtml

To rebuild the documentation automatically every time a change is made:

cd docs
sphinx-autobuild -b dirhtml --watch ../gymnasium --re-ignore "pickle$" . _build

You can then open http://localhost:8000 in your browser to watch a live updated version of the documentation.

Writing Tutorials Copy Chapter

We use Sphinx-Gallery to build the tutorials inside the docs/tutorials directory. Check docs/tutorials/demo.py to see an example of a tutorial and Sphinx-Gallery documentation for more information.

To convert Jupyter Notebooks to the python tutorials you can use this script.

If you want Sphinx-Gallery to execute the tutorial (which adds outputs and plots) then the file name should start with run_. Note that this adds to the build time so make sure the script doesn't take more than a few seconds to execute.