## File: README.md

# OpenSpiel: A Framework for Reinforcement Learning in Games

[](https://openspiel.readthedocs.io/en/latest/?badge=latest)

[](https://www.python.org)

OpenSpiel is a collection of environments and algorithms for research in general
reinforcement learning and search/planning in games. OpenSpiel supports n-player
(single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and
sequential, strictly turn-taking and simultaneous-move, perfect and imperfect
information games, as well as traditional multiagent environments such as
(partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel
also includes tools to analyze learning dynamics and other common evaluation
metrics. Games are represented as procedural extensive-form games, with some
natural extensions. The core API and games are implemented in C++ and exposed to
Python. Algorithms and tools are written both in C++ and Python.

To try OpenSpiel in Google Colaboratory, please refer to `open_spiel/colabs` subdirectory or start [here](https://colab.research.google.com/github/deepmind/open_spiel/blob/master/open_spiel/colabs/install_open_spiel.ipynb).



# Index

Please choose among the following options:

*   [Installing OpenSpiel](docs/install.md) (for Linux and MacOS; see separate
    [Windows Installation](docs/windows.md) instructions.)
*   [Introduction to OpenSpiel](docs/intro.md)
*   [API Overview and First Example](docs/concepts.md)
*   [API Reference](docs/api_reference.md)
*   [Overview of Implemented Games](docs/games.md)
*   [Overview of Implemented Algorithms](docs/algorithms.md)
*   [Developer Guide](docs/developer_guide.md)
*   [Using OpenSpiel as a C++ Library](docs/library.md)
*   [Guidelines and Contributing](docs/contributing.md)
*   [Authors](docs/authors.md)

For a longer introduction to the core concepts, formalisms, and terminology,
including an overview of the algorithms and some results, please see
[OpenSpiel: A Framework for Reinforcement Learning in Games](https://arxiv.org/abs/1908.09453).

For an overview of OpenSpiel and example uses of the core API, please check out
our tutorials:

*   [Motivation, Core API, Brief Intro to Replictor Dynamics and Imperfect
    Information Games](https://www.youtube.com/watch?v=8NCPqtPwlFQ) by Marc
    Lanctot.
    [(slides)](http://mlanctot.info/files/OpenSpiel_Tutorial_KU_Leuven_2022.pdf)
    [(colab)](https://colab.research.google.com/github/deepmind/open_spiel/blob/master/open_spiel/colabs/OpenSpielTutorial.ipynb)
*   [Motivation, Core API, Implementing CFR and REINFORCE on Kuhn poker, Leduc
    poker, and Goofspiel](https://www.youtube.com/watch?v=o6JNHoGUXCo) by Edward
    Lockhart.
    [(slides)](http://mlanctot.info/files/open_spiel_tutorial-mar2021-comarl.pdf)
    [(colab)](https://colab.research.google.com/github/deepmind/open_spiel/blob/master/open_spiel/colabs/CFR_and_REINFORCE.ipynb)

If you use OpenSpiel in your research, please cite the paper using the following
BibTeX:

```bibtex
@article{LanctotEtAl2019OpenSpiel,
  title     = {{OpenSpiel}: A Framework for Reinforcement Learning in Games},
  author    = {Marc Lanctot and Edward Lockhart and Jean-Baptiste Lespiau and
               Vinicius Zambaldi and Satyaki Upadhyay and Julien P\'{e}rolat and
               Sriram Srinivasan and Finbarr Timbers and Karl Tuyls and
               Shayegan Omidshafiei and Daniel Hennes and Dustin Morrill and
               Paul Muller and Timo Ewalds and Ryan Faulkner and J\'{a}nos Kram\'{a}r
               and Bart De Vylder and Brennan Saeta and James Bradbury and David Ding
               and Sebastian Borgeaud and Matthew Lai and Julian Schrittwieser and
               Thomas Anthony and Edward Hughes and Ivo Danihelka and Jonah Ryan-Davis},
  year      = {2019},
  eprint    = {1908.09453},
  archivePrefix = {arXiv},
  primaryClass = {cs.LG},
  journal   = {CoRR},
  volume    = {abs/1908.09453},
  url       = {http://arxiv.org/abs/1908.09453},
}
```

## Versioning

We use [Semantic Versioning](https://semver.org/).

---

## File: docs/api_reference/game_action_to_string.md

# OpenSpiel game methods: action_to_string

[Back to Core API reference](../api_reference.md) \


`action_to_string(player: int, action: int)`

Returns a string representation of the specified player's action, independent of
state.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("matrix_pd")
print(game.action_to_string(0, 0))
# Output: Cooperate

# Print first player's second action (1).
game = pyspiel.load_game("tic_tac_toe")
print(game.action_to_string(0, 1))
# Output: x(0, 1)
```

---

## File: docs/api_reference/game_deserialize_game_and_state.md

# OpenSpiel core functions: deserialize_game_and_state

[Back to Core API reference](../api_reference.md) \


`deserialize_game_and_state(game: pyspiel.Game, state: pyspiel.State)`

Returns a (game, state) tuple that is reconstructed from the serialized string
data.

Note: pickle can also be used to serialize / deserialize data, and the pickle
uses the same serialization methods.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("tic_tac_toe")
state = game.new_initial_state()
state.apply_action(4)
state.apply_action(2)
state.apply_action(1)
state.apply_action(5)

serialized_data = pyspiel.serialize_game_and_state(game, state)
print(serialized_data)

game_copy, state_copy = pyspiel.deserialize_game_and_state(serialized_data)
print(state_copy)

# Output:
# # Automatically generated by OpenSpiel SerializeGameAndState
# [Meta]
# Version: 1
#
# [Game]
# tic_tac_toe()
# [State]
# 4
# 2
# 1
# 5
#
#
# .xo
# .xo
# ...
```

---

## File: docs/api_reference/game_deserialize_state.md

# OpenSpiel game methods: deserialize_state

[Back to Core API reference](../api_reference.md) \


`deserialize_state(serialized_data: str)`

Reconstruct a state object from the state's serialized data (from
`state.serialize()`). The game used to reconstruct must be the same as the game
that created the original state.

To serialize a state along with the game, use `pyspiel.serialize_game_and_state`
instead.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("tic_tac_toe")
state = game.new_initial_state()
state.apply_action(4)
state.apply_action(2)
state.apply_action(1)
state.apply_action(5)

state_copy = game.deserialize_state(state.serialize())
print(state_copy)

# Output:
# .xo
# .xo
# ...
```

---

## File: docs/api_reference/game_information_state_tensor_shape_size.md

# OpenSpiel game methods: information_state_tensor_shape and information_state_tensor_size

[Back to Core API reference](../api_reference.md) \


1.  `information_state_tensor_shape()`
2.  `information_state_tensor_size()`

(1) Returns the information state tensor's shape: a list of integers
representing the size of each dimension.

(2) Returns the total number of values used to represent the information state
tensor.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("kuhn_poker")
print(game.information_state_tensor_shape())
print(game.information_state_tensor_size())

# Output:
# [11]
# 11
```

---

## File: docs/api_reference/game_max_chance_outcomes.md

# OpenSpiel game methods: max_chance_outcomes

[Back to Core API reference](../api_reference.md) \


`max_chance_outcomes`

Returns the maximum number of distinct chance outcomes at chance nodes in the
game.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("chess")
print(game.max_chance_outcomes())
# Outputs: 0   (no chance nodes in Chess)

game = pyspiel.load_game("markov_soccer")
print(game.max_chance_outcomes())
# Outputs: 4   (ball starting location, and who gets initiative)

game = pyspiel.load_game("leduc_poker")
print(game.max_chance_outcomes())
# Outputs: 6   (three cards in two suits)
```

---

## File: docs/api_reference/game_max_game_length.md

# OpenSpiel game methods: max_game_length

[Back to Core API reference](../api_reference.md) \


`max_game_length()`

The maximum length of any one game (in terms of number of decision nodes 
visited in the game tree).

For a simultaneous action game, this is the maximum number of joint decisions.
In a turn-based game, this is the maximum number of individual decisions summed
over all players. Outcomes of chance nodes are not included in this length.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("tic_tac_toe")
print(game.max_game_length())    # Output: 9

# Normal-form games always have one
game = pyspiel.load_game("blotto")
print(game.max_game_length())    # Output: 1

# The maximum is arbitrarily defined (and/or customizable) is some games.
game = pyspiel.load_game("coop_box_pushing")
print(game.max_game_length())    # Output: 100
game = pyspiel.load_game("coop_box_pushing(horizon=250)")
print(game.max_game_length())    # Output: 250
```

---

## File: docs/api_reference/game_max_min_utility.md

# OpenSpiel game methods: max_utility and min_utility

[Back to Core API reference](../api_reference.md) \


`max_utility()` \
`min_utility()`

Returns the maximum and minimum achievable utility (return in any given episode)
in the game.

## Examples:

```python
import pyspiel

# Win/loss game
game = pyspiel.load_game("tic_tac_toe")
print(game.min_utility())    # Output: -1
print(game.max_utility())    # Output: 1

# Win/los/draw game (draw counts as 0).
game = pyspiel.load_game("chess")
print(game.min_utility())    # Output: -1
print(game.max_utility())    # Output: 1

# Money game.
game = pyspiel.load_game("leduc_poked")
print (game.num_distinct_actions())
print(game.min_utility())    # Output: -13
print(game.max_utility())    # Output: 13
```

---

## File: docs/api_reference/game_new_initial_state.md

# OpenSpiel game methods: new_initial_state

[Back to Core API reference](../api_reference.md) \


`new_initial_state()`

Returns a new state object representing the first state of the game. Note, in
particular, this might be a chance node (where the current player is chance) in
games with chance events.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("hex")
state = game.new_initial_state()
print(state)

# Output:
# . . . . . . . . . . .
#  . . . . . . . . . . .
#   . . . . . . . . . . .
#    . . . . . . . . . . .
#     . . . . . . . . . . .
#      . . . . . . . . . . .
#       . . . . . . . . . . .
#        . . . . . . . . . . .
#         . . . . . . . . . . .
#          . . . . . . . . . . .
#           . . . . . . . . . . .
```

---

## File: docs/api_reference/game_num_distinct_actions.md

# OpenSpiel game methods: num_distinct_actions

[Back to Core API reference](../api_reference.md) \


`num_distinct_actions()`

Returns the number of state-independent actions in the game. Valid actions in a
game will always be between 0 and `num_distinct_actions() - 1`. This number can
be thought of as the fixed width of a policy head or Q-network. Legal actions
are always a subset of { 0, 1, ... , `num_distinct_actions() - 1` }.

## Examples:

```python
import pyspiel

game = pyspiel.load_game("tic_tac_toe")
print(game.num_distinct_actions())    # Output: 9

game = pyspiel.load_game("go")
print (game.num_distinct_actions())   # Output: 362

game = pyspiel.load_game("chess")
print (game.num_distinct_actions())   # Output: 4672

game = pyspiel.load_game("leduc_poker")
print (game.num_distinct_actions())   # Output: 3
```