File: README.md
OpenSpiel: A Framework for Reinforcement Learning in Games
OpenSpiel is a collection of environments and algorithms for research in general
reinforcement learning and search/planning in games. OpenSpiel supports n-player
(single- and multi- agent) zero-sum, cooperative and general-sum, one-shot and
sequential, strictly turn-taking and simultaneous-move, perfect and imperfect
information games, as well as traditional multiagent environments such as
(partially- and fully- observable) grid worlds and social dilemmas. OpenSpiel
also includes tools to analyze learning dynamics and other common evaluation
metrics. Games are represented as procedural extensive-form games, with some
natural extensions. The core API and games are implemented in C++ and exposed to
Python. Algorithms and tools are written both in C++ and Python.
To try OpenSpiel in Google Colaboratory, please refer to open_spiel/colabs subdirectory or start here.
Index
Please choose among the following options:
- Installing OpenSpiel (for Linux and MacOS; see separate
Windows Installation instructions.) - Introduction to OpenSpiel
- API Overview and First Example
- API Reference
- Overview of Implemented Games
- Overview of Implemented Algorithms
- Developer Guide
- Using OpenSpiel as a C++ Library
- Guidelines and Contributing
- Authors
For a longer introduction to the core concepts, formalisms, and terminology,
including an overview of the algorithms and some results, please see
OpenSpiel: A Framework for Reinforcement Learning in Games.
For an overview of OpenSpiel and example uses of the core API, please check out
our tutorials:
- Motivation, Core API, Brief Intro to Replictor Dynamics and Imperfect
Information Games by Marc
Lanctot.
(slides)
(colab) - Motivation, Core API, Implementing CFR and REINFORCE on Kuhn poker, Leduc
poker, and Goofspiel by Edward
Lockhart.
(slides)
(colab)
If you use OpenSpiel in your research, please cite the paper using the following
BibTeX:
@article{LanctotEtAl2019OpenSpiel,
title = {{OpenSpiel}: A Framework for Reinforcement Learning in Games},
author = {Marc Lanctot and Edward Lockhart and Jean-Baptiste Lespiau and
Vinicius Zambaldi and Satyaki Upadhyay and Julien P\'{e}rolat and
Sriram Srinivasan and Finbarr Timbers and Karl Tuyls and
Shayegan Omidshafiei and Daniel Hennes and Dustin Morrill and
Paul Muller and Timo Ewalds and Ryan Faulkner and J\'{a}nos Kram\'{a}r
and Bart De Vylder and Brennan Saeta and James Bradbury and David Ding
and Sebastian Borgeaud and Matthew Lai and Julian Schrittwieser and
Thomas Anthony and Edward Hughes and Ivo Danihelka and Jonah Ryan-Davis},
year = {2019},
eprint = {1908.09453},
archivePrefix = {arXiv},
primaryClass = {cs.LG},
journal = {CoRR},
volume = {abs/1908.09453},
url = {http://arxiv.org/abs/1908.09453},
}Versioning
We use Semantic Versioning.
File: docs/api_reference/game_action_to_string.md
OpenSpiel game methods: action_to_string
action_to_string(player: int, action: int)
Returns a string representation of the specified player's action, independent of
state.
Examples:
import pyspiel
game = pyspiel.load_game("matrix_pd")
print(game.action_to_string(0, 0))
# Output: Cooperate
# Print first player's second action (1).
game = pyspiel.load_game("tic_tac_toe")
print(game.action_to_string(0, 1))
# Output: x(0, 1)File: docs/api_reference/game_deserialize_game_and_state.md
OpenSpiel core functions: deserialize_game_and_state
deserialize_game_and_state(game: pyspiel.Game, state: pyspiel.State)
Returns a (game, state) tuple that is reconstructed from the serialized string
data.
Note: pickle can also be used to serialize / deserialize data, and the pickle
uses the same serialization methods.
Examples:
import pyspiel
game = pyspiel.load_game("tic_tac_toe")
state = game.new_initial_state()
state.apply_action(4)
state.apply_action(2)
state.apply_action(1)
state.apply_action(5)
serialized_data = pyspiel.serialize_game_and_state(game, state)
print(serialized_data)
game_copy, state_copy = pyspiel.deserialize_game_and_state(serialized_data)
print(state_copy)
# Output:
# # Automatically generated by OpenSpiel SerializeGameAndState
# [Meta]
# Version: 1
#
# [Game]
# tic_tac_toe()
# [State]
# 4
# 2
# 1
# 5
#
#
# .xo
# .xo
# ...File: docs/api_reference/game_deserialize_state.md
OpenSpiel game methods: deserialize_state
deserialize_state(serialized_data: str)
Reconstruct a state object from the state's serialized data (fromstate.serialize()). The game used to reconstruct must be the same as the game
that created the original state.
To serialize a state along with the game, use pyspiel.serialize_game_and_state
instead.
Examples:
import pyspiel
game = pyspiel.load_game("tic_tac_toe")
state = game.new_initial_state()
state.apply_action(4)
state.apply_action(2)
state.apply_action(1)
state.apply_action(5)
state_copy = game.deserialize_state(state.serialize())
print(state_copy)
# Output:
# .xo
# .xo
# ...File: docs/api_reference/game_information_state_tensor_shape_size.md
OpenSpiel game methods: information_state_tensor_shape and information_state_tensor_size
information_state_tensor_shape()information_state_tensor_size()
(1) Returns the information state tensor's shape: a list of integers
representing the size of each dimension.
(2) Returns the total number of values used to represent the information state
tensor.
Examples:
import pyspiel
game = pyspiel.load_game("kuhn_poker")
print(game.information_state_tensor_shape())
print(game.information_state_tensor_size())
# Output:
# [11]
# 11File: docs/api_reference/game_max_chance_outcomes.md
OpenSpiel game methods: max_chance_outcomes
max_chance_outcomes
Returns the maximum number of distinct chance outcomes at chance nodes in the
game.
Examples:
import pyspiel
game = pyspiel.load_game("chess")
print(game.max_chance_outcomes())
# Outputs: 0 (no chance nodes in Chess)
game = pyspiel.load_game("markov_soccer")
print(game.max_chance_outcomes())
# Outputs: 4 (ball starting location, and who gets initiative)
game = pyspiel.load_game("leduc_poker")
print(game.max_chance_outcomes())
# Outputs: 6 (three cards in two suits)File: docs/api_reference/game_max_game_length.md
OpenSpiel game methods: max_game_length
max_game_length()
The maximum length of any one game (in terms of number of decision nodes
visited in the game tree).
For a simultaneous action game, this is the maximum number of joint decisions.
In a turn-based game, this is the maximum number of individual decisions summed
over all players. Outcomes of chance nodes are not included in this length.
Examples:
import pyspiel
game = pyspiel.load_game("tic_tac_toe")
print(game.max_game_length()) # Output: 9
# Normal-form games always have one
game = pyspiel.load_game("blotto")
print(game.max_game_length()) # Output: 1
# The maximum is arbitrarily defined (and/or customizable) is some games.
game = pyspiel.load_game("coop_box_pushing")
print(game.max_game_length()) # Output: 100
game = pyspiel.load_game("coop_box_pushing(horizon=250)")
print(game.max_game_length()) # Output: 250File: docs/api_reference/game_max_min_utility.md
OpenSpiel game methods: max_utility and min_utility
max_utility() min_utility()
Returns the maximum and minimum achievable utility (return in any given episode)
in the game.
Examples:
import pyspiel
# Win/loss game
game = pyspiel.load_game("tic_tac_toe")
print(game.min_utility()) # Output: -1
print(game.max_utility()) # Output: 1
# Win/los/draw game (draw counts as 0).
game = pyspiel.load_game("chess")
print(game.min_utility()) # Output: -1
print(game.max_utility()) # Output: 1
# Money game.
game = pyspiel.load_game("leduc_poked")
print (game.num_distinct_actions())
print(game.min_utility()) # Output: -13
print(game.max_utility()) # Output: 13File: docs/api_reference/game_new_initial_state.md
OpenSpiel game methods: new_initial_state
new_initial_state()
Returns a new state object representing the first state of the game. Note, in
particular, this might be a chance node (where the current player is chance) in
games with chance events.
Examples:
import pyspiel
game = pyspiel.load_game("hex")
state = game.new_initial_state()
print(state)
# Output:
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .
# . . . . . . . . . . .File: docs/api_reference/game_num_distinct_actions.md
OpenSpiel game methods: num_distinct_actions
num_distinct_actions()
Returns the number of state-independent actions in the game. Valid actions in a
game will always be between 0 and num_distinct_actions() - 1. This number can
be thought of as the fixed width of a policy head or Q-network. Legal actions
are always a subset of { 0, 1, ... , num_distinct_actions() - 1 }.
Examples:
import pyspiel
game = pyspiel.load_game("tic_tac_toe")
print(game.num_distinct_actions()) # Output: 9
game = pyspiel.load_game("go")
print (game.num_distinct_actions()) # Output: 362
game = pyspiel.load_game("chess")
print (game.num_distinct_actions()) # Output: 4672
game = pyspiel.load_game("leduc_poker")
print (game.num_distinct_actions()) # Output: 3