## Terminology
This library facilitates the creation of environments that require **walkers**
to perform a **task** in an **arena**.
- **walkers** refer to detached bodies that can move around in the
environment.
- **arenas** refer to the surroundings in which the walkers and possibly other
objects exist.
- **tasks** refer to the specification of observations and rewards that are
passed from the "environment" to the "agent", along with runtime details
such as initialization and termination logic.
## Installation and requirements
See [the documentation for `dm_control`][installation-and-requirements].
## Quickstart
```python
from dm_control import composer
from dm_control.locomotion.examples import basic_cmu_2019
import numpy as np
# Build an example environment.
env = basic_cmu_2019.cmu_humanoid_run_walls()
# Get the `action_spec` describing the control inputs.
action_spec = env.action_spec()
# Step through the environment for one episode with random actions.
time_step = env.reset()
while not time_step.last():
action = np.random.uniform(action_spec.minimum, action_spec.maximum,
size=action_spec.shape)
time_step = env.step(action)
print("reward = {}, discount = {}, observations = {}.".format(
time_step.reward, time_step.discount, time_step.observation))
```
[`dm_control.viewer`] can also be used to visualize and interact with the
environment, e.g.:
```python
from dm_control import viewer
viewer.launch(environment_loader=basic_cmu_2019.cmu_humanoid_run_walls)
```
## Publications
This library contains environments that were adapted from several research
papers. Relevant references include:
- [Emergence of Locomotion Behaviours in Rich Environments (2017)][heess2017].
- [Learning human behaviors from motion capture by adversarial imitation
(2017)][merel2017].
- [Hierarchical visuomotor control of humanoids (2019)][merel2019a].
- [Neural probabilistic motor primitives for humanoid control (2019)][merel2019b].
- [Deep neuroethology of a virtual rodent (2020)][merel2020].
- [CoMic: Complementary Task Learning & Mimicry for Reusable Skills (2020)][hasenclever2020]
[installation-and-requirements]: ../../README.md#installation-and-requirements
[`dm_control.viewer`]: ../viewer/README.md
[heess2017]: https://arxiv.org/abs/1707.02286
[merel2017]: https://arxiv.org/abs/1707.02201
[merel2019a]: https://arxiv.org/abs/1811.09656
[merel2019b]: https://arxiv.org/abs/1811.11711
[merel2020]: https://openreview.net/pdf?id=SyxrxR4KPS
[hasenclever2020]: http://proceedings.mlr.press/v119/hasenclever20a.html
---
### Dm Control/Locomotion/Soccer/README (dm_control/locomotion/soccer/README.md)
# DeepMind MuJoCo Multi-Agent Soccer Environment.
This submodule contains the components and environment used in the following
works.
* [Emergent Coordination through Competition][boxhead]
([dynamic team play](https://www.youtube.com/watch?v=8nU35D8vAlo),
[defensive team play](https://www.youtube.com/watch?v=-gFQqB8L_mI)).
* [From Motor Control to Team Play in Simulated Humanoid Football][humanoid].
## Quickstart
```python
import numpy as np
from dm_control.locomotion import soccer as dm_soccer
# Instantiates a 2-vs-2 BOXHEAD soccer environment with episodes of 10 seconds
# each. Upon scoring, the environment reset player positions and the episode
# continues. In this example, players can physically block each other and the
# ball is trapped within an invisible box encapsulating the field.
env = dm_soccer.load(team_size=2,
time_limit=10.0,
disable_walker_contacts=False,
enable_field_box=True,
terminate_on_goal=False,
walker_type=dm_soccer.WalkerType.BOXHEAD)
# Retrieves action_specs for all 4 players.
action_specs = env.action_spec()
# Step through the environment for one episode with random actions.
timestep = env.reset()
while not timestep.last():
actions = []
for action_spec in action_specs:
action = np.random.uniform(
action_spec.minimum, action_spec.maximum, size=action_spec.shape)
actions.append(action)
timestep = env.step(actions)
for i in range(len(action_specs)):
print(
"Player {}: reward = {}, discount = {}, observations = {}.".format(
i, timestep.reward[i], timestep.discount, timestep.observation[i]))
```
## Rewards
The environment provides a reward of +1 to each player when their team scores a
goal, -1 when their team concedes a goal, or 0 if neither team scored on the
current timestep.
In addition to the sparse reward returned the environment, the player
observations also contain various environment statistics that may be used to
derive custom per-player shaping rewards. See `environment.observation_spec()`
for the additional statistics available to the agents.
## Episode terminations
If `terminate_on_goal` is set to `True`, episodes will terminate immediately
with a discount factor of 0 when either side scores a goal or if the
`time_limit` elapsed. If neither team scores within this time then the episode
will terminate with a discount factor of `1.0`.
If `terminate_on_goal` is set to `False`, players and ball positions are
randomly initialized if either team scores a goal. Episodes always terminate
after `time_limit` with a discount factor of `1.0`.
## Environment Viewer
To visualize an example environment instance using the `dm_control` interactive
viewer, execute `dm_control/locomotion/soccer/explore.py`.
[boxhead]: http://arxiv.org/abs/1902.07151
[humanoid]: https://arxiv.org/abs/2105.12196
---
### Dm Control/Locomotion/Tasks/Reference Pose/README (dm_control/locomotion/tasks/reference_pose/README.md)
# Reference pose tasks
This directory contains components to define tasks based on reference poses (e.g
motion capture data) as well as a motion capture tracking tasks. The tasks and
associated utils were developed as part of
[CoMic: Complementary Task Learning & Mimicry for Reusable Skills (2020)][hasenclever2020].
The reference data is stored in HDF5 files, which can be loaded using the
`HDF5TrajectoryLoader` class in `dm_control/locomotion/mocap/loader.py`. To
download the data used in the CoMic project, please use
`dm_control/locomotion/mocap/cmu_mocap_data.py`. In the reference pose tasks,
reference trajectories are represented as `Trajectory` objects (see
`dm_control/locomotion/mocap/trajectory.py`). For an example of how to construct
a task, see `dm_control/locomotion/examples/cmu_2020_tracking.py`.
[hasenclever2020]: https://proceedings.icml.cc/static/paper_files/icml/2020/5013-Paper.pdf
---
### Dm Control/Mjcf/README (dm_control/mjcf/README.md)
# PyMJCF
IMPORTANT: If you find yourself stuck while using PyMJCF, check out the various
IMPORTANT boxes on this page and the [Common gotchas](#common-gotchas) section
at the bottom to see if any of them is relevant.
This library provides a Python object model for MuJoCo's XML-based
[MJCF](http://www.mujoco.org/book/modeling.html) physics modeling language. The
goal of the library is to allow users to easily interact with and modify MJCF
models in Python, similarly to what the JavaScript DOM does for HTML.
A key feature of this library is the ability to easily compose multiple separate
MJCF models into a larger one. Disambiguation of duplicated names from different
models, or multiple instances of the same model, is handled automatically.
The following snippet provides a quick example of this library's typical use
case. Here, the `UpperBody` class can simply instantiate two copies of `Arm`,
thus reducing code duplication. The names of bodies, joints, or geoms of each
`Arm` are automatically prefixed by their parent's names, and so no name
collision occurs.
```python
from dm_control import mjcf
class Arm:
def __init__(self, name):
self.mjcf_model = mjcf.RootElement(model=name)
self.upper_arm = self.mjcf_model.worldbody.add('body', name='upper_arm')
self.shoulder = self.upper_arm.add('joint', name='shoulder', type='ball')
self.upper_arm.add('geom', name='upper_arm', type='capsule',
pos=[0, 0, -0.15], size=[0.045, 0.15])
self.forearm = self.upper_arm.add('body', name='forearm', pos=[0, 0, -0.3])
self.elbow = self.forearm.add('joint', name='elbow',
type='hinge', axis=[0, 1, 0])
self.forearm.add('geom', name='forearm', type='capsule',
pos=[0, 0, -0.15], size=[0.045, 0.15])
class UpperBody:
def __init__(self):
self.mjcf_model = mjcf.RootElement()
self.mjcf_model.worldbody.add(
'geom', name='torso', type='box', size=[0.15, 0.045, 0.25])
left_shoulder_site = self.mjcf_model.worldbody.add(
'site', size=[1e-6]*3, pos=[-0.15, 0, 0.25])
right_shoulder_site = self.mjcf_model.worldbody.add(
'site', size=[1e-6]*3, pos=[0.15, 0, 0.25])
self.left_arm = Arm(name='left_arm')
left_shoulder_site.attach(self.left_arm.mjcf_model)
self.right_arm = Arm(name='right_arm')
right_shoulder_site.attach(self.right_arm.mjcf_model)
body = UpperBody()
physics = mjcf.Physics.from_mjcf_model(body.mjcf_model)
```
## Basic operations
### Creating an MJCF model
In PyMJCF, the basic building block of a model is an `mjcf.Element`. This
corresponds to an element in the generated XML. However, user code _cannot_
instantiate a generic `mjcf.Element` object directly.
A valid model always consists of a single root `` element. This is
represented as the special `mjcf.RootElement` type in PyMJCF, which _can_ be
instantiated in user code to create an empty model.
```python
from dm_control import mjcf
mjcf_model = mjcf.RootElement()
print(mjcf_model) # MJCF Element:
```
### Adding new elements
Attributes of the new element can be passed as kwargs:
```python
my_box = mjcf_model.worldbody.add('geom', name='my_box',
type='box', pos=[0, .1, 0])
print(my_box) # MJCF Element:
```
### Parsing an existing XML document
Alternatively, if an existing XML file already exists, PyMJCF can parse it to
create a Python object:
```python
from dm_control import mjcf
# Parse from path
mjcf_model = mjcf.from_path(filename)
# Parse from file
with open(filename) as f:
mjcf_model = mjcf.from_file(f)
# Parse from string
with open(filename) as f:
xml_string = f.read()
mjcf_model = mjcf.from_xml_string(xml_string)
print(type(mjcf_model)) #
```
### Traversing through a model
Consider the following MJCF model:
```xml
```
The child elements and XML attributes of an `Element` object are exposed as
Python attributes. These attributes all have the same names as their XML
counterparts, with one exception: the `class` XML attribute is named `dclass` in
order to avoid a clash with the Python `class` keyword:
```python
my_geom = mjcf_model.worldbody.body['foo'].body['bar'].geom['my_geom']
print(isinstance(mjcf_model, mjcf.Element)) # True
print(my_geom.name) # 'my_geom'
print(my_geom.pos) # np.array([0., 1., 2.], dtype=float)
print(my_geom.class) # SyntaxError
print(my_geom.dclass) # 'brick'
```
Note that attribute values in the object model are **not** affected by defaults:
```python
print(mjcf_model.default.default['brick'].geom.rgba) # [1, 0, 0, 1]
print(my_geom.rgba) # None
```
### Finding elements without traversing
We can also find elements directly without having to traverse through the object
hierarchy:
```python
found_geom = mjcf_model.find('geom', 'my_geom')
print(found_geom == my_geom) # True
```
Find all elements of a given type:
```python
# Note that is also considered a joint
joints = mjcf_model.find_all('joint')
print(len(joints)) # 2
print(joints[0] == mjcf_model.worldbody.body['foo'].freejoint) # True
print(joints[1] == mjcf_model.worldbody.body['foo'].body['bar'].joint[0]) # True
```
Note that the order of elements returned by `find_all` is the same as the order
in which they are declared in the model.
### Modifying XML attributes
Attributes can be modified, added, or removed:
```python
my_geom.pos = [1, 2, 3]
print(my_geom.pos) # np.array([1., 2., 3.], dtype=float)
my_geom.quat = [0, 1, 0, 0]
print(my_geom.quat) # np.array([0., 1., 0., 0.], dtype=float)
del my_geom.quat
print(my_geom.quat) # None
```
Schema violations result in errors:
```python
print(my_geom.poss) # raise AttributeError (no child or attribute called poss)
my_geom.pos = 'invalid' # raise ValueError (assigning string to array)
my_geom.pos = [1, 2, 3, 4, 5, 6] # raise ValueError (array length is too long)
# raise ValueError (mass is a required attribute of )
del mjcf_model.find('body', 'foo').inertial.mass
```
### Uniqueness of identifiers
PyMJCF enforces the uniqueness of "identifier" attributes within a model.
Identifiers consist of the `class` attribute of a ``, and all `name`
attributes. Their uniqueness is only enforced within a particular namespace. For
example, a `` is allowed to have the same name as a ``, whereas
`` and `` actuators cannot have the same name.
```python
mjcf_model.worldbody.add('geom', name='my_geom')
foo = mjcf_model.worldbody.find('body', 'foo')
foo.add('my_geom') # Error, duplicated geom name
foo.add('foo') # OK, a geom can have the same name as a body
mjcf_model.find('geom', 'foo').name = 'my_geom' # Error, duplicated geom name
```
### Reference attributes
Some attributes are references to other elements. For example, the `joint`
attribute of an actuator refers to a `` element in the model.
An `mjcf.Element` can be directly assigned to these reference attributes:
```python
my_hinge = mjcf_model.find('joint', 'my_hinge')
my_actuator = mjcf_model.actuator.add('velocity', joint=my_hinge)
```
This is the recommended way to assign reference attributes, since it guarantees
that the reference is not invalidated if the referenced element is renamed.
Alternatively, a string can also be assigned to reference attributes. In this
case, PyMJCF does **not** attempt to verify that the named element actually
exists in the model.
IMPORTANT: If the element being referenced is in a different model to the
reference attribute (e.g. in an attached model), the reference **must** be
created by directly assigning an `mjcf.Element` object to the attribute rather
than a string. Strings assigned to reference attributes cannot contain '/',
since they are automatically scoped by PyMJCF upon attachment.
## Attaching models
In this section we will refer to an `mjcf.RootElement` simply as a "model".
Models can be _attached_ to other models in order to create compositional
scenes.
```python
arena = mjcf.RootElement()
arena.worldbody.add('geom', name='ground', type='plane', size=[10, 10, 1])
robot = mjcf.from_xml_file('robot.xml')
arena.attach(robot)
```
We refer to `arena` as the _parent model_, and `robot` as the _child model_ (or
the _attached model_).
### Attachment frames
When a model is attached to a site, an empty body is created in the parent
model. This empty body is called an _attachment frame_.
The attachment frame is created as a child of the body that contains the
attachment site, and it has the same position and orientation as the site. When
the XML is generated, the attachment frame's contents shadow the contents of the
attached model's ``. The attachment frame's name in the generated XML
is the child's `fully/qualified/prefix/`. The trailing slash ensures that the
attachment frame's name never collides with a user-defined body.
More concretely, if we have the following parent and child models:
```xml
```
Then the final generated XML will be:
```xml
```
IMPORTANT: The attachment frame is created _transparently_ to the user. In
particular, it is NOT treated as a regular `body` by PyMJCF. Its name in the
generated XML should be considered implementation detail and should NOT be
relied on.
Having said that, it is sometimes necessary to access the attachment frame, for
example to add a joint between the parent and the child model. The easiest way
to do this is to hold a reference to the object returned by a call to `attach`:
```python
attachment_frame = parent_model.attach('child')
attachment_frame.add('freejoint')
```
Alternatively, if a model has already been attached, the `find` function can be
used with the `attachment_frame` namespace in order to retrieve the attachment
frame. The `get_attachment_frame` convenience function in `mjcf.traversal_utils`
can find the child model's attachment frame without needing access to the parent
model.
```python
frame_1 = parent_model.find('attachment_frame', 'child')
# Convenience function: get the attachment frame directly from a child model
frame_2 = mjcf.traversal_utils.get_attachment_frame(child_model)
print(frame_1 == frame_2) # True
```
IMPORTANT: To encourage good modeling practices, the only allowed direct
children of an attachment frame are `` and ``. Other types of
elements should instead add be added to the `` of the attached model.
### Element ownership
IMPORTANT: Elements of child models do **not** appear when traversing through
the parent model.
### Default classes
PyMJCF ensures that default classes of a parent model _never_ affect any of its
child models. This minimises the possibility that two models become subtly
"incompatible", as a model always behaves in the same way regardless of what it
is attached to.
The way that PyMJCF achieves this in practice is to move everything in a model's
global `` context into a default class named `/`. In other words, a
PyMJCF-generated model never has anything in the global default context.
Instead, the generated model always looks like:
```xml
```
IMPORTANT: This transformation is _transparent_ to the user. Within Python, the
above geom rgba setting is accessed as if it were a global default, i.e.
`mjcf_model.default.geom.rgba`. Generally speaking, users should never have to
worry about PyMJCF's internal handling of defaults.
When a model is attached, its `/` default class turns into
`fully/qualified/prefix/`. The trailing slash ensures that this transformation
never conflicts with a user-named default class. More specifically, if we have
the following parent and child models:
```xml
```
Then the final generated XML will be:
```xml
```
### Global options
A model cannot be attached to another model if _any_ of the global options are
different. Global options consist of attributes of ``, `
