dm_alchemy: DeepMind Alchemy environment

Overview | Requirements | Installation | Usage | Documentation | Tutorial | Paper | Blog post

The DeepMind Alchemy environment is a meta-reinforcement learning benchmark that presents tasks sampled from a task distribution with deep underlying structure. It was created to test for the ability of agents to reason and plan via latent state inference, as well as useful exploration and experimentation. It is Unity-based.

Overview

This environment is provided through pre-packaged Docker containers.

This package consists of support code to run these Docker containers. You interact with the task environment via a dm_env Python interface.

Please see the documentation for more detailed information on the available tasks, actions and observations.

Requirements

dm_alchemy requires Docker, Python 3.6.1 or later and a x86-64 CPU with SSE4.2 support. We do not attempt to maintain a working version for Python 2.

Alchemy is intended to be run on Linux and is not officially supported on Mac and Windows. However, it can in principle be run on any platform (though installation may be more of a headache). In particular, on Windows, you will need to install and run Alchemy with WSL.

Note: We recommend using Python virtual environment to mitigate conflicts with your system's Python environment.

Download and install Docker:

• For Linux, install Docker-CE. Ensure that you can run Docker as a non-root user.
• Install Docker Desktop for OSX or Windows.

Ensure that docker is working correctly by running docker run -d gcr.io/deepmind-environments/alchemy:v1.0.0.

Installation

You can install dm_alchemy by cloning a local copy of our GitHub repository:

$ git clone https://github.com/deepmind/dm_alchemy.git
$ pip install wheel
$ pip install --upgrade setuptools
$ pip install ./dm_alchemy

To also install the dependencies for the examples/, install with:

$ pip install ./dm_alchemy[examples]

Usage

Once dm_alchemy is installed, to instantiate a dm_env instance run the following:

import dm_alchemy

LEVEL_NAME = ('alchemy/perceptual_mapping_'
              'randomized_with_rotation_and_random_bottleneck')
settings = dm_alchemy.EnvironmentSettings(seed=123, level_name=LEVEL_NAME)
env = dm_alchemy.load_from_docker(settings)

For more details see the introductory colab.

Citing Alchemy

If you use Alchemy in your work, please cite the accompanying technical report:

@article{wang2021alchemy,
    title={Alchemy: A structured task distribution for meta-reinforcement learning},
    author={Jane Wang and Michael King and Nicolas Porcel and Zeb Kurth-Nelson
        and Tina Zhu and Charlie Deck and Peter Choy and Mary Cassin and
        Malcolm Reynolds and Francis Song and Gavin Buttimore and David Reichert
        and Neil Rabinowitz and Loic Matthey and Demis Hassabis and Alex Lerchner
        and Matthew Botvinick},
    year={2021},
    journal={arXiv preprint arXiv:2102.02926},
    url={https://arxiv.org/abs/2102.02926},
}

Notice

This is not an officially supported Google product.