There was an issue with environment names, that have a slash in their name (see https://github.com/DLR-RM/rl-baselines3-zoo/pull/257). Also the naming scheme for models and repository IDs is just based on convention.

This PR implements normalization for environment names (replacing slashes with dashes) and encodes the naming scheme for models and repository IDs in little helper classes. The idea is, that those helper classes can be used by downstream libraries to comply with the naming scheme (such as the rl baselines zoo). If we ever need to change the naming scheme or other cases in which the environment name needs to be normalized come up, then we can implement them here and the downstream libraries immediately profit from that.

I also added a simple smoke test for pulling a model from the hub.

I am going through the notebook of Unit 1 of the deep RL course. However, I cannot run the package_to_hub function, which gives the following error:

HTTPError                                 Traceback (most recent call last)

[<ipython-input-26-97f48e41190b>](https://localhost:8080/#) in <module>
     25                eval_env=eval_env,
     26                repo_id="LorenzoPacchiardi/ppo-LunarLander-v2",
---> 27                commit_message="Upload PPO LunarLander-v2 trained agent (50 steps)")

6 frames

[/usr/local/lib/python3.7/dist-packages/requests/models.py](https://localhost:8080/#) in raise_for_status(self)
    939 
    940         if http_error_msg:
--> 941             raise HTTPError(http_error_msg, response=self)
    942 
    943     def close(self):

HTTPError: 400 Client Error: Bad Request for url: https://huggingface.co/api/models/LorenzoPacchiardi/ppo-LunarLander-v2/commit/main (Request ID: fhQtAuS_qa8bj_c6AI0v5)

I get a similar error with push_to_hub

I logged in to huggingface correctly with the token, and the load_from_hub function works fine.

Currently package_to_hub works only for OpenGL capable computers. It doesn't support any other option for generating video and it doesn't allow to upload model without a video. All new Mac OSes don't have OpenGL support any more.

Hi! I'm running this notebook https://github.com/huggingface/deep-rl-class/blob/main/unit1/unit1.ipynb from your DRL series. Installation of some libraries is causing some issues. For huggingface_sb3, it is:

Collecting huggingface_sb3
  Using cached huggingface_sb3-2.0.0-py3-none-any.whl (7.4 kB)
Requirement already satisfied: wasabi in ./rl/lib/python3.8/site-packages (from huggingface_sb3) (0.9.1)
Collecting cloudpickle==1.6
  Using cached cloudpickle-1.6.0-py3-none-any.whl (23 kB)
Collecting pyyaml==6.0
  Using cached PyYAML-6.0-cp38-cp38-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (701 kB)
Collecting huggingface-hub
  Using cached huggingface_hub-0.6.0-py3-none-any.whl (84 kB)
Collecting pickle5
  Using cached pickle5-0.0.11.tar.gz (132 kB)
  Preparing metadata (setup.py) ... done
Collecting typing-extensions>=3.7.4.3
  Using cached typing_extensions-4.2.0-py3-none-any.whl (24 kB)
Requirement already satisfied: packaging>=20.9 in ./rl/lib/python3.8/site-packages (from huggingface-hub->huggingface_sb3) (21.3)
Collecting filelock
  Using cached filelock-3.7.0-py3-none-any.whl (10 kB)
Collecting tqdm
  Using cached tqdm-4.64.0-py2.py3-none-any.whl (78 kB)
Collecting requests
  Using cached requests-2.27.1-py2.py3-none-any.whl (63 kB)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in ./rl/lib/python3.8/site-packages (from packaging>=20.9->huggingface-hub->huggingface_sb3) (3.0.9)
Collecting urllib3<1.27,>=1.21.1
  Using cached urllib3-1.26.9-py2.py3-none-any.whl (138 kB)
Collecting certifi>=2017.4.17
  Using cached certifi-2021.10.8-py2.py3-none-any.whl (149 kB)
Collecting charset-normalizer~=2.0.0
  Using cached charset_normalizer-2.0.12-py3-none-any.whl (39 kB)
Collecting idna<4,>=2.5
  Using cached idna-3.3-py3-none-any.whl (61 kB)
Using legacy 'setup.py install' for pickle5, since package 'wheel' is not installed.
Installing collected packages: pickle5, certifi, urllib3, typing-extensions, tqdm, pyyaml, idna, filelock, cloudpickle, charset-normalizer, requests, huggingface-hub, huggingface_sb3
  Running setup.py install for pickle5 ... error
  error: subprocess-exited-with-error
  
  × Running setup.py install for pickle5 did not run successfully.
  │ exit code: 1
  ╰─> [27 lines of output]
      running install
      /media/master/support/pip_envs/rl/lib/python3.8/site-packages/setuptools/command/install.py:34: SetuptoolsDeprecationWarning: setup.py install is deprecated. Use build and pip and other standards-based tools.
        warnings.warn(
      running build
      running build_py
      creating build
      creating build/lib.linux-x86_64-cpython-38
      creating build/lib.linux-x86_64-cpython-38/pickle5
      copying pickle5/__init__.py -> build/lib.linux-x86_64-cpython-38/pickle5
      copying pickle5/pickle.py -> build/lib.linux-x86_64-cpython-38/pickle5
      copying pickle5/pickletools.py -> build/lib.linux-x86_64-cpython-38/pickle5
      creating build/lib.linux-x86_64-cpython-38/pickle5/test
      copying pickle5/test/pickletester.py -> build/lib.linux-x86_64-cpython-38/pickle5/test
      copying pickle5/test/test_picklebuffer.py -> build/lib.linux-x86_64-cpython-38/pickle5/test
      copying pickle5/test/__init__.py -> build/lib.linux-x86_64-cpython-38/pickle5/test
      copying pickle5/test/test_pickle.py -> build/lib.linux-x86_64-cpython-38/pickle5/test
      running build_ext
      building 'pickle5._pickle' extension
      creating build/temp.linux-x86_64-cpython-38
      creating build/temp.linux-x86_64-cpython-38/pickle5
      x86_64-linux-gnu-gcc -Wno-unused-result -Wsign-compare -DNDEBUG -g -fwrapv -O2 -Wall -g -fstack-protector-strong -Wformat -Werror=format-security -g -fwrapv -O2 -fPIC -I/media/master/support/pip_envs/rl/include -I/usr/include/python3.8 -c pickle5/_pickle.c -o build/temp.linux-x86_64-cpython-38/pickle5/_pickle.o -std=c99
      In file included from pickle5/_pickle.c:2:
      pickle5/compat.h:1:10: fatal error: Python.h: No such file or directory
          1 | #include "Python.h"
            |          ^~~~~~~~~~
      compilation terminated.
      error: command '/usr/bin/x86_64-linux-gnu-gcc' failed with exit code 1
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: legacy-install-failure

× Encountered error while trying to install package.
╰─> pickle5

note: This is an issue with the package mentioned above, not pip.

I tried installing it with python 3.9 and 3.8 on Ubuntu 22.04 OS.

Are there any additional requirements to use your library?

This code snippet

env = VecVideoRecorder(
        eval_env,
        "./",  # Temporary video folder
        record_video_trigger=lambda x: x == 0,
        video_length=video_length,
        name_prefix="",
    )

generated a video file wherever the user is. In the temporary video folder, you can use tempfile.TemporaryDirectory() to automatically create a directory that will be deleted afterwards

I think you want to rebase, as we tend to do this in all the mixins in huggingface_hub. I think not having this is what caused this issue: https://github.com/huggingface/deep-rl-class/issues/20.

• add missing type hints
• fix push_to_hub (bug detected by pytype checker)
• cleanup
• add support for VecNormalize

closes #6

Demo (and training/loading code): https://huggingface.co/araffin/a2c-Pendulum-v1

The behavior with this PR is that once you push a Git tag with v* (usually v1.0.8 for example), which should ideally point to the commit that updates this line https://github.com/huggingface/huggingface_sb3/blob/main/setup.py#L10 (you can push the tag after the commit), it will automatically make a pypi release.

The only requirement is adding your secret (PYPI_TOKEN_DIST) to the repo settings

It's very easy to add TensorBoard logging with SB3, but pushing the files right now needs to be done manually. As an alternative, we could add a param to package_to_hub to pass the logs.

Related: https://github.com/huggingface/deep-rl-class/pull/19

At the moment, if generate_replay fails, the whole package_to_hub method fails. Ideally it would still push the metrics and other related information even if no video is generated

👋 so here's the SB3 v2.0:

With our new version we can use package_to_hub method that:

1. Save the model
2. Evaluate the model and generate a results.json
3. Generate a model card
4. Record a replay video of the agent
5. Push everything to the hub Here's an example : https://huggingface.co/ThomasSimonini/TEST2-Colab-ppo-LunarLander-v2 (very small training so the agent is bad)

If you want to try some examples directly on colab I've made a small test colab: https://colab.research.google.com/drive/1FhZ1w7smqPo8GQcW5qb2HmkggZVuok57?usp=sharing

The PyPi update is also automated thanks to @osanseviero

A big thanks to Omar who made a lot of tests with the library

I need a little bit of feedback for the documentation, I think it's not very clear.

• I explain the 2 cases: downstream and upstream
• In case 3-4 I explain how to use xvfb if you use colab or vm (because you don't have a screen to render so you can't generate a video without xvfb).

WDYT? Thanks