Tag Archives: MLAgents

Unity ML-Agents โ€” Part I: Set-up

This blog post will be the first of several as I get started with ML-Agents, a Unity toolkit for adding deep learning technology to your game dev projects. I am following Code Monkey’s YouTube tutorial, and these posts will roughly follow that video, with added notes for anything extra I ran into along the way.

I am working on a ThinkPad running Ubuntu 20.04.3, relying on an integrated GPU (Mesa Intelยฎ UHD Graphics 620 (WHL GT2)). I am currently using Unity Hub 2.4.5, Unity Editor 2020.3.18f1, Python 3.8.10, pip as my Python package manager, and venv for my virtual environment.

Resources for setting up ML-Agents

1. Create a Unity project

If you don’t already have one started, create a new Unity project.

2. Create a Python virtual environment

Next, create a Python virtual environment in your Unity project directory.

Virtual environments allow you to create an isolated environment for each of your Python projects, so that each project can have its own dependencies independent from every other project. If one project requires NumPY 1.20.1 and another requires NumPy 1.19.5, creating a virtual environment for each project is an easy way to make sure everyone’s NumPy needs are happily met.

To get venv to work on Ubuntu, I first had to install it using apt:

$ sudo apt install python3.8-venv

To create a new venv virtual environment, use the command venv :

$ python3 -m venv MyVenv

To activate/enter your venv, use the command activate. On my computer this looks like:

$ source MyVenv/bin/activate

Once you are inside your active virtual environment, you can deactivate it /exit it using the command deactivate:

$ deactivate

3. Install packages (in venv)

Update pip

Upgrade pip inside the venv virtual environment you created:

$ python3 -m pip install --upgrade pip

Upgrade setuptools:

$ python3 -m pip install --upgrade setuptools

Update CUDA if you have it

If you have a Nvidia GPU and will be using CUDA, you can install/update CUDA. Note that you may need an older version of CUDA to work with your PyTorch /ML-Agents /Unity set-up.

The Code Monkey ML-Agents YouTube tutorial provides some info on setting up with CUDA, and here are a couple links for more info:

Install PyTorch

The PyTorch website provides a helpful tool to select your set-up preferences and determine the command you’ll need to install PyTorch. I selected the stable 1.9.1 version* of PyTorch, Linux OS, Pip package manager, Python language, and CPU compute platform, and it provided me with this command:

$ python3 -m pip install torch==1.9.1+cpu torchvision==0.10.1+cpu torchaudio==0.9.1 -f https://download.pytorch.org/whl/torch_stable.html

*It turned out I actually needed a lower version of PyTorch to work with ML-Agents, which I discovered once I tried to install ML-Agents (below). Here’s the command that ended up working for me, to install PyTorch 1.8.2:

$ python -m pip install torch==1.8.2+cpu torchvision==0.9.2+cpu torchaudio==0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html

Install ML-Agents

Install ML-Agents in your Python virtual environment:

$ python3 -m pip install mlagents

At this point, with PyTorch 1.9.1, I received this error message during installation:

    Attempting uninstall: torch
    Found existing installation: torch 1.9.1+cpu
    Uninstalling torch-1.9.1+cpu:
    Successfully uninstalled torch-1.9.1+cpu
    ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behavior is the source of the following dependency conflicts.
    torchvision 0.10.1+cpu requires torch==1.9.1, but you have torch 1.8.1 which is incompatible.
    torchaudio 0.9.1 requires torch==1.9.1, but you have torch 1.8.1 which is incompatible.
    Successfully installed absl-py-0.14.1 attrs-21.2.0 cachetools-4.2.4 cattrs-1.5.0 certifi-2021.5.30 charset-normalizer-2.0.6 cloudpickle-2.0.0 google-auth-1.35.0 google-auth-oauthlib-0.4.6 grpcio-1.41.0 h5py-3.4.0 idna-3.2 markdown-3.3.4 mlagents-0.27.0 mlagents-envs-0.27.0 oauthlib-3.1.1 protobuf-3.18.1 pyasn1-0.4.8 pyasn1-modules-0.2.8 pyyaml-5.4.1 requests-2.26.0 requests-oauthlib-1.3.0 rsa-4.7.2 six-1.16.0 tensorboard-2.6.0 tensorboard-data-server-0.6.1 tensorboard-plugin-wit-1.8.0 torch-1.8.1 urllib3-1.26.7 werkzeug-2.0.2 wheel-0.37.0

Ok, so it uninstalled my brand new PyTorch 1.9.1, replaced it with PyTorch 1.8.1, and then complained that it needs 1.9.1… ๐Ÿ˜›

Reinstalling PyTorch using 1.8.2 seemed to resolve the dependency issues for me:

$ python -m pip install torch==1.8.2+cpu torchvision==0.9.2+cpu torchaudio==0.8.2 -f https://download.pytorch.org/whl/lts/1.8/torch_lts.html

Code Monkey also recommends installing ML-Agents using the 2020-resolver feature if you run into issues with package version requirements:

$ python3 -m pip install mlagents --use-feature=2020-resolver

Verify that it worked

Now! You can verify that you have Ml-Agents installed in your virtual environment by testing out the mlagents-learn command to read to help doc text:

mlagents-learn --help

That should hopefully now display the help text without any warnings or errors ๐Ÿ˜€

4. Install ML-Agents, but now in Unity

Ok. Now we can go into the Unity Editor and install the packages we need there.

To install the ML-Agents package in your Unity project, open up your project in the Unity Editor. For me this means first opening up Unity Hub, and then opening my project in the Editor from there.

Now that you’re in the Unity Editor, in your Unity project, go to “Window” and click on “Package Manager”. This is where you can manage, add, and remove Packages for your project. Selecting “Packages: In Project” will show the packages in your project, and selecting “Packages: Unity Registry” will show you all available packages.

To add the ML-Agents package to your project, select “Packages: Unity Registry”, find “ML Agent” and install the version you want. By default the packages that appear in the Registry are the latest stable versions (“Verified Packages”), but the Unity ML-Agents installation guide recommends enabling “Preview Packages” in order to use the latest release, as long as you’re not in a production stage. To enable preview packages, from the same Package Manager window click the gear icon to open Advanced Project Settings and check the box that says “Enable Preview Packages.”

To verify that you have ML-Agents installed correctly in your Unity project, create a new empty game object and see if you can add an ML-Agents Component to it. In the object’s Properties (either in the Inspector or right-click on “Properties”), there should now be an ML Agents option when you click “Add Component”. ๐Ÿ™‚

5. Install ML-Agents Extension, in Unity

At this point I needed to install the ML-Agents Extension package separately to continue following along the Code Monkey tutorial, since it is not included with the regular ML-Agents Unity package.

ML-Agents Extension installation resources:

To add the ML-Agents Extension package, you can go through either the Unity Package Manager, or you can add it directly to the manifest.json file in the Packages directory in your Unity project.

For either method you can use either a Git URL or a local path. Here’s steps for each of those 4 options:

Adding a package โ€” Unity Package Manager using Git URL

To add a new package from the Unity Package Manager, click the “+” button, which will give you options on how to add a new package. To add a package from a GitHub URL, click “Add package from git URL”, and paste in the URL to the repository you need.

The Git URL you will need will look like this, with the “git+” at the front, and the “#release_XX” at the end:

"git+https://github.com/Unity-Technologies/ml-agents.git?path=com.unity.ml-agents.extensions#release_18"

Adding a package โ€” Unity Package Manager using local package

You can also add a package from the Unity Package Manager using local source files. To locally install the ML-Agents Extensions package, first clone the ml-agents repository (https://github.com/Unity-Technologies/ml-agents) to somewhere that makes sense to you on your computer.

To make sure you have the correct version you need, check the version list in the README (https://github.com/Unity-Technologies/ml-agents#releases–documentation). You can either clone the specific branch you need:

$ git clone --branch release_18 https://github.com/Unity-Technologies/ml-agents.git

Or, you can clone the main branch, and later checkout/switch to the branch you need:

$ git clone https://github.com/Unity-Technologies/ml-agents.git
$ cd ml-agents
$ git checkout 

Once you have your package repo set up locally, select “Add package from disk” in the Unity Package Manager. For the ML-Agents Extension package, you will navigate to the com.unity.ml-agents.extensions folder. From there select the package.json file.

Adding a package โ€” Add Git URL to package manifest.json

You can also add a new package by directly editing your Project Packages manifest.json file. To add the ML-Agents Extension package using a Git URL, you will need to insert a line that looks like this:

"com.unity.ml-agents.extensions": "git+https://github.com/Unity-Technologies/ml-agents.git?path=com.unity.ml-agents.extensions#release_18"

Adding a package โ€” Add local path to package manifest.json

If you are using local source files to add your ML-Agents Extensions package, the line you add to your Project Packages manifest.json fill will instead need to include the path to your local com.unity.ml-agents.extensions directory:

"com.unity.ml-agents.extensions": "file:../MyPackages/MLAgents/ml-agents/com.unity.ml-agents.extensions"

To be continued…

Woo hoo! We’ve now finally got ML-Agents set up and we’re ready to training some little AI gamers.

Code Monkey’s ML-Agents video tutorial is great, and I would highly recommend checking it out. I hope this post is a helpful supplement to that tutorial for anyone else working in Linux, or anyone else who likes having written steps to refer to as well.

I’ll try to post the next segments soon, which will go over a basic intro to actually creating and using ML-Agents in your Unity game.

       ___-___
      |  o o  |
       =======
   >--/.......\--<
      \,,,,,,,/
       ()) ())

10-07-21