Reinforcement learning

RL Examples

This document will provide a simple example of using reinforcement learning to control PND humanoid robots. The following will introduce how to use the isaac_gym simulation platform to train the control algorithm of the robot.

Note

If you are using the urdf in the wiki, please make sure to change toe_left and toe_right to the following value:
collision name="toe_"
origin rpy="1.57 0 0" xyz="0 0 0"
In the RL code find _config. Change flip_visual_attachments = False to flip_visual_attachments = True
replace_cylinder_with_capsule = False to replace_cylinder_with_capsule = True

Download address

pnd_adam_gym_public-main

Hardware preparation

Since the isaac_gym simulation platform requires CUDA, this article recommends that the hardware be equipped with an NVIDIA graphics card (video memory > 8GB, RTX series graphics card) and install the corresponding graphics card driver.

python train.py --task=adam

If the following interface appears in isaac_gym, then the training begins.

Training begins

The terminal output window is as follows:

Terminal output

Close the visual interface by adding --headless.

python play.py --task=adam --headless

Environment preparation

It is recommended to configure this environment in the virtual environment conda.

Create a virtual environment
```
conda create -n pndrobot python=3.8
```
Activate the virtual environment
```
conda activate pndrobot
```

Install CUDA and pytorch

conda install pytorch==1.13.1 torchvision==0.14.1 torchaudio==0.13.1 pytorch-cuda=11.6 -c pytorch -c nvidia

Download the Isaac Gym Preview 3 simulation platform. After decompression, enter the python directory and install it using pip.
```
cd isaacgym/python && pip install -e.
```
Install the rsl_rl library
```
cd rsl_rl && pip install -e.
```

Install the official PND example

cd pnd_humanoid_robot && pip install -e.

Install other configurations

pip install numpy==1.23.5

pip install tensorboard

pip install opencv-python

Model training and usage

Activate the reinforcement learning virtual environment and switch to the legged_gym/scripts directory.
```
conda activate pndrobot

cd pnd_humanoid_robot/pnd_humanoid_robot_gym/scripts
```
Execute the training instruction and start training.
```
python train.py --task=pnd_humanoid_robot
```
Close the visual interface by adding --headless.

Run the test instruction.

python play.py --task=pnd_humanoid_robot

Acknowledgement

legged_gym

rsl_rl