Deep Reinforcement Learning for Autonomous Driving in AirSim
AirSim is an open source simulator for drones and cars developed by Microsoft.
In this article, we will introduce deep reinforcement learning using a single Windows machine instead of distributed, from the tutorial “Distributed Deep Reinforcement Learning for Autonomous Driving” using AirSim.
AirSim
AirSim is an open source simulator for drones and cars developed by Microsoft AI and Research Group.
For details of AirSim, please visit the following site.
https://github.com/Microsoft/AirSim
The Autonomous Driving Cookbook
The autonomous driving cookbook is published by Microsoft on GitHub.
For details of the autonomous driving cookbook, please visit the following site.
https://github.com/Microsoft/AutonomousDrivingCookbook
The autonomous driving cookbook currently has two tutorials, one of which is “Autonomous Driving using End-to-End Deep Learning: an AirSim tutorial” introduced in the following article,
another is “Distributed Deep Reinforcement Learning for Autonomous Driving” introduced in this article.
Distributed Deep Reinforcement Learning for Autonomous Driving
Distributed deep reinforcement learning for autonomous driving is a tutorial to estimate the steering angle from the front camera image using distributed deep reinforcement learning.
The title of the tutorial is distributed deep reinforcement learning, but it also makes it possible to train on a single machine for demonstration purposes.
In this article, we will introduce deep reinforcement learning using a single Windows machine, not distributed.
The tutorial is a style that executes the following three notebooks on Jupyter Notebook.
Environment Setup
The environment setup introduced in the following article can be used as it is.
However, since there is an additional package, install it with the following command.
pip install requests==2.18.4
Explore the algorithm
Explore the algorithm explains the reward function and network architecture.
Launch the local training job
Launch the local training job creates a training batch file train.bat.
In addition, the part of airsim_path='E:\\AD_Cookbook_AirSim\\' in notebook needs to be changed according to your own environment.
In the created batch file path\to\AutonomousDrivingCookbook\DistributedRL\Share\scripts_downpour\app\train.bat, training is performed by executing distributed_agent.py with parameters as follows.
python distributed_agent.py batch_update_frequency=10 ...
Since README.md is written as follows,
For demonstration purposes, the model can also be trained on a single machine (see instructions below). The model can take up to 5 days to train from scratch, but can train in a few hours using transfer learning. To train the model locally, a machine with a GPU is required.
We trained for three hours using transfer learning, but the behavior of the trained model was similar to sample_model.json provided as a reference.
It is written that multi-node training using transfer learning will take six hours, we think that it is better to prepare more than 10 hours.
Run the model
Run the model inputs the front camera image and state (speed) from AirSim launched in the neighborhood environment and outputs car_controls including steering angle to AirSim.
To launch AirSim, the following PowerShell command runs in the directory which AirSim simulator was extracted.
.AD_Cookbook_Start_AirSim.ps1 neighborhood
Also, you change the MODEL_FILENAME='sample_model.json' part in the notebook to point to the model you trained.
Summary
In this article, we introduced deep reinforcement learning using a single Windows machine instead of distributed, from the tutorial “Distributed Deep Reinforcement Learning for Autonomous Driving” using AirSim.
