This post marks the first month of working on my Ph.D. The initial work was employed on building the base hardware and software that will support the objectives of my Ph.D. Since we are going to build autonomous driving systems using deep learning and reinforcement learning, a computer specially tailored for these tasks is required. For that we will use DeepLar. On the software side, we will begin by training the agents in simulation, more precisely, in CARLA.
DeepLar
In the past months, DeepLar had two major problems: Failed boot due to NVIDIA persistenced daemon error (see issue #6), and GPU number 1 was often falling out of the bus (see issue #7). The first problem was solved by updating the OS and by enabling nvidia-persistenced. To fix the second problem, firsty I installed the most recent NVIDIA drivers but the problem remained unsolved. Apparently, the problem was the connection of the GPU and therefore a simple reseating solved the problem.
With the hardware problems solved, I began working on CARLA. We decided to install the server of the simulation in the DeepLar due to the excessive GPU memory required. For that I built a docker container with the CARLA server and then made a port tunneling from the DeepLar to my computer. One problem that arouse is the high latency which was causing a laggy simulation (see issue #8). To overcome this, I used a ethernet connection and the simulation was in real-time. Since DeepLar has only one ethernet port, we decided to buy a USB-ethernet conversor, which has not arrived yet.
CARLA
Training reinforcement learning agents is commonly done using OpenAI gym environments. After seaching for an implementation of such environment using CARLA, I found the following repository. This repository contains a lot more than we need, so I looked through the code and extracted the parts that are of interest to us (issue #1).
Since we are going to experiment different agents, towns, scenarios, inputs, etc. it is mandatory to develop a scalabe and flexible framework. So I developed a framework based on YAML files that enable the creation of different experiments (issue #2). To create a new experiment, there are three config files that must be written/adapted:
- obs_config.yaml: File with all sensors that the agent will use.
- policy.yaml: File with all the hyperparameters of the agent.
- train.yaml: File with all the configurations of the training, including the environment.
Once the config files are created, the training will be automatically configurated. This way we can save the config files and we have the complete information about the experiment, including the agent and the environment.
In order to visualize what happened in the episodes of the training and testing, a HUD was developed (issue #3). The goal of the HUD is to superimposed the video of the episode with critical information, such as the waypoints provided by the global planner, velocity, etc. Check the following example.
Besides the visual representation of each episode, we should also define each episode quantitatively. For that a folder with the stats of each episode was created (issue #13). This folder contains a csv file with some metrics per episode, such as reward, number of collisions, score, etc.
What's Next?
For now, we have the infrastructure to develop and train the agents, but we are missing the testing part. The next efforts will be devoted to add the testing part to the current infrastructure.
In addition, we will begin implementing a simple agent. The agent will be trained using Reinforcement Learning and consists of a RegNet as feature extractor and then a Soft Actor Critic (SAC). The inputs will be a single RGB camera, and the environment will contain a simple town with only straight tasks.
Furthermore, I'm expecting to test the USB-ethernet conversor and configure the LogiTech steering wheel to work in CARLA.