One of the hottest areas of application for deep learning is undoubtedly autonomously driving. While the first thing that comes to mind when you talk about the topic is a self-driving car, in fact almost any vehicle can be turned into an autonomous one. For example, it took only one night to train a model for self-steering urban waterways. Yet, cars would be the easiest to look at since this is the most widely discussed use case for autonomous driving.
Thus, we’ll go through the problem space, discuss its intricacies and build self-driving solution utilizing the Unity game engine, training a neural network on top of the Valohai platform. Regardless of the technologies used, you’ll get an understanding of the basics as well as the code to tweak for yourself.
We are not going to use a real car, but a simulator from Udacity created with the Unity game engine. Our code and approach are based on Manajit Pal’s excellent tutorial .
Main steps of the tutorial:
Collect data while manually driving the car in a simulator
Create a new Valohai machine learning project and link it to your data and code
Train a deep neural network to mimic the driving based on training data
Inference: Let the model loose in the simulator to drive for itself
Next, connect to the git repository where your Python code awaits. Click Settings -> Repository from Valohai and type the URL:
Then click Save . Remember that you can use your own repository/fork, too
Link a data store
The next step is setting up the data store connection.
Note: If you skipped the gather step altogether and decide to use the prebuilt data, you don’t need to set up the data store at all.
Note: If your data is in a public URL, there is no need to set up a data store in the Valohai project.
If you put your data in a private data store, set up a data store connection using your credentials. If you get stuck, more help can be found in Valohai Data Stores documentation .
Step #3: Train a deep learning self-driving model
After linking the git repository and data store (optional) to your Valohai project, we are ready to run our experiment and train the model.
Just click the big blue Create execution button on the right, skim through the default settings. These settings are defined in the git project’s valohai.yaml file (for more information on what the YAML file contains, see the Valohai docs ).
The train-model step we select is going to run the train.py script from within our git repository.
If you are using your own data, change the driving_dataset input URL to point to your own tar file.
Finally, click the blue Create execution button at the bottom of the page. Valohai will launch your cloud CPU/GPU instance, install the docker environment within the project, fetch your code and data and run the code. This MLOps is all handled by the Valohai platform so that you don’t need to do it by hand.
During the training process, you can observe your metrics like loss and valid_loss from the metadata tab. This is possible, because Valohai is clever enough to parse all JSON formatted outputs from the execution log and is able to visualize them automatically.
After the execution is done, you can find the saved model from the Outputs folder. Click on the download button and save the model.h5 file to your local hard disk.
Easy, isn't it? It looks a bit like magic. However, if you want to learn more about the model architecture and everything that happens under the hood, you can read the original article that explains the peculiarities of the convolutional neural network developed by Nvidia for training purposes.
Step #4: Inference
Next, we need to clone the git repository to our local computer because we are going to run inference locally. We want to see the car driving with our very own eyes!
Finally, copy the model.h5 you just downloaded to the same folder you cloned the repository.
Start the simulator, select AUTONOMOUS MODE and leave it running (the car will not move yet).
Next, go to your terminal and run:
python drive.py model.h5
The terminal output will come alive, the car will throttle ahead and hopefully stay on the road :).
In this tutorial, we demonstrated a “hello world” of the self-driving vehicle on top of the Valohai machine learning platform.
Many steps were left manual for the sake of understanding. In real life, one would automate the file uploading and downloading using the Valohai API / CLI and possibly add automated tests answering questions like “did the car stay on the road” or “did we hit anything”. These could be executed inside a docker container in Valohai, to make the entire pipeline fully automated.
Despite the possibilities that machine learning provides to propel autonomous driving, a study conducted by Capgemini in early 2019 saw that only 10% of the major automotive players use AI at scale. One way to break the spell would be to automate machine orchestration, version control, and CI/CD processes with machine learning tools like Valohai.
Read the "Automotive Machine Learning in Production" eBook to learn how the Valohai as a Machine Learning Operations pralform can help your team to build models 10X faster.
Valohai trialSign up for a free trial on Valohai.Join the waitlist and get invited to see Valohai in action.