By Wenchao Liu

There are numerous technologies used in a real self-driving car. However, when it comes to self-driving RC cars, people normally just use a subset of those technologies. The different technologies use different sensors and different algorithms. Here, I will go through three popular ones.

The simplest approach involves no sensors whatsoever. How is that possible? Well, it’s possible if you can just manually drive through the course once, record the steering and throttle inputs, and replay them at the same starting point. The drawback of this approach is that the car drifts, meaning the car deviates further from the original trajectory as it goes further. That said, this simple approach can tackle all the autonomous RC car challenges as long as they satisfy a few conditions. First, you have to have access to the course before the race. Second, the course does not change. Finally, you can place the car exactly where you originally put it when you recorded the data. It also helps if the rule only allows one car per race, so no other cars would pump into yours.

The second approach involves a camera and a neural network. The flagship product of this approach is the Donkey Car, which uses only one camera and one Raspberry Pi. You first have to drive through the course a couple times to collect training data for the neural network. Because of the computation constrains of the Pi, you have to upload the data to another more powerful computer, train the neural network there and transfer the trained model to the Pi. I have no personal experience with this approach, because cameras, computer vision and neural networks are too much for me! That said, I know for a fact that this approach doesn’t work in total darkness and might not work well if lighting changes a lot.

The third approach is Lidar-based, and it is my favorite. The pipeline is to use SLAM, collect waypoints by manually driving through the course, and use motion planning and trajectory tracking. SLAM refers to simultaneous localization and mapping, which means the car localizes itself and maps the environment at the same time. After the car has a map and knows where it is, you can manually drive the car and collect waypoints. Once you have the waypoints the car should hit, you use motion planning to plan for the trajectory through the waypoints and trajectory tracking to make the car follow the desired trajectory. This approach is the most powerful, because it can tackle dynamic environments. For instance, if a car stopped in front of you, your motion planning algorithm will give you another path to go around the car.

Here you have it, three approaches for making an RC car drive autonomously in a course. Real self-driving cars are definitely more sophisticated, but some ideas are very similar. For instance, Tesla’s approach is using mostly cameras and no Lidars. Other companies such as Waymo and GM Cruise use both cameras and Lidars. Only time will tell which one will prevail!