Miniature autonomous vehicle environment for sim-to-real transfer in reinforcement learning

Abstract

An experimental setup and preliminary validation of a platform for sim-to-real transfer in reinforcement learn ing for autonomous driving is presented. The platform features a 1:87 scale miniature autonomous vehicle, the tinycar, within a detailed miniature world that includes urban and rural settings. Key components include a simulation for training machine learning models, a digital twin with a tracking system using overhead cameras, an automatic repositioning mechanism of the miniature vehicle to reduce hu man intervention when training in the real-world, and an encoder based approach for reducing the state space dimension for the machine learning algorithms. The tinycar is equipped with a steering servo, DC motor, front-facing camera, and a custom PCB with an ESP32 micro-controller. A custom UDP-based network protocol enables real-time communication. The machine learning setup uses semantically segmented lanes of the streets as an input. These colored lanes can be directly produced by the simulation. In the real-world a machine learning based segmentation method is used to achieve the segmented lanes. Twomethodsare used to train a controller (actor): Imitation learning as a supervised learning method in which a Stanley controller serves as a teacher. Secondly, Twin Delayed Deep Deterministic Policy Gradient (TD3) is used to minimize the Cross-Track Error (CTE) of the miniature vehicle with respect to its lateral position in the street. Both methods are applied equally in simulation and in the real-world and are compared. Preliminary results show high accuracy in lane following and intersection navigation in simulation and real world, supported by precise real-time feedback from the tracking system. While full integration of the RL model is ongoing, the presented results show the platform’s potential to further investigate the sim-to-real as pects in autonomous driving.