Robust neural network for sim-to-real gap in end-to-end autonomous driving

Abstract

A neural network architecture for end-to-end autonomous driving is presented, which is robust against discrepancies in system dynamics during the training process and in application. The proposed network architecture presents a first step to alleviate the simulation to reality gap with respect to differences in system dynamics. A vehicle is trained to drive inside a given lane in the CARLA simulator. The data is used to train NVIDIA’s PilotNet. When an offset is given to the steering angle of the vehicle while the trained network is being applied, PilotNet will not keep the vehicle inside the lane as expected. A new architecture is proposed called PilotNet∆, which is robust against steering angle offsets. Experiments in the simulator show that the vehicle will stay in the lane, although the steering properties of the vehicle differ.