Localization of autonomous vehicles in complex, urban environments and the implementation based on a multi-Kalman approach in combination with a local dynamic map

Abstract

The performance of autonomous vehicles in complex urban environments is lacking compared to human driven vehicles. One of the reasons is the limitation of existing localization methods, which rely heavily on existing GNSS implementations with their known gaps, especially in urban canyons. Multipath effects combined with limited access to the sky are a major cause of errors in the range of 50 meters or more. To overcome this shortcoming, the authors have implemented a multi-localization-source, Kalman-based approach to improve the availability, reliability, and overall performance of vehicle self-localization. The system was implemented using a combination of GNSS, an inertial measurement unit, and a novel visual localization method based on landmarks stored in a cloud-hosted local dynamic map. The landmarks are dynamically received by the test vehicle via a 4G/5G connection from a geo-server. The information contains type, size, text, orientation, and position information of a landmark. For the implementation and evaluation, a ROS based system is installed on a test vehicle to collect sensor information and perform position computation in a real-world environment. Multiple landmarks such as traffic signs, trash cans, traffic lights or unique landmarks were identified as potential objects that can be used for localization. Unique and reappearing signs such as directions signs, the no parking sign or speed limit signs were implemented. The combination of unique and reappearing signs provides a good balance between implementation effort and localization performance in the context of a local dynamic map. The different signs were detected using a spherical camera and a single camera taken from a stereo camera setup. The signs were localized in the images and classified by different approaches using classic and machine learning based image recognition techniques. The size of the landmarks in the images and the position were used to reconstruct the angle and the distance to the vehicle. These can be transformed into a position in the relative coordinate system of the test vehicle. By matching these identified signs with the elements from the cloud-based local dynamic map, the location of the test vehicle could be estimated. Different localization and matching strategies were evaluated to be able to start the location with an estimation of the driving direction and one single, unique landmark, and extended to the parallel use of multiple landmarks to optimize performance, and robustness. The position estimated based on the visual localization is fused with acceleration, yaw rate, and GNSS position information to obtain an optimized, reliable localization. Therefore, a Kalman filter with multiple inputs was developed. Tests were performed on the test track for automated and connected driving (TAVF) in the Free and Hanseatic City of Hamburg in Germany. Performance criteria have been developed to assess both the system and the area under investigation. To evaluate the availability of basic GNSS, multiple test measurements of the number of available satellites and the estimated error of the GNSS position were performed. A significant number of locations were identified in which a purely satellite-based localization does not work. Tests with the combined position estimation were performed to measure the improvements. The measurements performed indicate a significant improvement, especially in situations where the GNSS signals are weak or disturbed. The activities are funded within the project “European Digital Dynamic Mapping” (EDDY).