Shared Semantic World Models

Autonomous vehicles require high-level semantic maps, which contain the activities of pedestrians and cars, to ensure safe navigation. High-level semantics can be obtained from mobile probe sensor data. Analyzing pedestrian trajectories obtained from mobile probe data is an effective approach to avoid collisions between autonomous vehicles and pedestrians. Such analyses of pedestrian trajectories can generate new information such as pedestrian behaviors in violation of traffic regulations. However, pedestrian trajectories obtained from mobile probe data significantly sparse and noisy, making it challenging to analyze pedestrian activity. To address this issue, we propose multiple daily data and graph-based approaches to treat sparse and noisy data for estimating the flow of pedestrians based on mobile probe data. To improve the sparseness of the data, multiple daily data are fused. After that, a pedestrian graph is created to enhance the region’s coverage by connecting the sparse data indicating the flow of pedestrians. This proposed approach successfully obtained pedestrian trajectory data from the sparse and noisy data. Moreover, it was possible to identify the potential locations where pedestrians tend to cross the street by analyzing the pedestrian flow. The results indicate that 83% of well-known regions where pedestrians tend to cross the street corresponded with those extracted using the proposed approach. Furthermore, a high-level semantic map of the regions where pedestrians tend to cross the street along a 1-km road is presented. The trajectory information obtained using the proposed approach is expected to be essential for understanding different scenarios of the interactions between individuals and autonomous vehicles.

Semantic maps are an important tool to provide robots with high-level knowledge about the environment, enabling them to better react to and interact with their surroundings. However, as a single measurement of the environment is solely a snapshot of a specific time, it does not necessarily reflect the underlying semantics. In this work, we propose a method to create a semantic map of a construction site by fusing multiple daily data. The construction site is measured by an unmanned aerial vehicle (UAV) equipped with a LiDAR. We extract clusters above ground level from the measurements and classify them using either a random forest or a deep learning based classifier. Furthermore, we combine the classification results of several measurements to generalize the classification of the single measurements and create a general semantic map of the working site. We measured two construction fields for our evaluation. The classification models can achieve an average intersection over union (IoU) score of 69.2 during classification on the Sanbongi field, which is used for training, validation and testing and an IoU score of 49.16 on a hold-out testing field. In a final step, we show how the semantic map can be employed to suggest a parking spot for a dump truck, and in addition, show that the semantic map can be utilized to improve path planning inside the construction site.