Recent state-of-the-art approaches for semantic segmentation usually work well from frontal views and within a limited distance range. During landing these criteria are often violated, posing a major challenge to the performance of existing semantic segmentation approaches. While objects that are too close to the camera are usually partially visible, objects that are far away are mostly represented by too few pixels in an image to be distinctive enough. Although viewpoint changes can be addressed by an aerial dataset, the lack of distinctiveness and context makes the distance range issue harder to be addressed (see Figure above). Handling such cases, where the neural networks commonly fail, current systems attempt to use Simultaneous Localization And Mapping (SLAM) to localize the robot with respect to a pose, where semantics are still functional. While SLAM can partially alleviate the problem, this is not always sufficient and it is dependent on the quality of SLAM estimates, which can vary significantly onboard UAVs with limited resources. In this project, we plan to use direct image-to-image localization to enable propagation of semantics from higher altitudes, where more context is available, to lower ones. Thus, the distance range problem can be addressed, enabling the propagation of semantic scene understanding to boost robustness.
Recent state-of-the-art approaches for semantic segmentation usually work well from frontal views and within a limited distance range. During landing these criteria are often violated, posing a major challenge to the performance of existing semantic segmentation approaches. While objects that are too close to the camera are usually partially visible, objects that are far away are mostly represented by too few pixels in an image to be distinctive enough. Although viewpoint changes can be addressed by an aerial dataset, the lack of distinctiveness and context makes the distance range issue harder to be addressed (see Figure above). Handling such cases, where the neural networks commonly fail, current systems attempt to use Simultaneous Localization And Mapping (SLAM) to localize the robot with respect to a pose, where semantics are still functional. While SLAM can partially alleviate the problem, this is not always sufficient and it is dependent on the quality of SLAM estimates, which can vary significantly onboard UAVs with limited resources. In this project, we plan to use direct image-to-image localization to enable propagation of semantics from higher altitudes, where more context is available, to lower ones. Thus, the distance range problem can be addressed, enabling the propagation of semantic scene understanding to boost robustness.
- Literature review on existing semantic propagation approaches.
- Selection of a suitable solution to address the aerial localization problem.
- Development of a semantic propagation method to be integrated into a localization pipeline.
- Evaluation and comparison of the proposed method against state-of-the-art approaches and report writing.
- Literature review on existing semantic propagation approaches. - Selection of a suitable solution to address the aerial localization problem. - Development of a semantic propagation method to be integrated into a localization pipeline. - Evaluation and comparison of the proposed method against state-of-the-art approaches and report writing.