Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Semantic Place Recognition for Multi-Robot Applications
Place Recognition using deep learning semantic segmentation for multi-robot applications.
Keywords: Place Recognition, Semantic Segmentation, Deep Learning, Computer Vision, Robotics
With the emergence of powerful techniques for robotic egomotion estimation and map building that follow the SLAM (Simultaneous Localization And Mapping) paradigm, Place Recognition has become of fundamental importance for robotic autonomy, enabling global accurate maps, relocalization and even collaboration between different robots performing SLAM. Place recognition is, however, a challenging task, due to the large variability in a scene's appearance that can be observed in the real world, caused by changes in illumination, seasons, or presence of occlusions and dynamic objects. Considering Place Recognition using images captured by different robots (e.g. aerial and ground robots) is especially challenging given that the same place can be visited from very different viewpoints (see image above).
Motivated by the remarkable progress that deep neural networks have been experiencing towards semantic scene understanding, several approaches have attempted to incorporate semantic knowledge to improve Place Recognition. The main idea of augmenting Place Recognition with semantic information is based on the fact that semantics are resilient to transient variations of the appearance of a place and the conditions of the scene when capturing it in an image (e.g. viewpoint, illumination).
A common strategy for Semantic Place Recognition is to focus on scene features found on informative structures (e.g. buildings) and discard ambiguous and non-discriminative features (e.g. sky, grass). As such, the goal of this project is to develop a semantic segmentation strategy to improve a Place Recognition system by taking in consideration the most discriminative features in the scene in order to improve Place Recognition for a multi-robot application in which large changes in viewpoint are commonly present.
With the emergence of powerful techniques for robotic egomotion estimation and map building that follow the SLAM (Simultaneous Localization And Mapping) paradigm, Place Recognition has become of fundamental importance for robotic autonomy, enabling global accurate maps, relocalization and even collaboration between different robots performing SLAM. Place recognition is, however, a challenging task, due to the large variability in a scene's appearance that can be observed in the real world, caused by changes in illumination, seasons, or presence of occlusions and dynamic objects. Considering Place Recognition using images captured by different robots (e.g. aerial and ground robots) is especially challenging given that the same place can be visited from very different viewpoints (see image above).
Motivated by the remarkable progress that deep neural networks have been experiencing towards semantic scene understanding, several approaches have attempted to incorporate semantic knowledge to improve Place Recognition. The main idea of augmenting Place Recognition with semantic information is based on the fact that semantics are resilient to transient variations of the appearance of a place and the conditions of the scene when capturing it in an image (e.g. viewpoint, illumination).
A common strategy for Semantic Place Recognition is to focus on scene features found on informative structures (e.g. buildings) and discard ambiguous and non-discriminative features (e.g. sky, grass). As such, the goal of this project is to develop a semantic segmentation strategy to improve a Place Recognition system by taking in consideration the most discriminative features in the scene in order to improve Place Recognition for a multi-robot application in which large changes in viewpoint are commonly present.
- WP1: Familiarization with our existing Place Recognition system implementation.
- WP2: Research into existing state-of-the-art Semantic Segmentation techniques.
- WP3: Development of a Semantic Segmentation approach to be integrated into an existing Place Recognition system.
- WP4: Experimentation of the method against state-of-the-art approaches in Place Recognition and report writing.
- WP1: Familiarization with our existing Place Recognition system implementation. - WP2: Research into existing state-of-the-art Semantic Segmentation techniques. - WP3: Development of a Semantic Segmentation approach to be integrated into an existing Place Recognition system. - WP4: Experimentation of the method against state-of-the-art approaches in Place Recognition and report writing.
The student taking this project needs to be highly motivated, preferably with strong analytical skills, while experience in coding in C/C++, Python and Deep Learning would be very beneficial.
The student taking this project needs to be highly motivated, preferably with strong analytical skills, while experience in coding in C/C++, Python and Deep Learning would be very beneficial.
- Fabiola Maffra, fmaffra@mavt.ethz.ch
- Lucas Teixeira, lteixeira@mavt.ethz.ch
- Fabiola Maffra, fmaffra@mavt.ethz.ch - Lucas Teixeira, lteixeira@mavt.ethz.ch