Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Efficient Object Segmentation from RGB-D (MT)
The goal of this project is to develop an object segmentation framework for RGB-D images.
Keywords: Object Segmentation, RGB-D Perception, Computer Vision, Deep learning
Object segmentation is a crucial part of the perception pipeline of robots: it is a prerequisite skill for understanding complex scenes and operating in real-world environments.
Most current state-of-the-art learning-based object segmentation algorithms operate solely on RGB images, without making efficient use of the depth sensing modality that is commonly available in robotic systems. Furthermore, these approaches require large amounts of labeled data and are restricted to work only for the set of object classes used during training, thus failing to detect and segment unknown objects.
In this project, we want to build a robust and efficient perception pipeline that combines RGB and depth inputs to discover and segment objects in the observed scene with no prior assumptions on their aspect or their semantic class.
The first step will be for the student to familiarize himself/herself with the literature, the current solutions, and the available sensors to get an understanding of the problem. A next step would be the implementation of a state-of-the-art object segmentation method as a baseline. Further, the method can be improved in terms of its accuracy or speed and evaluated on a collected real-world dataset. Depending on the interests and skills of the student, in addition to classical computer vision techniques the work can explore how the system can benefit from including deep learning methods.
Related work:
- https://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnumber=8275331 (unsupervised, edge detection based)
- https://arxiv.org/pdf/1703.06370.pdf (weakly supervised, objectness based)
- https://arxiv.org/pdf/1709.06158.pdf (example annotated dataset)
Object segmentation is a crucial part of the perception pipeline of robots: it is a prerequisite skill for understanding complex scenes and operating in real-world environments. Most current state-of-the-art learning-based object segmentation algorithms operate solely on RGB images, without making efficient use of the depth sensing modality that is commonly available in robotic systems. Furthermore, these approaches require large amounts of labeled data and are restricted to work only for the set of object classes used during training, thus failing to detect and segment unknown objects.
In this project, we want to build a robust and efficient perception pipeline that combines RGB and depth inputs to discover and segment objects in the observed scene with no prior assumptions on their aspect or their semantic class.
The first step will be for the student to familiarize himself/herself with the literature, the current solutions, and the available sensors to get an understanding of the problem. A next step would be the implementation of a state-of-the-art object segmentation method as a baseline. Further, the method can be improved in terms of its accuracy or speed and evaluated on a collected real-world dataset. Depending on the interests and skills of the student, in addition to classical computer vision techniques the work can explore how the system can benefit from including deep learning methods.
- Literature review on object segmentation from RGB, Depth and combined RGB-D data.
- Implementation of a segmentation method that combines color and depth cues to discover object candidates in an image.
- Collection and annotation of a small-scale real-world dataset containing scenes with various objects of different scales.
- Evaluation of the segmentation algorithm.
- (Optional) Extension of the method to include prior information, e.g. the classes of the objects or their rough location.
- Literature review on object segmentation from RGB, Depth and combined RGB-D data. - Implementation of a segmentation method that combines color and depth cues to discover object candidates in an image. - Collection and annotation of a small-scale real-world dataset containing scenes with various objects of different scales. - Evaluation of the segmentation algorithm. - (Optional) Extension of the method to include prior information, e.g. the classes of the objects or their rough location.
- Highly motivated and independent student
- Strong interest in Computer Vision and/or Deep Learning
- C++ knowledge
- Experience with ROS and 3D sensors is beneficial
- Highly motivated and independent student - Strong interest in Computer Vision and/or Deep Learning - C++ knowledge - Experience with ROS and 3D sensors is beneficial
If you are interested in this project, please send your transcripts and CV to - Michael Pantic michael.pantic@mavt.ethz.ch - Margarita Grinvald margarita.grinvald@mavt.ethz.ch
If you are interested in this project, please send your transcripts and CV to - Michael Pantic michael.pantic@mavt.ethz.ch - Margarita Grinvald margarita.grinvald@mavt.ethz.ch