Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Image - Semantics Consistency
In this project we will exploit generative models (e.g. GANs) to provide introspection to semantic segmentation networks in open-world settings as faced by mobile robots.
Keywords: Deep Learning, Generative models, Semantic Segmentation, Robotics Scene Understanding.
Current state of the art semantic segmentation methods are based on deep learning. However, this methods are known to fail when the deployment environment does not follow the data distribution of the training dataset [1]. By taking advantage of state of the art generative models, this project will propose a way of detecting inconsistencies produced by the semantic segmentation network. The student will build on top of, and improve, previous research [2,3] with an extra objective for modularity.
[1] Blum H, Sarlin PE, Nieto J, Siegwart R, Cadena C. The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation. arXiv preprint arXiv:1904.03215. 2019 Apr 5.
[2] Haldimann D, Blum H, Siegwart R, Cadena C. This is not what I imagined: Error Detection for Semantic Segmentation through Visual Dissimilarity. ArXiv, abs/1909.00676. 2019 March 15.
[3] Lis K, Nakka K, Salzmann M, Fua P. Detecting the Unexpected via Image Resynthesis. arXiv preprint arXiv:1904.07595. 2019 Apr 16.
Current state of the art semantic segmentation methods are based on deep learning. However, this methods are known to fail when the deployment environment does not follow the data distribution of the training dataset [1]. By taking advantage of state of the art generative models, this project will propose a way of detecting inconsistencies produced by the semantic segmentation network. The student will build on top of, and improve, previous research [2,3] with an extra objective for modularity.
[1] Blum H, Sarlin PE, Nieto J, Siegwart R, Cadena C. The Fishyscapes Benchmark: Measuring Blind Spots in Semantic Segmentation. arXiv preprint arXiv:1904.03215. 2019 Apr 5.
[2] Haldimann D, Blum H, Siegwart R, Cadena C. This is not what I imagined: Error Detection for Semantic Segmentation through Visual Dissimilarity. ArXiv, abs/1909.00676. 2019 March 15.
[3] Lis K, Nakka K, Salzmann M, Fua P. Detecting the Unexpected via Image Resynthesis. arXiv preprint arXiv:1904.07595. 2019 Apr 16.
- Implement state of the art semantic segmentation networks.
- Implement state of the art GANs for image generation.
- Investigate distance learning and cycle consistency.
- Propose, implement and validated a method for detecting inconsistencies (mistakes) on semantic segmentation pipelines.
- Implement state of the art semantic segmentation networks. - Implement state of the art GANs for image generation. - Investigate distance learning and cycle consistency. - Propose, implement and validated a method for detecting inconsistencies (mistakes) on semantic segmentation pipelines.
- Highly motivated and independent student.
- Strong C++ and/or Python coding skills.
- Experience with deep learning frameworks (e.g: Tensorflow, PyTorch).
- Students from outside of D-MAVT (particularly, from D-INFK, D-ITET, D-PHYS, and D-MATH) are also highly encouraged to apply.
- Highly motivated and independent student. - Strong C++ and/or Python coding skills. - Experience with deep learning frameworks (e.g: Tensorflow, PyTorch). - Students from outside of D-MAVT (particularly, from D-INFK, D-ITET, D-PHYS, and D-MATH) are also highly encouraged to apply.
If you are interested, please send your transcripts and CV to Cesar Cadena (cesarc@ethz.ch) and Hermann Blum (blumh@ethz.ch).
If you are interested, please send your transcripts and CV to Cesar Cadena (cesarc@ethz.ch) and Hermann Blum (blumh@ethz.ch).