Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Neural Representations for Robotic Mapping
Perceiving and understanding one's environment is a crucial component to many robotic tasks. This project aims at leveraging recent advances in computer vision, in particular neural scene representations, for robotic mapping.
Keywords: Robotics, Computer Vision, Deep Learning, Neural Nets, Neural Representations, Scene Understanding, Mapping, SLAM
Perceiving and understanding one's environment is a crucial component to many robotic tasks from planning to interaction. In particular, the robot requires an accurate representation of the geometry of its surroundings (Dense map) as well as an understanding of the different objects in the scene (Semantic map).
Traditional approaches integrate sensor measurements to obtain a dense reconstruction of an environment [1], and use convolutional neural networks (CNNs) to integrate semantic information into that map [2]. Recent advances in computer vision use neural representations to encode the geometry of entire objects [3, 4]. These have the potential to unify semantics and geometry in a lightweight representation and allow the robot to reason about more than only observed parts of an object.
In this project, the goal is to leverage neural representations for robotic mapping. This can include the development of systems to extract such latent codes from sensor data, update the object representations based on multiple observations and making the obtained representation utilizable for robotic tasks such as planning. The methods can be developed using an existing simulation environment with ground truth and publicly available datasets for training and/or further testing. In case of project success, the student is invited to publish his or her work.
[1] H. Oleynikova et al., “Voxblox: Incremental 3D Euclidean Signed Distance Fields for on-board MAV planning”, IROS 2017.
[2] M. Grinvald et al., “Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery”, IROS 2019.
[3] J. Park et al., “DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation”, CVPR 2019.
[4] K. Genova et al., “Local Deep Implicit Functions for 3D Shape”, CVPR 2020.
Perceiving and understanding one's environment is a crucial component to many robotic tasks from planning to interaction. In particular, the robot requires an accurate representation of the geometry of its surroundings (Dense map) as well as an understanding of the different objects in the scene (Semantic map).
Traditional approaches integrate sensor measurements to obtain a dense reconstruction of an environment [1], and use convolutional neural networks (CNNs) to integrate semantic information into that map [2]. Recent advances in computer vision use neural representations to encode the geometry of entire objects [3, 4]. These have the potential to unify semantics and geometry in a lightweight representation and allow the robot to reason about more than only observed parts of an object.
In this project, the goal is to leverage neural representations for robotic mapping. This can include the development of systems to extract such latent codes from sensor data, update the object representations based on multiple observations and making the obtained representation utilizable for robotic tasks such as planning. The methods can be developed using an existing simulation environment with ground truth and publicly available datasets for training and/or further testing. In case of project success, the student is invited to publish his or her work.
[1] H. Oleynikova et al., “Voxblox: Incremental 3D Euclidean Signed Distance Fields for on-board MAV planning”, IROS 2017.
[2] M. Grinvald et al., “Volumetric Instance-Aware Semantic Mapping and 3D Object Discovery”, IROS 2019.
[3] J. Park et al., “DeepSDF: Learning Continuous Signed Distance Functions for Shape Representation”, CVPR 2019.
[4] K. Genova et al., “Local Deep Implicit Functions for 3D Shape”, CVPR 2020.
- Literature Review.
- Development of a system to extract neural object representations from sensor data.
- Development of a system to update neural object representations from multiple observations.
- Evaluation of the proposed system.
- Literature Review. - Development of a system to extract neural object representations from sensor data. - Development of a system to update neural object representations from multiple observations. - Evaluation of the proposed system.
- Highly motivated and independent student.
- Interest in Computer Vision and Robotics.
- Solid programming skills (Python and/or C++) are mandatory.
- Experience with Deep Learning (Tensorflow / PyTorch) is a plus.
- Highly motivated and independent student. - Interest in Computer Vision and Robotics. - Solid programming skills (Python and/or C++) are mandatory. - Experience with Deep Learning (Tensorflow / PyTorch) is a plus.
If you are interested please send an email with a short CV and motivation to schmluk@mavt.ethz.ch and andrei.cramariuc@mavt.ethz.ch.
If you are interested please send an email with a short CV and motivation to schmluk@mavt.ethz.ch and andrei.cramariuc@mavt.ethz.ch.