Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Learning Object Permanence with Memory based Neural Networks
In this project we want to extend previous vision pipelines to take into account both spatial and temporal aspects using deep learning. This would improve the robot’s awareness of its surroundings for navigation and control tasks.
In order to act in our environments, autonomous robots must be aware of their surroundings and consider also elements that can not be seen at a specific time instant. For that purpose, keeping track of objects in the scene is a key aspect. However, state of the art detection pipelines are usually neglecting such spatial and temporal consistency and make a prediction based on the current image only.
As previously stated, we want to extend previous pipelines to take into account spatial and temporal aspects in order to improve the robot’s awareness of its surroundings.
In order to act in our environments, autonomous robots must be aware of their surroundings and consider also elements that can not be seen at a specific time instant. For that purpose, keeping track of objects in the scene is a key aspect. However, state of the art detection pipelines are usually neglecting such spatial and temporal consistency and make a prediction based on the current image only. As previously stated, we want to extend previous pipelines to take into account spatial and temporal aspects in order to improve the robot’s awareness of its surroundings.
- Extensive literature review
- Development of a suitable pipeline for generating training data
- Development of a learning pipeline for spatial and temporal awareness
- Demonstrate on real hardware (potentially with sim2real transfer)
- Extensive literature review - Development of a suitable pipeline for generating training data - Development of a learning pipeline for spatial and temporal awareness - Demonstrate on real hardware (potentially with sim2real transfer)
- Experience with deep learning (LSTM, autoencoders)
- Experience with pytorch/tensorflow
- Autonomous working skills
- C++ and ROS experience is a plus
- Experience with deep learning (LSTM, autoencoders) - Experience with pytorch/tensorflow - Autonomous working skills - C++ and ROS experience is a plus
David Hoeller, dhoeller@ethz.ch
Andreea Tulbure, tulbure@mavt.ethz.ch
David Hoeller, dhoeller@ethz.ch Andreea Tulbure, tulbure@mavt.ethz.ch