Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Continual Learning for Novel Semantic Discovery
The overarching goal of this project is to enable robots that can continually and autonomously learn from their environment by adapting to novel environments and learn to identify new object categories. The established approach of supervised learning on large datasets always has problems with domain and sim-to-real gaps. This line of work therefore represents a shift towards learning during the robot’s mission, naturally removing any domain gaps but also reducing the amount of supervision that can be applied.
The overarching goal of this project is to enable robots that can continually and autonomously learn from their environment by adapting to novel environments and learn to identify new object categories. The established approach of supervised learning on large datasets always has problems with domain and sim-to-real gaps. This line of work therefore represents a shift towards learning during the robot’s mission, naturally removing any domain gaps but also reducing the amount of supervision that can be applied.
There have been exploratory studies of such robotic systems (https://arxiv.org/abs/1907.10008, https://proceedings.mlr.press/v164/blum22a.html, http://arxiv.org/abs/2201.01073, http://arxiv.org/abs/2108.04562), which however all miss a systemic way for evaluation and comparison.
This thesis will develop a simulation-based evaluation protocol for continual semantic learning on robots. Building on existing simulation frameworks (https://aihabitat.org, https://github.com/DLR-RM/oaisys), we define environments (offices, living spaces, planetary surfaces, urban driving), clear datasets for prior knowledge (what kind of data has the robot seen before?), and a set of novel scenes that the robot is tested in. We then evaluate and improve the recently proposed algorithms.
**Special Opportunity** As part of this project, we try to offer the opportunity of a stay at the German Aerospace Center (DLR).
The overarching goal of this project is to enable robots that can continually and autonomously learn from their environment by adapting to novel environments and learn to identify new object categories. The established approach of supervised learning on large datasets always has problems with domain and sim-to-real gaps. This line of work therefore represents a shift towards learning during the robot’s mission, naturally removing any domain gaps but also reducing the amount of supervision that can be applied. There have been exploratory studies of such robotic systems (https://arxiv.org/abs/1907.10008, https://proceedings.mlr.press/v164/blum22a.html, http://arxiv.org/abs/2201.01073, http://arxiv.org/abs/2108.04562), which however all miss a systemic way for evaluation and comparison. This thesis will develop a simulation-based evaluation protocol for continual semantic learning on robots. Building on existing simulation frameworks (https://aihabitat.org, https://github.com/DLR-RM/oaisys), we define environments (offices, living spaces, planetary surfaces, urban driving), clear datasets for prior knowledge (what kind of data has the robot seen before?), and a set of novel scenes that the robot is tested in. We then evaluate and improve the recently proposed algorithms. **Special Opportunity** As part of this project, we try to offer the opportunity of a stay at the German Aerospace Center (DLR).
- Familiarize yourself with the literature
- Create a controlled learning task in the simulator
- Evaluate a baseline method
- Iterate by adding more environments and improving the method
- Define clear metrics and perform extensive final evaluations
- Familiarize yourself with the literature - Create a controlled learning task in the simulator - Evaluate a baseline method - Iterate by adding more environments and improving the method - Define clear metrics and perform extensive final evaluations
- Highly dedicated and motivated student
- Familiar with python, c++, and either tensorflow or pytorch
- Existing experience with simulators or ROS is a plus
- Highly dedicated and motivated student - Familiar with python, c++, and either tensorflow or pytorch - Existing experience with simulators or ROS is a plus
Please send your CV and transcript to blumh@ethz.ch and Marcus.Mueller@dlr.de
Please send your CV and transcript to blumh@ethz.ch and Marcus.Mueller@dlr.de