Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Multiview motion model based object detection
Learned object detection using multiple views of the same scene in a joint representation
Keywords: Deep Learning, Object Detection, Robotics
Semantic understanding of the environment and what objects are present is important to many robotic tasks, such as performing simple navigation, semantically aware mapping or mobile manipulation. Semantics enable performing many difficult tasks and are an essential upcoming tool in robotics.
Current challenges in existing implementations of object detects is the available dataset. To remain generic current state of the art deep learning based object detection is typically done from single images. Meaning that for each frame a robot would have to redetect the currently visible objects and reassociate them with the detections done in the previous frame. In addition to this being inefficient it does not fully leverage a lot of the available information, including the fact that the images are sequential and that the robot has an estimate for the poses at which the images were taken.
As a goal in this project we aim to use all the available information, to generate a joint prediction based on multiple images. We aim to leverage both the fact that the images are sequential as well as the availability of poses to better detect the object. Through this we hope to increase accuracy and speed, while at the same time reducing the necessity of frame to frame tracking.
Semantic understanding of the environment and what objects are present is important to many robotic tasks, such as performing simple navigation, semantically aware mapping or mobile manipulation. Semantics enable performing many difficult tasks and are an essential upcoming tool in robotics.
Current challenges in existing implementations of object detects is the available dataset. To remain generic current state of the art deep learning based object detection is typically done from single images. Meaning that for each frame a robot would have to redetect the currently visible objects and reassociate them with the detections done in the previous frame. In addition to this being inefficient it does not fully leverage a lot of the available information, including the fact that the images are sequential and that the robot has an estimate for the poses at which the images were taken.
As a goal in this project we aim to use all the available information, to generate a joint prediction based on multiple images. We aim to leverage both the fact that the images are sequential as well as the availability of poses to better detect the object. Through this we hope to increase accuracy and speed, while at the same time reducing the necessity of frame to frame tracking.
- Literature review for object detection and multiview deep learning.
- Finding or creating useful datasets.
- Implementing various baseline approaches that naively combine images.
- Exploring ways to integrate additional information such as poses or the sequential nature of the images into the predictions.
- Evaluation of results.
- Literature review for object detection and multiview deep learning. - Finding or creating useful datasets. - Implementing various baseline approaches that naively combine images. - Exploring ways to integrate additional information such as poses or the sequential nature of the images into the predictions. - Evaluation of results.
- Highly motivated and independent student.
- Interest in computer vision and deep learning.
- Excellent programming skills (Python) mandatory.
- Experience with object detection or learning preferable.
- Enrolled at ETH Zurich.
- Highly motivated and independent student. - Interest in computer vision and deep learning. - Excellent programming skills (Python) mandatory. - Experience with object detection or learning preferable. - Enrolled at ETH Zurich.
Please send your cv and transcripts to: Andrei Cramariuc (andrei.cramariuc@mavt.ethz.ch) and Florian Tschopp (florian.tschopp@mavt.ethz.ch)
Please send your cv and transcripts to: Andrei Cramariuc (andrei.cramariuc@mavt.ethz.ch) and Florian Tschopp (florian.tschopp@mavt.ethz.ch)