Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Learning Pose Estimation for Partially Occluded Objects from Simulation
This project addresses the task of 6D pose estimation for general-purpose objects, particularly when dealing with occlusion. We aim to leverage recent deep learning methods and synthetic data generation schemes to enable robust object manipulation.
Precise object pose estimation is essential to carry out robotic manipulation tasks effectively. Robotic systems usually rely on cameras and depth sensors to estimate the object's pose, which are prone to losing visual and geometric information due to self-occlusion during grasping. This problem is further exacerbated when involving multi-contact systems such as dexterous robotic hands. This project addresses the task of 6D pose estimation for general-purpose objects, particularly when dealing with occlusion. We aim to leverage recent deep learning methods and synthetic data generation schemes to enable robust object manipulation.
The first part of the project will focus on using a state-of-the-art photorealistic simulator to generate synthetic training data under varying degrees of occlusion. This involves simulating visual conditions observed during robotic manipulation for everyday objects and applying useful data augmentation techniques to collect a training dataset.
A key challenge in the project is to develop a pipeline for pose estimation of unseen objects. Previous attempts at addressing this challenge have involved direct regression of position and orientation from images, as well as employing key points for tracking objects dynamically. Building upon this foundation, we aim to explore novel architectures specifically focussing on robotic manipulation.
Precise object pose estimation is essential to carry out robotic manipulation tasks effectively. Robotic systems usually rely on cameras and depth sensors to estimate the object's pose, which are prone to losing visual and geometric information due to self-occlusion during grasping. This problem is further exacerbated when involving multi-contact systems such as dexterous robotic hands. This project addresses the task of 6D pose estimation for general-purpose objects, particularly when dealing with occlusion. We aim to leverage recent deep learning methods and synthetic data generation schemes to enable robust object manipulation. The first part of the project will focus on using a state-of-the-art photorealistic simulator to generate synthetic training data under varying degrees of occlusion. This involves simulating visual conditions observed during robotic manipulation for everyday objects and applying useful data augmentation techniques to collect a training dataset. A key challenge in the project is to develop a pipeline for pose estimation of unseen objects. Previous attempts at addressing this challenge have involved direct regression of position and orientation from images, as well as employing key points for tracking objects dynamically. Building upon this foundation, we aim to explore novel architectures specifically focussing on robotic manipulation.
- Literature review on existing methods for 6D pose estimation and object detection under occlusion
- Develop a pipeline to generate synthetic training data using a photo-realistic simulator
- Collect a training dataset with multiple objects covering various possible occlusion scenarios
- Implement and train a real-time architecture for 6D pose estimation using RGB-D images
- Perform benchmarking of the developed algorithm on real-world datasets
- Literature review on existing methods for 6D pose estimation and object detection under occlusion - Develop a pipeline to generate synthetic training data using a photo-realistic simulator - Collect a training dataset with multiple objects covering various possible occlusion scenarios - Implement and train a real-time architecture for 6D pose estimation using RGB-D images - Perform benchmarking of the developed algorithm on real-world datasets
- Highly motivated and autonomous student
- Programming experience in Python and PyTorch and Deep Learning
- (Preferred) Experience with Orbit or Isaac Sim
- Highly motivated and autonomous student - Programming experience in Python and PyTorch and Deep Learning - (Preferred) Experience with Orbit or Isaac Sim
Object Pose Estimation under Occlusion.pdf
