Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
A Cognitive Computer Vision Framework for the Automated Robotic Assembly of Lego Pieces
The goal of this MSc thesis is to design and implement a cognitive computer vision framework capable of understanding and extracting the knowledge for putting together Lego pieces into Lego assemblies based on a visual rendering of assembly instructions.
See attached document for further detail.
Keywords: Segmentation, Object recognition, 3D reconstruction
Within the context of Industry 4.0, manufacturing is gearing away from monolithic assembly lines towards flexible collaborative assembly pods suitable for (mass) customized fabrication. A key aspect in this scenario is availability of a seamless yet robust environment for teaching individual work steps to cobots even by non-technical people (e.g, by demonstration, or from ingesting a manual).
The goal of this project is to explore an initial step in this direction by designing and implementing a cognitive computer vision framework capable of understanding and extracting the knowledge for putting together Lego pieces into Lego assemblies based on a visual rendering of assembly instructions (see the attached Figure for an example).
Within the context of Industry 4.0, manufacturing is gearing away from monolithic assembly lines towards flexible collaborative assembly pods suitable for (mass) customized fabrication. A key aspect in this scenario is availability of a seamless yet robust environment for teaching individual work steps to cobots even by non-technical people (e.g, by demonstration, or from ingesting a manual).
The goal of this project is to explore an initial step in this direction by designing and implementing a cognitive computer vision framework capable of understanding and extracting the knowledge for putting together Lego pieces into Lego assemblies based on a visual rendering of assembly instructions (see the attached Figure for an example).
The thesis is roughly composed of the following work packages:
1. Definition of a suitable semantic description for the assembly task
2. Generation and maintenance of a small Lego workpiece library (CAD renderings, or actual reconstructed models)
3. Recognition of Lego pieces on workspace and matching to items in the library
4. Segmentation of pictorial instructions in the form of CAD renderings into a semantic graph of assembly instructions
- Segmentation of pictorial elements in pdf into a directed graph of work steps
- Recognition of Lego pieces in each pictorial element. Extraction of semantic information (i.e., Lego brick IDs and their configurations involved)
- Generation of assembly instructions based on visual delta between subsequent pictorial elements
5. Implementation and validation of a prototype system capable of assembling individual steps as per the extracted assembly instructions (in simulation or on physical UR3 robot)
The thesis is roughly composed of the following work packages:
1. Definition of a suitable semantic description for the assembly task
2. Generation and maintenance of a small Lego workpiece library (CAD renderings, or actual reconstructed models)
3. Recognition of Lego pieces on workspace and matching to items in the library
4. Segmentation of pictorial instructions in the form of CAD renderings into a semantic graph of assembly instructions
- Segmentation of pictorial elements in pdf into a directed graph of work steps
- Recognition of Lego pieces in each pictorial element. Extraction of semantic information (i.e., Lego brick IDs and their configurations involved)
- Generation of assembly instructions based on visual delta between subsequent pictorial elements
5. Implementation and validation of a prototype system capable of assembling individual steps as per the extracted assembly instructions (in simulation or on physical UR3 robot)
The student is expected to have a solid working knowledge of C++ on Linux as well as prior experience with computer vision (segmentation, object recognition, 3D reconstruction). The student is further expected to show strong self-motivation and dedication to see the project through.
The student is expected to have a solid working knowledge of C++ on Linux as well as prior experience with computer vision (segmentation, object recognition, 3D reconstruction). The student is further expected to show strong self-motivation and dedication to see the project through.
This MSc Thesis will be conducted at IBM Research – Zurich, at its Ruschlikon, Switzerland lab. The student will be jointly supervised by the two supervisors.
IBM Research supervisor: Martin Rufli <mru@zurich.ibm.com>
ETH supervisor: Margarita Chli <chlim@ethz.ch>
This MSc Thesis will be conducted at IBM Research – Zurich, at its Ruschlikon, Switzerland lab. The student will be jointly supervised by the two supervisors.
IBM Research supervisor: Martin Rufli <mru@zurich.ibm.com>