Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Development and evaluation of a stereo setup for object tracking in surgery/assembly based on synthetic training data
Using computer vision approaches, process states of surgery and manual assembly can be detected automatically. In this work, an automated deep learning pipeline based on synthetic training data for process state detection should be developed and evaluated. Base for this is a priorly developed stereo camera setup that is to be adapted.
Specific tasks of the project can be adapted to fit the skills and interests of the student.
Keywords: Deep learning, neural networks, object tracking, action recognition, object detection, stereo vision, stereo reconstruction, synthetic training data, computer vision
Smart cameras automatically collect information from images using computer vision algorithms. Actions and decisions can be triggered using this information.
Together with medical and industrial partners, we are researching smart camera applications in the field of surgery and manual manufacturing. Using the cameras, process states can automatically be determined and personnel can be supported with information about realized, missed or upcoming tasks. Further, they can be alarmed if mistakes happen. In our research, we are focused on the application scenarios of such smart cameras.
Smart cameras automatically collect information from images using computer vision algorithms. Actions and decisions can be triggered using this information. Together with medical and industrial partners, we are researching smart camera applications in the field of surgery and manual manufacturing. Using the cameras, process states can automatically be determined and personnel can be supported with information about realized, missed or upcoming tasks. Further, they can be alarmed if mistakes happen. In our research, we are focused on the application scenarios of such smart cameras.
In previous projects, we realized an experimental stereo camera setup that allows for deep learning based object tracking. This setup should be used and adapted to study an application scenario from surgery or assembly in depth. To allow for an easy setup of new applications, a deep learning pipeline based on synthetic training data should be developed and evaluated.
Key elements of the thesis are the following:
- Realizing a pipeline for the creation of synthetic training data and training of a neural network for object tracking.
- Training models to track objects that are relevant in the chosen application.
- Developing an approach to use the tracking information to determine application states.
- Conducting an evaluation study on the developed approach.
In previous projects, we realized an experimental stereo camera setup that allows for deep learning based object tracking. This setup should be used and adapted to study an application scenario from surgery or assembly in depth. To allow for an easy setup of new applications, a deep learning pipeline based on synthetic training data should be developed and evaluated. Key elements of the thesis are the following: - Realizing a pipeline for the creation of synthetic training data and training of a neural network for object tracking. - Training models to track objects that are relevant in the chosen application. - Developing an approach to use the tracking information to determine application states. - Conducting an evaluation study on the developed approach.
You are highly interested in realizing applied computer vision approaches Desired are interest and skills in programming (Python is preferred) An understanding of using Blender for image rendering will be beneficial Experience with deep learning and training of neural networks such as YOLO, MobileNet etc. will help You can work independently and are eager to interact with an industrial partner You approach problems systematically and manage your projects methodically You are a team player, curious and come up with innovative ideas Willingness to integrate into a professional work environment
We focus on human-centred product development and regard the link between research and education as the key to excellence in training. We see ourselves as a partner for industry and promote the continuous transfer of knowledge through cooperation, as well as the training and further education of students and graduates to strengthen the competitiveness of mechanical engineering industry.
We focus on human-centred product development and regard the link between research and education as the key to excellence in training. We see ourselves as a partner for industry and promote the continuous transfer of knowledge through cooperation, as well as the training and further education of students and graduates to strengthen the competitiveness of mechanical engineering industry.
Master thesis. Start: From September 2022
Jonas Conrad: conradj@ethz.ch
Tobias Stauffer: tobiasta@ethz.ch
Jonas Conrad: conradj@ethz.ch Tobias Stauffer: tobiasta@ethz.ch