Real-Time Hand Contact Detection from Sparse Views for Infection Prevention in Surgery

This thesis is done in collaboration with the USZ and is centered around a dataset of over 20 real anesthesia inductions collected from 6 RGB-D cameras. The goal will be to create a light-weight and real-time capable hand contact detection system from sparse views (only 1-2 camera views). The proposed system will be used to detect potential breaches in hand hygiene protocols and warn medical staff before contact with the patient. The dataset is already annotated with 3D human body poses and meshes of the medical staff and the relevant objects and key areas in the room.

This thesis is done in collaboration with the USZ and is centered around a dataset of over 20 real anesthesia inductions collected from 6 RGB-D cameras. The goal will be to create a light-weight and real-time capable hand contact detection system from sparse views (only 1-2 camera views). The proposed system will be used to detect potential breaches in hand hygiene protocols and warn medical staff before contact with the patient. The dataset is already annotated with 3D human body poses and meshes of the medical staff and the relevant objects and key areas in the room.

You will be introduced to the dataset and our key project partners. Your tasks include: - Literature research for appropriate methods for real-time hand contact detection - Implementation and fine-tuning of the proposed detection system - Implementation of a feedback mechanism to the medical staff

You will be introduced to the dataset and our key project partners. Your tasks include:

- Literature research for appropriate methods for real-time hand contact detection

- Implementation and fine-tuning of the proposed detection system

- Implementation of a feedback mechanism to the medical staff

- Strong programming skills (Python, C#, C++, …)
- Experience with machine learning, data science or computer vision (Pytorch, OpenCV, …)
- The ability to take initiative and shape the direction of the project
- Enthusiasm for tackling practical challenges

As part of our research at the AR Lab within the Human Behavior Group we are working on automatically analyzing a user’s interaction with his environment in scenarios such as surgery or in human-robot collaboration. By collecting real-world datasets during those scenarios and using them for machine learning tasks such as activity recognition, object pose estimation or image segmentation we can gain an understanding of how a user performed during a given task. We can then utilize this information to provide the user with real-time feedback on his task using mixed reality devices, such as the Microsoft HoloLens, that can guide him and prevent him from doing mistakes.

As part of our research at the AR Lab within the Human Behavior Group we are working on automatically analyzing a user’s interaction with his environment in scenarios such as surgery or in human-robot collaboration. By collecting real-world datasets during those scenarios and using them for machine learning tasks such as activity recognition, object pose estimation or image segmentation we can gain an understanding of how a user performed during a given task. We can then utilize this information to provide the user with real-time feedback on his task using mixed reality devices, such as the Microsoft HoloLens, that can guide him and prevent him from doing mistakes.

- Collaboration with USZ
- Master thesis
- CV / ML

Please send your CV and masters grades to Sophokles Ktistakis (ktistaks@ethz.ch)

Please send your CV and masters grades to Sophokles Ktistakis (ktistaks@ethz.ch)