Register now After registration you will be able to apply for this opportunity online.
Proactive Human-Robot Collaboration using Human Sensory Information (MT)
The goal of this thesis is to achieve a proactive human-robot collaboration (HRC) using real-time sensor measurements from the user. Currently, we implemented a HRC system that enables different collaborative tasks (object handovers, hand following, etc.). However, at the moment this system is purely reactive and requires gestures or speech commands to interact with the robot. Thus, we would like to implement a human intent prediction, based on sensor measurements, such as skeleton tracking, gaze tracking, IMU measurements or even heartrate and blood pressure (through a smartwatch). This could be based on AI or other probabilistic models, where the system proactively prepares the next action or even pre-empts it entirely. This would require a series of case studies to gather data about human behavior during interaction with our system, with subsequent data analysis and training or fitting of the models. The whole Pipeline should be demonstrated on a select use-case.
Keywords: AI, eye tracking, hand tracking, machine perception, control systems, ROS2
We explore the possibilities of HRI
We explore the possibilities of HRI
see abstract
see abstract
Requirements Willing to learn unity or know unity/C-sharp development for Hololens Some prior Knowledge in AI & Modelling Basics of computer vision
Not specified
Not specified
Lucas Gimeno and Sophokles Ktistakis
(gimenol@ethz.ch, ktistaks@ethz.ch)
Lucas Gimeno and Sophokles Ktistakis (gimenol@ethz.ch, ktistaks@ethz.ch)
