Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
ANYmal Operator meets Mixed Reality
This project aims to build an intuitive mixed reality (MR) interface for robot operators using Apple’s ARKit capabilities. Building on an existing iOS application, we want to provide the operators with a more immersive and informative interface. This involves showing the robot's sensory readings, sketching a global path for it, and playing back the robot's states in the application.
Keywords: ios, app, robotics, augmented reality
Quadrupedal robots are increasingly recognized for their unparalleled locomotion capabilities, particularly in navigating complex, unstructured environments. However, a notable challenge arises in operating these robots in unstructured environments: the perceptual gap between the robot's onboard sensory inputs and the remote operator's understanding of the environment. The lack of direct environmental perception and situational awareness often hinders the operator's ability to make informed decisions and issue precise commands, potentially compromising the robot's effectiveness and safety.
This project aims to tackle this issue by building an intuitive mixed reality (MR) interface for robot operators using Apple’s ARKit capabilities. Building on an existing iOS application, we want to provide the operators with a more immersive and informative interface. This involves showing the robot's sensory readings, sketching a global path for it, and playing back the robot's states in the application.
References:
- Apple Developer: Augmented Reality
- Hoenig, Wolfgang, et al. "Mixed reality for robotics." IEEE/RSJ IROS, 2015.
Quadrupedal robots are increasingly recognized for their unparalleled locomotion capabilities, particularly in navigating complex, unstructured environments. However, a notable challenge arises in operating these robots in unstructured environments: the perceptual gap between the robot's onboard sensory inputs and the remote operator's understanding of the environment. The lack of direct environmental perception and situational awareness often hinders the operator's ability to make informed decisions and issue precise commands, potentially compromising the robot's effectiveness and safety.
This project aims to tackle this issue by building an intuitive mixed reality (MR) interface for robot operators using Apple’s ARKit capabilities. Building on an existing iOS application, we want to provide the operators with a more immersive and informative interface. This involves showing the robot's sensory readings, sketching a global path for it, and playing back the robot's states in the application.
References:
- Apple Developer: Augmented Reality - Hoenig, Wolfgang, et al. "Mixed reality for robotics." IEEE/RSJ IROS, 2015.
- Improving an existing iOS app with more customization options
- Visualizing the entire robot state (including joint state)
- Conducting tests in various outdoor and indoor environments
- Comparing the designed app to the current PC application (rViz)
- Improving an existing iOS app with more customization options - Visualizing the entire robot state (including joint state) - Conducting tests in various outdoor and indoor environments - Comparing the designed app to the current PC application (rViz)
- Highly motivated and autonomous student
- Experience with iOS app development
- Knowledge of Python and ROS
- Highly motivated and autonomous student - Experience with iOS app development - Knowledge of Python and ROS