Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Integration of Calibrated Projection Mapping with Anymal for Industrial Augmented Reality
This project involves the integration of a digital projector with the walking robot “Anymal C” for industrial Augmented Reality. A combination of mobile robotics and game engine graphics will allow for projecting digital information onto 3D environments such as construction sites.
Keywords: Legged Robotics, Augmented Reality, Unreal Engine, State Estimation, Construction Automation
This project will work towards the integration of a digital projector mounted onto the back of Anymal C, enabling a mobile augmented reality (AR) interface capable of displaying complex information onto large physical scenes such as construction sites.
A calibration routine will be developed that can register the intrinsic/extrinsic properties of a digital projector with the existing onboard sensors of Anymal C (LiDAR, Depth Cameras). This calibration information along with the onboard state estimation of the robot will then be used to replicate the projector properties as a virtual camera in a game engine scene that mirrors the physical world (Using Unreal Engine). This mirrored scene will provide the graphical elements of the augmented interface, and methods will need to be developed to best synchronize and align the robot state and map with this virtual scene (known CAD geometry). Depending on the performance of the projection during motion, the state estimation may be adapted to improve the augmented reality projection.
This project will work towards the integration of a digital projector mounted onto the back of Anymal C, enabling a mobile augmented reality (AR) interface capable of displaying complex information onto large physical scenes such as construction sites.
A calibration routine will be developed that can register the intrinsic/extrinsic properties of a digital projector with the existing onboard sensors of Anymal C (LiDAR, Depth Cameras). This calibration information along with the onboard state estimation of the robot will then be used to replicate the projector properties as a virtual camera in a game engine scene that mirrors the physical world (Using Unreal Engine). This mirrored scene will provide the graphical elements of the augmented interface, and methods will need to be developed to best synchronize and align the robot state and map with this virtual scene (known CAD geometry). Depending on the performance of the projection during motion, the state estimation may be adapted to improve the augmented reality projection.
- Integration and Intrinsic/Extrinsic Calibration of Digital Projector with Anymal C
- Replication of Projector as Virtual Camera in Game Engine Scene (Unreal Engine)
- Synchronization (and probable modification) of robot state estimation with virtual scene
- Registration of pre-modelled game-engine scene with mapped environment
- Development of demo application for projected augmented reality onto scene
- Documentation of integration including measured accuracy and latency
- Integration and Intrinsic/Extrinsic Calibration of Digital Projector with Anymal C - Replication of Projector as Virtual Camera in Game Engine Scene (Unreal Engine) - Synchronization (and probable modification) of robot state estimation with virtual scene - Registration of pre-modelled game-engine scene with mapped environment - Development of demo application for projected augmented reality onto scene - Documentation of integration including measured accuracy and latency
- Knowledge of C++
- Familiarity with basic State Estimation
- Experience with game engines (Unity/Unreal) and ROS preferred
- Knowledge of C++ - Familiarity with basic State Estimation - Experience with game engines (Unity/Unreal) and ROS preferred
Perry Franklin - fperry@mavt.ethz.ch
Ryan Luke Johns - johns@arch.ethz.ch
Perry Franklin - fperry@mavt.ethz.ch Ryan Luke Johns - johns@arch.ethz.ch