Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Environment Mapping for large-scale Robot Teleoperation
In this project, we would like to develop a volumetric mapping pipeline, enabling real-world operation of our full-scale excavator HEAP.
Keywords: environment mapping, teleoperation, LiDAR, camera, fusion, deep learning
Teleoperation for large scale machinery is currently under development and most solutions rely on streaming high-quality raw images to the operator. However, there is no solution available that recovers the depth information which is one of the most important features for the human operator. In this project, we want to create a full 3D map around HEAP and use it to render a 3rd person view of the machine and its surroundings.
Depending on the time available and the scope of the project, the mapping pipeline can be extended to handle dynamic changes in the environment. Additionally, maps can be augmented with RGB information. If the project is done as a master thesis, we could further include generative machine learning models to fill holes and render occluded parts of the scene.
This project involves a mix of simulation (for quick debugging), dataset processing, and possibly hardware experiments. Ideally, potential candidates have experience with cameras and/or lidars.
We can also tailor the project to your needs (see the requirements below) and we are looking forward to your application.
Literature:
[1] Steinbrücker F, Sturm J, Cremers D. Volumetric 3D mapping in real-time on a CPU. In2014 IEEE International Conference on Robotics and Automation (ICRA) 2014 May 31 (pp. 2021-2028). IEEE.
[2] Kähler O, Prisacariu VA, Murray DW. Real-time large-scale dense 3D reconstruction with loop closure. In European Conference on Computer Vision 2016 Oct 8 (pp. 500-516). Springer, Cham.
[3] Oleynikova H, Taylor Z, Fehr M, Siegwart R, Nieto J. Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning. In2017 Ieee/rsj International Conference on Intelligent Robots and Systems (iros) 2017 Sep 24 (pp. 1366-1373). IEEE.
[4] Bârsan IA, Liu P, Pollefeys M, Geiger A. Robust dense mapping for large-scale dynamic environments. In2018 IEEE International Conference on Robotics and Automation (ICRA) 2018 May 21 (pp. 7510-7517). IEEE.
Teleoperation for large scale machinery is currently under development and most solutions rely on streaming high-quality raw images to the operator. However, there is no solution available that recovers the depth information which is one of the most important features for the human operator. In this project, we want to create a full 3D map around HEAP and use it to render a 3rd person view of the machine and its surroundings.
Depending on the time available and the scope of the project, the mapping pipeline can be extended to handle dynamic changes in the environment. Additionally, maps can be augmented with RGB information. If the project is done as a master thesis, we could further include generative machine learning models to fill holes and render occluded parts of the scene.
This project involves a mix of simulation (for quick debugging), dataset processing, and possibly hardware experiments. Ideally, potential candidates have experience with cameras and/or lidars.
We can also tailor the project to your needs (see the requirements below) and we are looking forward to your application.
Literature:
[1] Steinbrücker F, Sturm J, Cremers D. Volumetric 3D mapping in real-time on a CPU. In2014 IEEE International Conference on Robotics and Automation (ICRA) 2014 May 31 (pp. 2021-2028). IEEE.
[2] Kähler O, Prisacariu VA, Murray DW. Real-time large-scale dense 3D reconstruction with loop closure. In European Conference on Computer Vision 2016 Oct 8 (pp. 500-516). Springer, Cham.
[3] Oleynikova H, Taylor Z, Fehr M, Siegwart R, Nieto J. Voxblox: Incremental 3d euclidean signed distance fields for on-board mav planning. In2017 Ieee/rsj International Conference on Intelligent Robots and Systems (iros) 2017 Sep 24 (pp. 1366-1373). IEEE.
[4] Bârsan IA, Liu P, Pollefeys M, Geiger A. Robust dense mapping for large-scale dynamic environments. In2018 IEEE International Conference on Robotics and Automation (ICRA) 2018 May 21 (pp. 7510-7517). IEEE.
- Literature review on volumetric mapping
- Testing available open-source packages
- Implementation of a mapping pipeline
- Evaluation on datasets and on the real robot
- For MA only: usage of generative machine learning models
- Literature review on volumetric mapping - Testing available open-source packages - Implementation of a mapping pipeline - Evaluation on datasets and on the real robot - For MA only: usage of generative machine learning models
- High motivation and interest in the topic
- Structured, independent and goal-oriented working behavior
- Good programming skills in C++
- Experience with a camera and/or lidar sensor
- Computer vision related class
- High motivation and interest in the topic - Structured, independent and goal-oriented working behavior - Good programming skills in C++ - Experience with a camera and/or lidar sensor - Computer vision related class
- Edo Jelavic, jelavice@ethz.ch
- Julian Nubert, julian.nubert@mavt.ethz.ch
- Edo Jelavic, jelavice@ethz.ch - Julian Nubert, julian.nubert@mavt.ethz.ch