Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Depth Learning for Multi-Sensor SLAM systems
Accurate online dense 3D reconstruction is critical for applications such as virtual and augmented reality. The advent of depth sensors such as Time-of-Flight cameras and accurate RGB stereo depth sensors helped make these systems applicable to real-world scenarios. No depth sensor is, however, noise free. Noisy depth measurements result in erroneous 3D shape predictions. To decrease the noise of the depth sensor, we aim to feed the current shape reconstruction as additional information to a depth denoising/refinement network. This effectively closes the loop between depth capture and shape reconstruction and should help improve the accuracy of online dense 3D reconstruction systems. By leveraging multiple depth sensors, we can additionally investigate how to best find synergies across depth sensors.
**Method:** We will study the setup where the camera poses are assumed to be known and the task is to build a 3D scene from a sequence of depth frames coming from either a single depth sensor or from two depth sensors. The first task is to implement and integrate a depth rendering algorithm into our existing codebase and then to investigate meaningful ways to design and supervise the system for accurate scene reconstruction.
**Requirement**s: A basic understanding of computer vision (e.g. attended Image Analysis and Computer Vision) and experience working with Python and Linux.
**Professor:** Luc Van Gool
The project is co-supervised by Erik Sandström and Martin Oswald.
**References:**
[1] RoutedFusion: Learning Real-time Depth Map Fusion, CVPR 2020
[2] A volumetric method for building complex models from range images, SIGGRAPH 1996
**Method:** We will study the setup where the camera poses are assumed to be known and the task is to build a 3D scene from a sequence of depth frames coming from either a single depth sensor or from two depth sensors. The first task is to implement and integrate a depth rendering algorithm into our existing codebase and then to investigate meaningful ways to design and supervise the system for accurate scene reconstruction.
**Requirement**s: A basic understanding of computer vision (e.g. attended Image Analysis and Computer Vision) and experience working with Python and Linux.
**Professor:** Luc Van Gool
The project is co-supervised by Erik Sandström and Martin Oswald.
[2] A volumetric method for building complex models from range images, SIGGRAPH 1996
1. Research existing techniques that combine depth refinement with dense 3D reconstruction
2. Implement a ray-marching algorithm for depth rendering
3. Design a meaningful depth learning network
4. Design an appropriate loss function; supervision, self-supervision/continual learning?
5. Investigate single and multi-sensor depth fusion, when is depth learning helpful?
1. Research existing techniques that combine depth refinement with dense 3D reconstruction
2. Implement a ray-marching algorithm for depth rendering
3. Design a meaningful depth learning network
4. Design an appropriate loss function; supervision, self-supervision/continual learning?
5. Investigate single and multi-sensor depth fusion, when is depth learning helpful?
To apply please contact Erik Sandström at: esandstroem@vision.ee.ethz.ch
To apply please contact Erik Sandström at: esandstroem@vision.ee.ethz.ch