Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
3D Multi-View Surface Reconstruction Using RGBD Data
The goal of this project is to implement and evaluate state-of-the-art 3D surface reconstruction methods using multi-modality sensor data.
Keywords: 3D Reconstruction, Implicit Functions, Neural Radiance Fields, Signed Distance Function, Deep Learning, Computer Vision
High quality 3D surface reconstruction is a challenging problem in computer vision. Recently proposed methods that combine neural radiance fields [1] and signed distance functions [2] achieved state-of-the-art surface reconstruction performance on different acquisition scenarios [3-6]. However, most of these methods use the data from single camera with multiple view directions. Our goal in this project is to combine multi-modality sensors with different resolution and sensitivity to achieve high quality 3D surface reconstruction.
High quality 3D surface reconstruction is a challenging problem in computer vision. Recently proposed methods that combine neural radiance fields [1] and signed distance functions [2] achieved state-of-the-art surface reconstruction performance on different acquisition scenarios [3-6]. However, most of these methods use the data from single camera with multiple view directions. Our goal in this project is to combine multi-modality sensors with different resolution and sensitivity to achieve high quality 3D surface reconstruction.
The aim of this project is the implementation and/or evaluation of state-of-the-art 3D surface reconstruction methods [3-7] on different datasets. The results will serve as the baselines for the developed hardware and software solutions. The effects of adding multi-modality sensor data will be explored to achieve optimal acquisition strategies and hardware setups.
Your qualifications / what we are looking for:
- Prior experience with image or volume reconstruction methods
- Excellent programming skills in Python including PyTorch or TensorFlow
- Knowledge of common computer vision practices
- Genuine interest in open and reproducible research
- Experience with version control systems (Git)
- Good English communication skills
What we offer:
- A project with potential real-world applications
- The possibility to bring in your own ideas and combine them with state-of-the-art algorithms
- Close supervision by an interdisciplinary team of experts
- The possibility to acquire your own data (optimize hardware setup for software development)
References:
[1] Mildenhall, Ben, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. "Nerf: Representing scenes as neural radiance fields for view synthesis." Communications of the ACM 65, no. 1 (2021): 99-106.
[2] Osher, Stanley, Ronald Fedkiw, and K. Piechor. "Level set methods and dynamic implicit surfaces." Appl. Mech. Rev. 57, no. 3 (2004): B15-B15.
[3] Wang, Peng, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. "Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction." arXiv preprint arXiv:2106.10689 (2021).
[4] Yariv, Lior, Jiatao Gu, Yoni Kasten, and Yaron Lipman. "Volume rendering of neural implicit surfaces." Advances in Neural Information Processing Systems 34 (2021): 4805-4815.
[5] Azinović, Dejan, Ricardo Martin-Brualla, Dan B. Goldman, Matthias Nießner, and Justus Thies. "Neural rgb-d surface reconstruction." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6290-6301. 2022.
[6] Li, Zhaoshuo, Thomas Müller, Alex Evans, Russell H. Taylor, Mathias Unberath, Ming-Yu Liu, and Chen-Hsuan Lin. "Neuralangelo: High-Fidelity Neural Surface Reconstruction." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8456-8465. 2023.
[7] Schonberger, Johannes L., and Jan-Michael Frahm. "Structure-from-motion revisited." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104-4113. 2016.
The aim of this project is the implementation and/or evaluation of state-of-the-art 3D surface reconstruction methods [3-7] on different datasets. The results will serve as the baselines for the developed hardware and software solutions. The effects of adding multi-modality sensor data will be explored to achieve optimal acquisition strategies and hardware setups.
Your qualifications / what we are looking for: - Prior experience with image or volume reconstruction methods - Excellent programming skills in Python including PyTorch or TensorFlow - Knowledge of common computer vision practices - Genuine interest in open and reproducible research - Experience with version control systems (Git) - Good English communication skills
What we offer: - A project with potential real-world applications - The possibility to bring in your own ideas and combine them with state-of-the-art algorithms - Close supervision by an interdisciplinary team of experts - The possibility to acquire your own data (optimize hardware setup for software development)
References: [1] Mildenhall, Ben, Pratul P. Srinivasan, Matthew Tancik, Jonathan T. Barron, Ravi Ramamoorthi, and Ren Ng. "Nerf: Representing scenes as neural radiance fields for view synthesis." Communications of the ACM 65, no. 1 (2021): 99-106. [2] Osher, Stanley, Ronald Fedkiw, and K. Piechor. "Level set methods and dynamic implicit surfaces." Appl. Mech. Rev. 57, no. 3 (2004): B15-B15. [3] Wang, Peng, Lingjie Liu, Yuan Liu, Christian Theobalt, Taku Komura, and Wenping Wang. "Neus: Learning neural implicit surfaces by volume rendering for multi-view reconstruction." arXiv preprint arXiv:2106.10689 (2021). [4] Yariv, Lior, Jiatao Gu, Yoni Kasten, and Yaron Lipman. "Volume rendering of neural implicit surfaces." Advances in Neural Information Processing Systems 34 (2021): 4805-4815. [5] Azinović, Dejan, Ricardo Martin-Brualla, Dan B. Goldman, Matthias Nießner, and Justus Thies. "Neural rgb-d surface reconstruction." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6290-6301. 2022. [6] Li, Zhaoshuo, Thomas Müller, Alex Evans, Russell H. Taylor, Mathias Unberath, Ming-Yu Liu, and Chen-Hsuan Lin. "Neuralangelo: High-Fidelity Neural Surface Reconstruction." In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8456-8465. 2023. [7] Schonberger, Johannes L., and Jan-Michael Frahm. "Structure-from-motion revisited." In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4104-4113. 2016.
Please send your CV and transcript to Dr. Berkan Lafci (berkan.lafci@uzh.ch). Links to previous work (e.g., your GitHub profile) are highly appreciated.
Please send your CV and transcript to Dr. Berkan Lafci (berkan.lafci@uzh.ch). Links to previous work (e.g., your GitHub profile) are highly appreciated.
