Register now After registration you will be able to apply for this opportunity online.
Robust Multi-Modal 3D Reconstruction
Recent advances in neural scene reconstruction, such as Neural Radiance Fields (NeRF) and Gaussian Splatting, have significantly improved the performance of downstream tasks, including novel view synthesis and geometric reconstruction. Building on these innovations, multi-modal approaches have been explored to incorporate additional scene attributes such as depth, surface normals, thermal data, and semantic information to enrich existing scene representations. However, current multi-modal methods often rely on a tightly coupled correspondence between RGB data and other modalities, which limits their applicability in uncontrolled, real-world scenarios.
Keywords: Multi-modal fusion, Visual-based 3D reconstruction, Thermal imaging
This project aims to address the challenge of strong cross-modal dependencies in 3D reconstruction, with a particular focus on RGB and thermal imaging. The goal is to develop robust techniques that can establish correspondences between different modalities, or align image features in three-dimensional space, thereby enabling accurate reconstruction without the need for strict pixel-level alignment across modalities.
This project aims to address the challenge of strong cross-modal dependencies in 3D reconstruction, with a particular focus on RGB and thermal imaging. The goal is to develop robust techniques that can establish correspondences between different modalities, or align image features in three-dimensional space, thereby enabling accurate reconstruction without the need for strict pixel-level alignment across modalities.
- Conduct a comprehensive literature review on multi-modal 3D reconstruction techniques, with an emphasis on robustness and cross-modal integration
- Investigate contrastive learning approaches for identifying correspondences between RGB and non-RGB modalities
- Explore strategies for 3D-space feature alignment and rendering-based pose optimization
- Extend the reconstruction framework to incorporate additional modalities, including semantic labels and inputs with weak or no geometric correlation
- Conduct a comprehensive literature review on multi-modal 3D reconstruction techniques, with an emphasis on robustness and cross-modal integration - Investigate contrastive learning approaches for identifying correspondences between RGB and non-RGB modalities - Explore strategies for 3D-space feature alignment and rendering-based pose optimization - Extend the reconstruction framework to incorporate additional modalities, including semantic labels and inputs with weak or no geometric correlation
