Register now After registration you will be able to apply for this opportunity online.
Reconstructing liquids from multiple views with 3D Gaussian Splatting
This project reconstructs liquids from multi-view imagery, segmenting fluid regions using methods like Mask2Former and reconstructing static scenes with 3D Gaussian Splatting or Mast3r. The identified fluid clusters initialize a particle-based simulation, refined for temporal consistency and enhanced by optional thermal data and visual language models for fluid properties.
Keywords: 3D reconstruction, Gaussian Splatting, physics simulation
This project focuses on the reconstruction of liquids using multi-view imagery, with the potential integration of thermal camera data. Initially, it aims to reconstruct a single, static frame, with temporal reconstruction as a subsequent goal. Fluid regions in posed or unposed images are segmented using semantic segmentation methods, such as Mask2Former. The static scene is then reconstructed using 3D Gaussian Splatting [1] or Mast3r [2]. Segmented fluid regions are identified as clusters of Gaussians, which serve as inputs to a particle-based physical simulation (e.g., Taichi or NVIDIA FleX), where Gaussian properties inform initial particle parameters. Optionally, visual language models (e.g., GPT) may provide fluid properties like viscosity. For image sequences, the simulation enforces temporal consistency through constraints from subsequent frames, optimized via back-propagation. Thermal data could further refine physical parameters and assist in fluid segmentation. This approach combines advanced reconstruction techniques with physics-based simulation to achieve accurate, dynamic fluid modeling.
[1] Kerbl, B., Kopanas, G., Leimkühler, T. and Drettakis, G., 2023. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4), pp.139-1.
[2] Leroy, V., Cabon, Y. and Revaud, J., 2025. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision (pp. 71-91). Springer, Cham.
Other sources:
Dynamic Fluid Synthesis with Gaussian Splatting, https://amysteriouscat.github.io/GaussianSplashing/
PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics https://arxiv.org/abs/2311.12198
This project focuses on the reconstruction of liquids using multi-view imagery, with the potential integration of thermal camera data. Initially, it aims to reconstruct a single, static frame, with temporal reconstruction as a subsequent goal. Fluid regions in posed or unposed images are segmented using semantic segmentation methods, such as Mask2Former. The static scene is then reconstructed using 3D Gaussian Splatting [1] or Mast3r [2]. Segmented fluid regions are identified as clusters of Gaussians, which serve as inputs to a particle-based physical simulation (e.g., Taichi or NVIDIA FleX), where Gaussian properties inform initial particle parameters. Optionally, visual language models (e.g., GPT) may provide fluid properties like viscosity. For image sequences, the simulation enforces temporal consistency through constraints from subsequent frames, optimized via back-propagation. Thermal data could further refine physical parameters and assist in fluid segmentation. This approach combines advanced reconstruction techniques with physics-based simulation to achieve accurate, dynamic fluid modeling.
[1] Kerbl, B., Kopanas, G., Leimkühler, T. and Drettakis, G., 2023. 3d gaussian splatting for real-time radiance field rendering. ACM Trans. Graph., 42(4), pp.139-1. [2] Leroy, V., Cabon, Y. and Revaud, J., 2025. Grounding image matching in 3d with mast3r. In European Conference on Computer Vision (pp. 71-91). Springer, Cham.
Other sources: Dynamic Fluid Synthesis with Gaussian Splatting, https://amysteriouscat.github.io/GaussianSplashing/ PhysGaussian: Physics-Integrated 3D Gaussians for Generative Dynamics https://arxiv.org/abs/2311.12198
