Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Exploiting Priors Learned by Diffusion Generative Models for Domain-Adaptive Semantic Segmentation
In this project, the students will work with the hottest computational imaging topic of 2022 – Deep Denoising Diffusion models – to exploit these pretrained models in new domain settings with limited semantic annotations.
Deep Learning models are notoriously data-hungry but work exceptionally well in many mapping and imaging applications if fed enough data. However, often annotated training data is scarce or not immediately available. A typical approach to supervised learning on a new domain is a combination of the following ingredients: (1) a sufficiently good learned model initialization trained on a similar domain and (2) domain adaptation by fine-tuning the initial model for the target domain. In the field of semantic image segmentation, ImageNet initialization has become ubiquitous [1]. However, recently developed generative diffusion models set a new standard for synthesizing realistic images [2] and therefore are a good candidate to act as learned priors for downstream tasks, such as semantic segmentation [3]. In this project, the students will explore this avenue in multiple application contexts, including standard computer vision and remote sensing benchmarks. The project is suitable for bachelor, Master project, and Master Thesis. Group work is possible at Bachelor or Master project level.
Deep Learning models are notoriously data-hungry but work exceptionally well in many mapping and imaging applications if fed enough data. However, often annotated training data is scarce or not immediately available. A typical approach to supervised learning on a new domain is a combination of the following ingredients: (1) a sufficiently good learned model initialization trained on a similar domain and (2) domain adaptation by fine-tuning the initial model for the target domain. In the field of semantic image segmentation, ImageNet initialization has become ubiquitous [1]. However, recently developed generative diffusion models set a new standard for synthesizing realistic images [2] and therefore are a good candidate to act as learned priors for downstream tasks, such as semantic segmentation [3]. In this project, the students will explore this avenue in multiple application contexts, including standard computer vision and remote sensing benchmarks. The project is suitable for bachelor, Master project, and Master Thesis. Group work is possible at Bachelor or Master project level.
The student is expected to (1) perform a review of the relevant literature and baseline solutions to the problem, (2) reproduce given code bases, (3) propose experiment objectives and code changes, and (4) organize findings in the final report. Stretch goals include strengthening the methodology of the approach (e.g., with domain adaptation techniques [4]) and submission to a top-tier conference.
Settings for applications
● Python, PyTorch, Linux shell;
● Knowledge of (deep) machine learning and computer vision/image analysis.
The student is expected to (1) perform a review of the relevant literature and baseline solutions to the problem, (2) reproduce given code bases, (3) propose experiment objectives and code changes, and (4) organize findings in the final report. Stretch goals include strengthening the methodology of the approach (e.g., with domain adaptation techniques [4]) and submission to a top-tier conference. Settings for applications ● Python, PyTorch, Linux shell; ● Knowledge of (deep) machine learning and computer vision/image analysis.
Anton Obukhov (anton.obukhov@geod.baug.ethz.ch), Photogrammetry and Remote Sensing, ETH Zürich
Anton Obukhov (anton.obukhov@geod.baug.ethz.ch), Photogrammetry and Remote Sensing, ETH Zürich