Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Guided Spatial Mapping for Mixed Reality Scenarios
In this project, we want to investigate how we can actively guide a user to improve spatial mapping of the environment in mixed reality scenarios through feedback on the quality of the map. The quality of the map will be assessed based on uncertainties in the reconstruction. Besides the guidance through the quantitative metric, the mixed reality interface is expected to enable the user to intuitively assess the quality of the reconstructed mesh on overlay.
Spatial mapping of the environment is an integral part of mixed reality (MR) scenarios. The reconstruction of the environment creates the necessary foundation to naturally fuse the real and the virtual world through proper placement of virtual holograms. Inadequate spatial mapping can lead to offsets or intersections of the holograms and the real world which disturb the user experience. As the spatial map is built on-the-fly through the sensors in the MR interface (Microsoft HoloLens 2), the quality of the reconstruction is tightly coupled with the user’s movement and gaze directions.
In this project, we want to investigate how we can actively guide the user to improve spatial mapping of the environment through feedback on the status of the map. The status of the map will be assessed based on uncertainties in the reconstruction (uncertainty-aware mapping framework will be provided). Besides the guidance through the quantitative metric, the mixed reality interface is expected to enable the user to intuitively assess the quality of the reconstructed mesh on overlay.
References:
[1] https://docs.microsoft.com/en-us/windows/mixed-reality/design/spatial-mapping
Spatial mapping of the environment is an integral part of mixed reality (MR) scenarios. The reconstruction of the environment creates the necessary foundation to naturally fuse the real and the virtual world through proper placement of virtual holograms. Inadequate spatial mapping can lead to offsets or intersections of the holograms and the real world which disturb the user experience. As the spatial map is built on-the-fly through the sensors in the MR interface (Microsoft HoloLens 2), the quality of the reconstruction is tightly coupled with the user’s movement and gaze directions.
In this project, we want to investigate how we can actively guide the user to improve spatial mapping of the environment through feedback on the status of the map. The status of the map will be assessed based on uncertainties in the reconstruction (uncertainty-aware mapping framework will be provided). Besides the guidance through the quantitative metric, the mixed reality interface is expected to enable the user to intuitively assess the quality of the reconstructed mesh on overlay.
- Literature review
- Integration of uncertainty-aware 2D spatial mapping with the MR interface.
- Design of a suitable visualization strategy of uncertainties to the operator.
- Extensive evaluation of the proposed strategy.
- (Optional) Extension to active user requests (map resolution) or corrections in the map.
- Literature review - Integration of uncertainty-aware 2D spatial mapping with the MR interface. - Design of a suitable visualization strategy of uncertainties to the operator. - Extensive evaluation of the proposed strategy. - (Optional) Extension to active user requests (map resolution) or corrections in the map.
- High motivation and interest in the topic.
- Methodological and goal-oriented approach.
- Programming skills (C++/C#).
- Prior experience with Unity and/or MR is a plus.
- High motivation and interest in the topic. - Methodological and goal-oriented approach. - Programming skills (C++/C#). - Prior experience with Unity and/or MR is a plus.
Mike Allenspach (mike.allenspach@mavt.ethz.ch)
Antonia Hüfner (ahuefner@ethz.ch)
Mike Allenspach (mike.allenspach@mavt.ethz.ch) Antonia Hüfner (ahuefner@ethz.ch)
fs22-MR-mapping.pdf
