Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Multi-View 6DoF Object Pose Estimation on HoloLens
The goal of this project is to implement an 6DoF object pose estimation method that utilizes the embedded sensors of head-mounted devices like the Microsoft HoloLens to improve the accuracy of the 6DoF pose estimation. The proposed method will be thoroughly evaluated and compared against single-view, stereo, and multi-view baselines.
Accurately estimating the 6DoF pose of surgical instruments is an active question in clinical research. The pose of surgical instruments is highly relevant for variety of applications, ranging from surgical training simulators to navigation systems in precision surgery. Hereby, the egocentric view has shown great potential due to the proximity of the camera to the instruments in use, which minimizes occlusions.
Single-view pose estimation methods generally suffer from depth ambiguities. In contrast, stereo or multi-view methods can solve these depth ambiguities by implicit or explicit triangulation. Recent HMDs are usually equipped with multiple RGB, grayscale, or IR cameras to sense their environment.
The goal of this project is to implement a object pose estimation method that utilizes multiple or all of these available sensors to improve the accuracy of the 6DoF pose estimation. Ideally, single-view [1], stereo, and multi-view [2] methods are compared. A synthetic and real training dataset including camera parameters is provided [3].
[1] Haugaard, R. L., & Buch, A. G. (2022). Surfemb: Dense and continuous correspondence distributions for object pose estimation with learnt surface embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6749-6758).
[2] Haugaard, R. L., & Iversen, T. M. (2023, May). Multi-view object pose estimation from correspondence distributions and epipolar geometry. In 2023 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1786-1792). IEEE.
[3] Hein, J., Cavalcanti, N., Suter, D., Zingg, L., Carrillo, F., Farshad, M., ... & Fürnstahl, P. (2023). Next-generation Surgical Navigation: Multi-view Marker-less 6DoF Pose Estimation of Surgical Instruments. arXiv preprint arXiv:2305.03535.
Accurately estimating the 6DoF pose of surgical instruments is an active question in clinical research. The pose of surgical instruments is highly relevant for variety of applications, ranging from surgical training simulators to navigation systems in precision surgery. Hereby, the egocentric view has shown great potential due to the proximity of the camera to the instruments in use, which minimizes occlusions.
Single-view pose estimation methods generally suffer from depth ambiguities. In contrast, stereo or multi-view methods can solve these depth ambiguities by implicit or explicit triangulation. Recent HMDs are usually equipped with multiple RGB, grayscale, or IR cameras to sense their environment. The goal of this project is to implement a object pose estimation method that utilizes multiple or all of these available sensors to improve the accuracy of the 6DoF pose estimation. Ideally, single-view [1], stereo, and multi-view [2] methods are compared. A synthetic and real training dataset including camera parameters is provided [3].
[1] Haugaard, R. L., & Buch, A. G. (2022). Surfemb: Dense and continuous correspondence distributions for object pose estimation with learnt surface embeddings. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (pp. 6749-6758).
[2] Haugaard, R. L., & Iversen, T. M. (2023, May). Multi-view object pose estimation from correspondence distributions and epipolar geometry. In 2023 IEEE International Conference on Robotics and Automation (ICRA) (pp. 1786-1792). IEEE.
[3] Hein, J., Cavalcanti, N., Suter, D., Zingg, L., Carrillo, F., Farshad, M., ... & Fürnstahl, P. (2023). Next-generation Surgical Navigation: Multi-view Marker-less 6DoF Pose Estimation of Surgical Instruments. arXiv preprint arXiv:2305.03535.
Develop a multi-view 6DoF object pose estimation method for HMDs.
Develop a multi-view 6DoF object pose estimation method for HMDs.
