Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
3D Anatomy Reconstruction using a Magnetic Endoscope
We wish to reconstruct 3D meshes of internal human anatomy using videos obtained from a magnetically controlled endoscope.
Keywords: computer vision, SLAM, robotics, endoscopy, structure from motion, optimization, medical robotics
Simultaneous location and mapping (SLAM) and structure from motion (sfm) are two similar techniques from robotics and computer vision and can be used to simultaneously retrieve camera poses and reconstruct the 3D geometry of the observed scene.
Endoscopy, a minimally invasive surgery technique, uses cameras to inspect human anatomy from inside, or to assist surgery. Camera pose information is useful for augmented reality, for example superimposing preoperative images on the surgeon's camera view. Reconstructing 3D anatomy may also be helpful for diagnostics, or aiding the surgeon to position tools inside complicated anatomy.
In our lab, we have developed endoscopes that can wirelessly be controlled using magnetic fields. Therefore, these endoscopes are similar to a mobile robot with an onboard camera.
We wish to adapt existing tools from robotics and computer vision for such a task. In contrast to most robotics tasks however, the scene can experience large deformations, the image quality is reduced, and there may be undesirable artifacts such as specular reflection, smoke, or occlusion from surgical tools.
Simultaneous location and mapping (SLAM) and structure from motion (sfm) are two similar techniques from robotics and computer vision and can be used to simultaneously retrieve camera poses and reconstruct the 3D geometry of the observed scene.
Endoscopy, a minimally invasive surgery technique, uses cameras to inspect human anatomy from inside, or to assist surgery. Camera pose information is useful for augmented reality, for example superimposing preoperative images on the surgeon's camera view. Reconstructing 3D anatomy may also be helpful for diagnostics, or aiding the surgeon to position tools inside complicated anatomy.
In our lab, we have developed endoscopes that can wirelessly be controlled using magnetic fields. Therefore, these endoscopes are similar to a mobile robot with an onboard camera.
We wish to adapt existing tools from robotics and computer vision for such a task. In contrast to most robotics tasks however, the scene can experience large deformations, the image quality is reduced, and there may be undesirable artifacts such as specular reflection, smoke, or occlusion from surgical tools.
- Development of new magnetic endoscopic hardware. applicants should feel comfortable doing CAD design and some basic electronics.
- Adaptation of existing frameworks (OrbSLAM) for our task. Applicants should feel comfortable developing C++ software.
- Evaluation on ex-vivo anatomy models and possibly in-vivo experiments.
- Development of new magnetic endoscopic hardware. applicants should feel comfortable doing CAD design and some basic electronics. - Adaptation of existing frameworks (OrbSLAM) for our task. Applicants should feel comfortable developing C++ software. - Evaluation on ex-vivo anatomy models and possibly in-vivo experiments.