Register now After registration you will be able to apply for this opportunity online.
3D reconstruction of zebrafish larvae based on acoustic rotating
The project is a collaboration between ARSL and CVL. For acoustics, Prof. Daniel will guide you and for AI, Prof. Fisher Yu will guide you. We plan to develop a special 3D reconstruction algorithm for zebrafish larvae.
In this project, we first perform the rotation manipulation of zebrafish using an acoustically actuated capillary. Then, we would like to realize the precise 3D reconstruction of the in vivo organs of live zebrafish larvae using CV and AI algorithms. We will fabricate a microchannel chip, which can develop a single polarized vortex. By adjusting the acoustic excitation parameters, we will change the rotational speed and direction. Finally, we will program our special 3D reconstruction algorithms and software.
Keywords: 3D reconstruction; AI; CV;Rotation; Micromanipulation; Acoustics;
Currently, on-chip rotation of organisms has played a crucial role in multidisciplinary applications in developmental biology, medicine, and life sciences. This work is a clever demonstration of a novel platform to perform on-chip rotation of zebrafish larvae. Zebrafish are an excellent alternative to higher mammalian models, which are widely used in the development of drugs, toxicity studies, cancer diagnosis, and understanding of human diseases. Therefore, this work is important fundamental research in the community of microrobotic manipulation, which will open the way for many following studies.
As shown in Fig.1 our platform is distinct from reported rotation demonstrations with multiple small bubble arrays which have dispersed reaction forces, and complex flow patterns, and cannot promise the synchronized resonance frequency among bubbles. On the other hand, compared to micromanipulation using optical, magnetic, electric fields, and conventional methods based on tweezer or agar. Our acoustic rotational platform is bio-compatible, easy-to-fabricate, and cost-effective.
To further demonstrate the potential of our acoustofluidic device for zebrafish-based biological applications, we performed rotation manipulation on three types of fluorescently labeled zebrafish larvae. As shown in Fig. 2a, through rotating manipulation, we visualized the distribution of blood vessels through the kdrl:GFP transgenic zebrafish larva from multiple view angles. Fig. 2b shows the prox1a>Lyn-Citrine; cmlc2:GFP transgenic zebrafish larva with its heart, eyes, brain, and spinal chord fluorescently labeled. During rotation, we can observe the entire profile of the heart and its beating. Notably, also distinct neuronal subpopulations in the retina are visualizable. Fig. 2c shows the rotational image sequences of the fluorescently labeled eyes of an alpha-crystallin:YFP transgenic zebrafish larva.
The controllable out-of-plane rotation of zebrafish larvae empowers exciting 3D reconstruction from the captured multi-view 2D image sequences. We first took images of the zebrafish larva from different view angles during several rotation cycles. Imaging preprocesses, including removing noise, enhancing contrast, and adjusting brightness and sharpness were performed to improve the image quality. Subsequently, a feature-based alignment algorithm was used to align images by identifying common features, such as corners and edges. The aligned images are then used to reconstruct a point cloud of the larva using a process called stereo matching. The point cloud is further converted into a mesh by connecting the points in a triangular pattern. By averaging the positions of the points in the mesh, the mesh is smoothed to remove any sharp edges or artifacts. A hole-filling process is then used to fill the holes in the mesh. Fig. 3 shows such an example rendering from the 3D mesh with a visualized ellipsoidal swim bladder inside the larva. 3D reconstruction of other internal organs (heart, liver, bone, etc.) of zebrafish larvae is challenging and suffers from problems such as being off the axis of rotation, small size, occluded by other objects, and out of focus at certain viewing angles. Increasing the camera's frame rate, reducing its rotation speed, and optimizing the geometric parameters of the channel can reveal more detailed information about the larvae.
Currently, on-chip rotation of organisms has played a crucial role in multidisciplinary applications in developmental biology, medicine, and life sciences. This work is a clever demonstration of a novel platform to perform on-chip rotation of zebrafish larvae. Zebrafish are an excellent alternative to higher mammalian models, which are widely used in the development of drugs, toxicity studies, cancer diagnosis, and understanding of human diseases. Therefore, this work is important fundamental research in the community of microrobotic manipulation, which will open the way for many following studies.
As shown in Fig.1 our platform is distinct from reported rotation demonstrations with multiple small bubble arrays which have dispersed reaction forces, and complex flow patterns, and cannot promise the synchronized resonance frequency among bubbles. On the other hand, compared to micromanipulation using optical, magnetic, electric fields, and conventional methods based on tweezer or agar. Our acoustic rotational platform is bio-compatible, easy-to-fabricate, and cost-effective.
To further demonstrate the potential of our acoustofluidic device for zebrafish-based biological applications, we performed rotation manipulation on three types of fluorescently labeled zebrafish larvae. As shown in Fig. 2a, through rotating manipulation, we visualized the distribution of blood vessels through the kdrl:GFP transgenic zebrafish larva from multiple view angles. Fig. 2b shows the prox1a>Lyn-Citrine; cmlc2:GFP transgenic zebrafish larva with its heart, eyes, brain, and spinal chord fluorescently labeled. During rotation, we can observe the entire profile of the heart and its beating. Notably, also distinct neuronal subpopulations in the retina are visualizable. Fig. 2c shows the rotational image sequences of the fluorescently labeled eyes of an alpha-crystallin:YFP transgenic zebrafish larva.
The controllable out-of-plane rotation of zebrafish larvae empowers exciting 3D reconstruction from the captured multi-view 2D image sequences. We first took images of the zebrafish larva from different view angles during several rotation cycles. Imaging preprocesses, including removing noise, enhancing contrast, and adjusting brightness and sharpness were performed to improve the image quality. Subsequently, a feature-based alignment algorithm was used to align images by identifying common features, such as corners and edges. The aligned images are then used to reconstruct a point cloud of the larva using a process called stereo matching. The point cloud is further converted into a mesh by connecting the points in a triangular pattern. By averaging the positions of the points in the mesh, the mesh is smoothed to remove any sharp edges or artifacts. A hole-filling process is then used to fill the holes in the mesh. Fig. 3 shows such an example rendering from the 3D mesh with a visualized ellipsoidal swim bladder inside the larva. 3D reconstruction of other internal organs (heart, liver, bone, etc.) of zebrafish larvae is challenging and suffers from problems such as being off the axis of rotation, small size, occluded by other objects, and out of focus at certain viewing angles. Increasing the camera's frame rate, reducing its rotation speed, and optimizing the geometric parameters of the channel can reveal more detailed information about the larvae.
The project is a collaboration between ARSL and CVL. For acoustics, Prof. Daniel will guide you and for AI, Prof. Fisher Yu will guide you. We plan to develop a special 3D reconstruction algorithm for zebrafish larvae.
The project includes the following tasks:
1. Set up the acoustic excitation system.
2. Realize the rotation of zebrafish.
3. Develop the 3D reconstruction algorithm with AI techniques.
4. Realize the 3D reconstruction of zebrafish where the organs in the body can be observed.
Your background:
1. 3D reconstruction;
2. Computer version;
3. Python.
The project is a collaboration between ARSL and CVL. For acoustics, Prof. Daniel will guide you and for AI, Prof. Fisher Yu will guide you. We plan to develop a special 3D reconstruction algorithm for zebrafish larvae.
The project includes the following tasks: 1. Set up the acoustic excitation system. 2. Realize the rotation of zebrafish.
3. Develop the 3D reconstruction algorithm with AI techniques. 4. Realize the 3D reconstruction of zebrafish where the organs in the body can be observed.
Your background: 1. 3D reconstruction; 2. Computer version; 3. Python.
Please send your CV and transcript of records to Prof. Daniel Ahmed: dahmed@ethz.ch, Prog. Fisher Yu:fisheryu@ethz.ch, and Zhiyuan Zhang: zhiyzhang@ethz.ch
Please send your CV and transcript of records to Prof. Daniel Ahmed: dahmed@ethz.ch, Prog. Fisher Yu:fisheryu@ethz.ch, and Zhiyuan Zhang: zhiyzhang@ethz.ch