Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Optical Marker Denoising with Temporal Models
We want to explore temporal neural networks for the task of denoising corrupt marker locations which often happens in state-of-the-art optical motion capturing systems.
Optical marker tracking is the industry standard when it comes to high-quality motion capturing for movie and video game productions. Such systems work by tracking reflective markers attached to a suit that the actors are wearing. Typically, actors wear more than 40 markers.
Despite high tracking accuracy, the reconstructed poses still have to be cleaned up. This is because the system sometimes loses track of markers due to occlusions, markers fall of during capture, or the system mislabels nearby markers. Hence, for industry-grade quality, a tremendous amount of manual cleanup is performed.
Deep Learning methods hold great promise in automatizing some of these cleanup tasks. Recently, [1] proposed a neural network that denoises corrupt marker positions demonstrating impressive results. However, they report that temporal models were less successful for the task at hand and they compensate for this by post-processing the network’s outputs (i.e. smoothing and retargeting).
Optical marker tracking is the industry standard when it comes to high-quality motion capturing for movie and video game productions. Such systems work by tracking reflective markers attached to a suit that the actors are wearing. Typically, actors wear more than 40 markers.
Despite high tracking accuracy, the reconstructed poses still have to be cleaned up. This is because the system sometimes loses track of markers due to occlusions, markers fall of during capture, or the system mislabels nearby markers. Hence, for industry-grade quality, a tremendous amount of manual cleanup is performed.
Deep Learning methods hold great promise in automatizing some of these cleanup tasks. Recently, [1] proposed a neural network that denoises corrupt marker positions demonstrating impressive results. However, they report that temporal models were less successful for the task at hand and they compensate for this by post-processing the network’s outputs (i.e. smoothing and retargeting).
In this thesis, we want to explore temporal models for task presented in [1] with the aim to replace the post-processing steps mentioned above. The work can be split in two parts: 1) reproducing the results from [1] 2) Exploring temporal models resulting in similar or higher accuracy and producing visually pleasing results without requiring post-processing steps. Experience with deep learning frameworks (pytorch or tensorflow) is helpful. Some starter code will be made available.
In this thesis, we want to explore temporal models for task presented in [1] with the aim to replace the post-processing steps mentioned above. The work can be split in two parts: 1) reproducing the results from [1] 2) Exploring temporal models resulting in similar or higher accuracy and producing visually pleasing results without requiring post-processing steps. Experience with deep learning frameworks (pytorch or tensorflow) is helpful. Some starter code will be made available.