Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Learning to paint with an autonomous spray painting UAV
PaintCopter [1] is an autonomous UAV capable of spray painting on complex 3D
surfaces. Currently the system is capable of painting basic elements such as lines, gradients and
area-fill with a color of choice (see Fig.b). The motivation for this project is to find a way to
decompose a provided textur
Interpreting an image as an end-result of a generative process is a well-known subject in computer
vision/graphics (ref. inverse-rendering). Humans have a very mature understanding of how to
interpret observations of the real-world. The goal of this project is to enable a computer with such
capabilities in creating rich representations of the world, for the chosen task of painting on 3D
surfaces.
Painting is a complex process of applying several strokes with different tools to a canvas in order to
generate a desired appearance. In this context, the goal of an inverse-rendering problem is that of
going from a provided image/texture to the process that can achieve it.
Several works [2,3,4,5] exist that approach such problems using recurrent networks, reinforcement
learning and imitation learning. However, these works are suited for 2D drawing/paintings and
operate on a small scale. Moreover, most of the works require extensive training data which is
expensive to collect for such a complex problem. So, the big questions to answer as a part of this
thesis is to choose an appropriate direction such that, with minimal supervision, it can be made to
work in 3D. Scaling to larger scenes would be preferable, though not crucial.
Interpreting an image as an end-result of a generative process is a well-known subject in computer vision/graphics (ref. inverse-rendering). Humans have a very mature understanding of how to interpret observations of the real-world. The goal of this project is to enable a computer with such capabilities in creating rich representations of the world, for the chosen task of painting on 3D surfaces. Painting is a complex process of applying several strokes with different tools to a canvas in order to generate a desired appearance. In this context, the goal of an inverse-rendering problem is that of going from a provided image/texture to the process that can achieve it. Several works [2,3,4,5] exist that approach such problems using recurrent networks, reinforcement learning and imitation learning. However, these works are suited for 2D drawing/paintings and operate on a small scale. Moreover, most of the works require extensive training data which is expensive to collect for such a complex problem. So, the big questions to answer as a part of this thesis is to choose an appropriate direction such that, with minimal supervision, it can be made to work in 3D. Scaling to larger scenes would be preferable, though not crucial.
The thesis involves following work packages:
1. Research the state-of-the-art in inverse-graphics/rendering and learning paradigms such as
recurrent networks, reinforcement leaning, imitation learning etc.
2. Try some open-source implementations
3. Familiarize with an existing simulator setup that is needed for testing various approaches
4. Implement an appropriate strategy to paint a provided texture on a 3D surface
5. Compare the quality of the end-result
The thesis involves following work packages: 1. Research the state-of-the-art in inverse-graphics/rendering and learning paradigms such as recurrent networks, reinforcement leaning, imitation learning etc. 2. Try some open-source implementations 3. Familiarize with an existing simulator setup that is needed for testing various approaches 4. Implement an appropriate strategy to paint a provided texture on a 3D surface 5. Compare the quality of the end-result
Tensorflow or related software
C/C++ (preferably)
Tensorflow or related software C/C++ (preferably)
Anurag Sai Vempati – avempati@ethz.ch
Cesar Cadena - cesarc@ethz.ch
Jen Jen Chung - jenjen.chung@mavt.ethz.ch
Anurag Sai Vempati – avempati@ethz.ch Cesar Cadena - cesarc@ethz.ch Jen Jen Chung - jenjen.chung@mavt.ethz.ch