Register now Registered users can see all details and apply online for the opportunities.
Convolutional neural networks for billiard state estimation
This project will use deep learning to recognise balls on a billiard table as part of the Automatic Control Lab's "DeepGreen" snooker robot project. We are building a robot able to take human-level accuracy shots, and this project will improve this accuracy further using reliable computer vision.
Keywords: Computer vision, neural networks, machine learning, deep learning, robotics
The DeepGreen snooker robot project aims to build a robot capable of challenging the best human players. Snooker (a billiard game played on a much larger 1.8 x 3.8m table) is challenging because of the very long shots and tight pockets, and because of the complex strategies used by players to block their opponents.
The robot consists of a robot arm with linear-motor cueing action, supported by a ceiling camera and a cue-mounted camera. It is already capable of taking near-human-level accurate shots, but would be further improved by more reliable ball detection by the cameras. A video of the robot in action can be found here:
https://vimeo.com/335260829
We currently use a range of filters, e.g. blob detection and colour thresholding, to detect balls from the two cameras and classify their colours (red, white, black, etc.). We wish to extend the vision algorithm to include a neural network (NN) able to detect, as NNs are well suited to the detection of balls at different distances and under different lighting conditions. The new vision system will form part of the automated pipeline for carrying out robot shots.
The DeepGreen snooker robot project aims to build a robot capable of challenging the best human players. Snooker (a billiard game played on a much larger 1.8 x 3.8m table) is challenging because of the very long shots and tight pockets, and because of the complex strategies used by players to block their opponents.
The robot consists of a robot arm with linear-motor cueing action, supported by a ceiling camera and a cue-mounted camera. It is already capable of taking near-human-level accurate shots, but would be further improved by more reliable ball detection by the cameras. A video of the robot in action can be found here:
https://vimeo.com/335260829
We currently use a range of filters, e.g. blob detection and colour thresholding, to detect balls from the two cameras and classify their colours (red, white, black, etc.). We wish to extend the vision algorithm to include a neural network (NN) able to detect, as NNs are well suited to the detection of balls at different distances and under different lighting conditions. The new vision system will form part of the automated pipeline for carrying out robot shots.
The project will proceed according to the following steps:
1) Evaluate the reliability of the existing detection algorithms.
2) Choose a NN architecture for the new detector, and specify how many training images may be required.
3) Gather an appropriate number of training images, and classify the balls in the images. Potentially outsource this task using e.g. Amazon Mechanical Turk.
4) Train a NN (possibly including pre-trained features relevant to ball detection) to recognise billiard balls.
5) Determine a suitable approach to classifying balls by colour in a reliable manner.
6) Adjust the real-time image filter in the robot to make sure ball detection is robust over successive frames gathered by the camera.
The project will proceed according to the following steps:
1) Evaluate the reliability of the existing detection algorithms.
2) Choose a NN architecture for the new detector, and specify how many training images may be required.
3) Gather an appropriate number of training images, and classify the balls in the images. Potentially outsource this task using e.g. Amazon Mechanical Turk.
4) Train a NN (possibly including pre-trained features relevant to ball detection) to recognise billiard balls.
5) Determine a suitable approach to classifying balls by colour in a reliable manner.
6) Adjust the real-time image filter in the robot to make sure ball detection is robust over successive frames gathered by the camera.
Joe Warrington (warrington@control.ee.ethz.ch)
Nikos Kariotoglou (nkarioto@control.ee.ethz.ch)
Joe Warrington (warrington@control.ee.ethz.ch)
Nikos Kariotoglou (nkarioto@control.ee.ethz.ch)
