Project 1: Deep Learning for Image-Based Robot Control from Human Demonstrations
The power of deep learning has enabled robots to interact with the world directly from raw images, by mapping pixels to motor actions in an end-to-end manner. Recent works have shown this to be successful for tasks such as picking up objects, operating a hammer, and folding a towel (see video on right). The method uses imitation learning: image-action pairs are collected during demonstrations, which then form a dataset for training a convolutional neural network. However, this requires a very large number of demonstrations to enable the robot to generalise to small changes in the environment, such as object positions and illumination effects.
In this project, you will study automatic control of robot arms using the above method, but with a difference. Rather than controlling the robot in an end-to-end manner with a single neural network, two different neural networks will be trained. The first will be trained to detect an object and estimate its pose, and the second will be trained to control the robot based on this object pose. As such, the overall pipeline is broken down into a modular approach, whilst still retaining the power of deep learning. You will explore whether this is able to train a robot with fewer demonstrations than the end-to-end approach. Experiments will initially be conducted in simulation, and you will be able to evaluate your method on the robot in our lab, on a range of different tasks.
Click here and here to read some related papers.