Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Controlling the Legs of a 12t Walking Excavator Using Reinforcement Learning (RL)
The goal of this project is to learn a control policy using RL that adjusts the height and the orientation of the cabin of a 12t walking excavator [1]. The policy will be trained in simulation and transferred to the machine for deployment (sim-to-real).
Compared to regular excavators, walking excavators are particularly hard to operate. Each leg features another 3 joints which have to be controlled individually. This makes it extremely difficult to keep all wheels in contact with the ground while evenly distributing the machine’s weight. To facilitate the operation of such machines, a Chassis Balancing Controller has been developed that optimizes the force distribution between the wheels given high-level inputs such as cabin orientation and height [2, 3]. However, this controller requires expensive high-performance hydraulic valves that have been retrofitted to our excavator.
The aim of this project is to leverage RL to deal with off-the-shelf, less performant, hydraulic valves. Therefore, the goal is to artificially degrade the performance of the high-performance valves installed on our machine by introducing delays and dead-zones and converging to the behavior of the standard valves step-by-step. Then, an RL policy which controls the four main leg joints is trained to deal with this degradation.
This project does not involve hardware modifications. The excavator is in a state, where a controller policy trained in simulation can be directly deployed.
[1] HEAP the Autonomous Walking excavator, Jud et al., unpublished
[2] Force Control for Active Chassis Balancing, Hutter et al., 2017
[3] https://www.youtube.com/watch?v=5_Eq8CxKkvM&list=PLE-BQwvVGf8Gs-ThVoEFlNDQ9kVne5qNQ&index=13
Compared to regular excavators, walking excavators are particularly hard to operate. Each leg features another 3 joints which have to be controlled individually. This makes it extremely difficult to keep all wheels in contact with the ground while evenly distributing the machine’s weight. To facilitate the operation of such machines, a Chassis Balancing Controller has been developed that optimizes the force distribution between the wheels given high-level inputs such as cabin orientation and height [2, 3]. However, this controller requires expensive high-performance hydraulic valves that have been retrofitted to our excavator.
The aim of this project is to leverage RL to deal with off-the-shelf, less performant, hydraulic valves. Therefore, the goal is to artificially degrade the performance of the high-performance valves installed on our machine by introducing delays and dead-zones and converging to the behavior of the standard valves step-by-step. Then, an RL policy which controls the four main leg joints is trained to deal with this degradation.
This project does not involve hardware modifications. The excavator is in a state, where a controller policy trained in simulation can be directly deployed.
[1] HEAP the Autonomous Walking excavator, Jud et al., unpublished
[2] Force Control for Active Chassis Balancing, Hutter et al., 2017
- High motivation and interest in the topic
- Structure, independent and goal oriented working behavior
- Experience with C++ and Python
- Experience with ROS, RL, Pytorch is a plus
- High motivation and interest in the topic - Structure, independent and goal oriented working behavior - Experience with C++ and Python - Experience with ROS, RL, Pytorch is a plus
