Register now After registration you will be able to apply for this opportunity online.
Reinforcement Learning for Excavation Planning
We aim to develop a reinforcement learning-based global excavation planner that can plan for the long term and execute a wide range of excavation geometries. The system will be deployed on our legged excavator.
Reinforcement learning has demonstrated significant success in decision-making and behavior planning with discrete states and action spaces. In this project, we plan to develop and extend a global excavation planner responsible for selecting the next digging area and the actions required to move soil around the site. This requires long-term planning for the sequence of excavation and an understanding of which areas are accessible and where the excavator could potentially become trapped.
We developed using Jax a simulation environment, Terra [3], where agents can be trained even in millions of parallel environments on multiple GPUs. The first part of the project will focus on modifying the simulation environment to have a continous state space and include 3D soil to simulate are real contruction site. The second part of the project will focus on deploying the system and integrating it with the current stack to dig geometries that were not achieved so far.
References:
[1] PPO for LUX AI
[2] Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, Deepmind
[3] Terra: https://github.com/leggedrobotics/terra
Reinforcement learning has demonstrated significant success in decision-making and behavior planning with discrete states and action spaces. In this project, we plan to develop and extend a global excavation planner responsible for selecting the next digging area and the actions required to move soil around the site. This requires long-term planning for the sequence of excavation and an understanding of which areas are accessible and where the excavator could potentially become trapped. We developed using Jax a simulation environment, Terra [3], where agents can be trained even in millions of parallel environments on multiple GPUs. The first part of the project will focus on modifying the simulation environment to have a continous state space and include 3D soil to simulate are real contruction site. The second part of the project will focus on deploying the system and integrating it with the current stack to dig geometries that were not achieved so far.
References: [1] PPO for LUX AI [2] Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, Deepmind [3] Terra: https://github.com/leggedrobotics/terra
Reinforcement learning has shown remarkable success in decision-making and behavior planning within environments characterized by discrete states and actions. In this project, we aim to develop and enhance a global excavation planner that is tasked with selecting the next excavation site and coordinating the necessary actions to redistribute soil across the site. This involves intricate long-term planning to sequence excavation activities, while also considering accessibility and potential areas where the excavator might become trapped.
We have developed a simulation environment called Terra [3], using Jax, which supports training agents in millions of parallel environments across multiple GPUs. The initial phase of the project will involve adapting the simulation environment to support a continuous state space and incorporate 3D soil modeling to more accurately mimic a real construction site. The second phase will focus on deploying the system and integrating it with existing technology stacks to achieve excavation geometries previously unattainable.
References:
[1] PPO for LUX AI
[2] Mastering Atari, Go, Chess, and Shogi by Planning with a Learned Model, Deepmind
[3] Terra: https://github.com/leggedrobotics/terra
Reinforcement learning has shown remarkable success in decision-making and behavior planning within environments characterized by discrete states and actions. In this project, we aim to develop and enhance a global excavation planner that is tasked with selecting the next excavation site and coordinating the necessary actions to redistribute soil across the site. This involves intricate long-term planning to sequence excavation activities, while also considering accessibility and potential areas where the excavator might become trapped.
We have developed a simulation environment called Terra [3], using Jax, which supports training agents in millions of parallel environments across multiple GPUs. The initial phase of the project will involve adapting the simulation environment to support a continuous state space and incorporate 3D soil modeling to more accurately mimic a real construction site. The second phase will focus on deploying the system and integrating it with existing technology stacks to achieve excavation geometries previously unattainable.
References: [1] PPO for LUX AI [2] Mastering Atari, Go, Chess, and Shogi by Planning with a Learned Model, Deepmind [3] Terra: https://github.com/leggedrobotics/terra
- Experience in PyTorch and training neural networks
- Experience with GPU-accelerated environments (preferred)
- Experience in PyTorch and training neural networks - Experience with GPU-accelerated environments (preferred)
