Register now After registration you will be able to apply for this opportunity online.
Reinforcement Learning for Excavation Planning
We aim to develop a reinforcement learning-based global excavation planner that can plan for the long term and execute a wide range of excavation geometries. The system will be deployed on our legged excavator.
Reinforcement learning has demonstrated significant success in decision-making and behavior planning with discrete states and action spaces. In this project, we plan to develop and extend a global excavation planner responsible for selecting the next digging area and the actions required to move soil around the site. This requires long-term planning for the sequence of excavation and an understanding of which areas are accessible and where the excavator could potentially become trapped.
We developed using Jax a simulation environment, Terra, where agents can be trained even in millions of parallel environments on multiple GPUs. The first part of the project will focus on modifying the simulation environment to include 3D soil and multiple agents to simulate are real contruction site. The second part of the project will focus on deploying the system and integrating it with the current stack to dig geometries that were not achieved so far.
References:
[1] PPO for LUX AI
[2] Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, Deepmind
[3] IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures, Deepmind
Reinforcement learning has demonstrated significant success in decision-making and behavior planning with discrete states and action spaces. In this project, we plan to develop and extend a global excavation planner responsible for selecting the next digging area and the actions required to move soil around the site. This requires long-term planning for the sequence of excavation and an understanding of which areas are accessible and where the excavator could potentially become trapped. We developed using Jax a simulation environment, Terra, where agents can be trained even in millions of parallel environments on multiple GPUs. The first part of the project will focus on modifying the simulation environment to include 3D soil and multiple agents to simulate are real contruction site. The second part of the project will focus on deploying the system and integrating it with the current stack to dig geometries that were not achieved so far.
References: [1] PPO for LUX AI [2] Mastering Atari, Go, Chess and Shogi by Planning with a Learned Model, Deepmind [3] IMPALA: Scalable Distributed Deep-RL with Importance Weighted Actor-Learner Architectures, Deepmind
- Expand the Terra env to include 3D soil and/or multiple agents (such as trucks, wheel loaders, ...)
- Train the agent(s), design curriculum and architecture
- Deploy the system on realistic task
- Expand the Terra env to include 3D soil and/or multiple agents (such as trucks, wheel loaders, ...) - Train the agent(s), design curriculum and architecture - Deploy the system on realistic task
- Experience in PyTorch and training neural networks
- Experience with GPU-accelerated environments (preferred)
- Experience in PyTorch and training neural networks - Experience with GPU-accelerated environments (preferred)
Reinforcement_learning_for_excavation_planning.docx
