Register now After registration you will be able to apply for this opportunity online.
Crowd Navigation using Reinforcement Learning
This project aims to address the challenges faced by planners in dynamic environments with fast-moving crowds. It proposes a new planner utilizing 3D sensing and Reinforcement Learning to understand human movement patterns, with the core objective of developing a reliable 3D sensor encoding integrated into a novel planner architecture within the Orbit Framework, specifically designed for crowded environments.
Keywords: Reinforcement Learning, 3D Sensors, LiDAR, Planning, Navigation, Isaac Sim, Orbit
This project seeks to overcome the limitations faced by planners in dynamic environments, particularly those with fast-moving crowds. While recent advancements in end-to-end trained local planners have demonstrated success in demanding scenarios, their primary focus is on static scenes. However, the need for high update frequencies in dynamic environments remains a significant challenge [3,4]. To address these issues, we propose developing a new planner using 3D sensing that is able to reason about human movement patterns. For training, we aim to leverage the latest developments in Reinforcement Learning (RL) [1,2], known for its efficacy in locomotion tasks. In such an approach, the planner will learn from experience how humans move and the best way to plan a path around them. The core objective of this project is the development of the 3D sensor encoding that is fed into a novel planner architecture, all trained within the Orbit Framework [5] and specifically designed for reliability in crowded environments.
References:
[1] Nikita Rudin, David Hoeller, Philipp Reist, Marco Hutter, Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning, Proceedings of the 5th Conference on Robot Learning, PMLR 164:91-100, 2022.
[2] Miki, T., Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V. and Hutter, M., 2022. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7
[3] Roth, P., Nubert, J., Yang, F., Mittal, M. and Hutter, M., 2023. ViPlanner: Visual Semantic Imperative Learning for Local Navigation. arXiv preprint arXiv:2310.00982.
[4] Yang, F., Wang, C., Cadena, C. and Hutter, M., 2023. iPlanner: Imperative Path Planning. arXiv preprint arXiv:2302.11434.
[5] Mittal, M., Yu, C., Yu, Q., Liu, J., Rudin, N., Hoeller, D., Yuan, J.L., Singh, R., Guo, Y., Mazhar, H. and Mandlekar, A., 2023. Orbit: A unified simulation framework for interactive robot learning environments. IEEE Robotics and Automation Letters.
This project seeks to overcome the limitations faced by planners in dynamic environments, particularly those with fast-moving crowds. While recent advancements in end-to-end trained local planners have demonstrated success in demanding scenarios, their primary focus is on static scenes. However, the need for high update frequencies in dynamic environments remains a significant challenge [3,4]. To address these issues, we propose developing a new planner using 3D sensing that is able to reason about human movement patterns. For training, we aim to leverage the latest developments in Reinforcement Learning (RL) [1,2], known for its efficacy in locomotion tasks. In such an approach, the planner will learn from experience how humans move and the best way to plan a path around them. The core objective of this project is the development of the 3D sensor encoding that is fed into a novel planner architecture, all trained within the Orbit Framework [5] and specifically designed for reliability in crowded environments.
References:
[1] Nikita Rudin, David Hoeller, Philipp Reist, Marco Hutter, Learning to Walk in Minutes Using Massively Parallel Deep Reinforcement Learning, Proceedings of the 5th Conference on Robot Learning, PMLR 164:91-100, 2022.
[2] Miki, T., Lee, J., Hwangbo, J., Wellhausen, L., Koltun, V. and Hutter, M., 2022. Learning robust perceptive locomotion for quadrupedal robots in the wild. Science Robotics, 7
[3] Roth, P., Nubert, J., Yang, F., Mittal, M. and Hutter, M., 2023. ViPlanner: Visual Semantic Imperative Learning for Local Navigation. arXiv preprint arXiv:2310.00982.
[4] Yang, F., Wang, C., Cadena, C. and Hutter, M., 2023. iPlanner: Imperative Path Planning. arXiv preprint arXiv:2302.11434.
[5] Mittal, M., Yu, C., Yu, Q., Liu, J., Rudin, N., Hoeller, D., Yuan, J.L., Singh, R., Guo, Y., Mazhar, H. and Mandlekar, A., 2023. Orbit: A unified simulation framework for interactive robot learning environments. IEEE Robotics and Automation Letters.
Literature review on local planning algorithms and training strategies
Implementation of a novel planner with 3D Sensor encoding
Integration with ANYmal planning and navigation framework for fast local planning
Real-world evaluation of trained planners
Literature review on local planning algorithms and training strategies
Implementation of a novel planner with 3D Sensor encoding
Integration with ANYmal planning and navigation framework for fast local planning
Real-world evaluation of trained planners
Highly motivated for the topic
Programming experience (C++/Python)
Experience with deep learning projects (preferably with RL)
Knowledge of planning, perception, and robot dynamics is a plus
Highly motivated for the topic
Programming experience (C++/Python)
Experience with deep learning projects (preferably with RL)
Knowledge of planning, perception, and robot dynamics is a plus
Please send your CV, TOR and a short motivational statement to Fan Yang (fanyang1@ethz.ch), Takahiro Miki (tamiki@ethz.ch) and Pascal Roth (rothpa@ethz.ch)
Please send your CV, TOR and a short motivational statement to Fan Yang (fanyang1@ethz.ch), Takahiro Miki (tamiki@ethz.ch) and Pascal Roth (rothpa@ethz.ch)
MA_crowd_navigation.pdf
