Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Online Safe Locomotion Learning in the Wild
Reinforcement learning (RL) can potentially solve complex problems in a purely data-driven manner. Still, the state-of-the-art in applying RL in robotics, relies heavily on high-fidelity simulators. While learning in simulation allows to circumvent sample complexity challenges that are common in model-free RL, even slight distribution shift ("sim-to-real gap") between simulation and the real system can cause these algorithms to easily fail. Recent advances in model-based reinforcement learning have led to superior sample efficiency, enabling online learning without a simulator. Nonetheless, learning online cannot cause any damage and should adhere to safety requirements (for obvious reasons). The proposed project aims to demonstrate how existing safe model-based RL methods can be used to solve the foregoing challenges.
The project aims to answer the following research questions:
How to model safe locomotion tasks for a real robotic system as a constrained RL problem?
Can we use existing methods such as the one proposed by @as2022constrained to safely learn effective locomotion policies?
Answering the above questions will encompass hands-on experience with a real robotic system (such as ANYmal) together with learning to implement and test cutting-edge RL methods. As RL on real hardware is not yet fully explored, we expect to unearth various challenges concerning the effectiveness of our methods in the online learning setting. Accordingly, an equally important goal of the project is to accurately identify these challenges and propose methodological improvements that can help address them.
A starting point would be to create a model of a typical locomotion task in Isaac Orbit as a proof-of-concept. Following that, the second part of the project will be dedicated to extending the proof-of-concept to a real system.
The project aims to answer the following research questions:
How to model safe locomotion tasks for a real robotic system as a constrained RL problem? Can we use existing methods such as the one proposed by @as2022constrained to safely learn effective locomotion policies?
Answering the above questions will encompass hands-on experience with a real robotic system (such as ANYmal) together with learning to implement and test cutting-edge RL methods. As RL on real hardware is not yet fully explored, we expect to unearth various challenges concerning the effectiveness of our methods in the online learning setting. Accordingly, an equally important goal of the project is to accurately identify these challenges and propose methodological improvements that can help address them.
A starting point would be to create a model of a typical locomotion task in Isaac Orbit as a proof-of-concept. Following that, the second part of the project will be dedicated to extending the proof-of-concept to a real system.
Not specified
If you are a Master's student with
- basic knowledge in reinforcement learning, for instance, by taking Probabilistic Artificial Intelligence or Foundations of Reinforcement Learning courses;
- strong background in robotics and programming C++, ROS,
please reach out to Yarden As (yarden.as@inf.ethz.ch) or Chenhao Li (chenhao.li@inf.ethz.ch). Feel free to share any previous materials, such as public code that you wrote, that could be relevant in demonstrating the above requirements.
If you are a Master's student with - basic knowledge in reinforcement learning, for instance, by taking Probabilistic Artificial Intelligence or Foundations of Reinforcement Learning courses; - strong background in robotics and programming C++, ROS,
please reach out to Yarden As (yarden.as@inf.ethz.ch) or Chenhao Li (chenhao.li@inf.ethz.ch). Feel free to share any previous materials, such as public code that you wrote, that could be relevant in demonstrating the above requirements.
safe-locomotion-online-learning.pdf
