Register now After registration you will be able to apply for this opportunity online.
Model-free RL for highly flexible Legged Robots
Soft robots, characterized by highly flexible components, hold the potential to attain remarkable locomotion capabilities that surpass those of traditional rigid robots. However, effectively modeling, simulating, and controlling these advanced designs present significant challenges. The nonlinear dynamics inherent in these systems elude accurate representation by model-based controllers, while model-free algorithms face limitations in accessing efficient simulation environments necessary for agent training. Thus, developing robust and effective reinforcement learning (RL) approaches tailored to the unique characteristics of soft robots is imperative. In this student project, we aim to explore and apply RL techniques to enhance the locomotion performance of soft robots by addressing the aforementioned challenges. Through innovative modeling strategies, sophisticated simulation environments, and state-of-the-art model-free RL algorithms, we seek to unlock the full potential of legged robots with flexible components, enabling them to achieve exceptional locomotion capabilities and pave the way for advancements in this field.
This thesis should work on how to integrate model-free RL actuator controllers in
simulation environments that use implicit time-integration schemes. A mini physics engine
for unconstrained mechanical systems with an interface to the rsl_rl reinforcement learning
environment will be developed. In addition, various control tasks for simple, highly flexible, and
underactuated benchmark systems will be studied. Upon a successful concept study, the strategy can
be implemented in a more involved physics engine that can handle multi-body systems composed of
interacting rigid bodies and highly flexible nonlinear rods. The project is in cooperation with the
Institute for Nonlinear Mechanics at the University of Stuttgart (Germany).
Upon successful execution, the proposed strategy can be designed and deployed on small-scale legged robots.
This thesis should work on how to integrate model-free RL actuator controllers in simulation environments that use implicit time-integration schemes. A mini physics engine for unconstrained mechanical systems with an interface to the rsl_rl reinforcement learning environment will be developed. In addition, various control tasks for simple, highly flexible, and underactuated benchmark systems will be studied. Upon a successful concept study, the strategy can be implemented in a more involved physics engine that can handle multi-body systems composed of interacting rigid bodies and highly flexible nonlinear rods. The project is in cooperation with the Institute for Nonlinear Mechanics at the University of Stuttgart (Germany).
Upon successful execution, the proposed strategy can be designed and deployed on small-scale legged robots.
- Implementing simple base-line rigid systems (e.g. inverted pendulum, cart-pole, 2D Biped)
- Interfacing RL algorithms, training, and analysis of the base-line systems
- Integrate the code stack into the physics engine Cardillo (not yet released)
- Training and analysis of the base-line systems with various different types of highly flexible components
- Comparison with the base-line rigid systems in terms of learning performance, speed, and capabilities
- Implementing simple base-line rigid systems (e.g. inverted pendulum, cart-pole, 2D Biped) - Interfacing RL algorithms, training, and analysis of the base-line systems - Integrate the code stack into the physics engine Cardillo (not yet released) - Training and analysis of the base-line systems with various different types of highly flexible components - Comparison with the base-line rigid systems in terms of learning performance, speed, and capabilities
- Prior knowledge of artificial neural networks
- Prior knowledge in Computational Dynamics for Robotic Systems
- Strong coding skills in Python
- Strong communication skills and motivation
- Prior knowledge of artificial neural networks - Prior knowledge in Computational Dynamics for Robotic Systems - Strong coding skills in Python - Strong communication skills and motivation
Your application should include a brief motivational statement, your transcript of records and your CV. Filip Bjelonic (fbjelonic@ethz.ch) Simon Eugster (eugster@inm.uni-stuttgart.de)
Your application should include a brief motivational statement, your transcript of records and your CV. Filip Bjelonic (fbjelonic@ethz.ch) Simon Eugster (eugster@inm.uni-stuttgart.de)
