Register now After registration you will be able to apply for this opportunity online.
Pretraining for RL
This project addresses sampling inefficiency in classical reinforcement learning by exploring smart weight initialization. Inspired by computer vision, we aim to enhance learning across different hardware (cross embodiment) and skills (cross skills) using pre-trained representations, reducing training times and potentially improving the overall effectiveness of reinforcement learning policies.
This project addresses the sampling inefficiency inherent in classical reinforcement learning approaches. These approaches typically require training from scratch despite having nearly identical observation spaces.
Taking inspiration from computer vision, we will investigate the effect of smart weight initialization on learning the same skill across multiple hardware (cross embodiment) and different skills on the same hardware (cross skills).
Previous work has demonstrated that pre-trained visual representations derived from large-scale computer vision datasets can compete with or surpass the performance of ground-truth state representations. This success in the perception module of policies provides a promising indication that similar pre-training strategies could enhance the overall effectiveness of reinforcement learning policies. A key contribution of this project is a careful analysis of the advantages and pitfalls of initializing with pre-trained weights to avoid making exploration in RL even harder and getting stuck in local minima. This project will aim to answer the questions of how and when, and its results are an excellent starting point for an impactful publication.
This project addresses the sampling inefficiency inherent in classical reinforcement learning approaches. These approaches typically require training from scratch despite having nearly identical observation spaces. Taking inspiration from computer vision, we will investigate the effect of smart weight initialization on learning the same skill across multiple hardware (cross embodiment) and different skills on the same hardware (cross skills). Previous work has demonstrated that pre-trained visual representations derived from large-scale computer vision datasets can compete with or surpass the performance of ground-truth state representations. This success in the perception module of policies provides a promising indication that similar pre-training strategies could enhance the overall effectiveness of reinforcement learning policies. A key contribution of this project is a careful analysis of the advantages and pitfalls of initializing with pre-trained weights to avoid making exploration in RL even harder and getting stuck in local minima. This project will aim to answer the questions of how and when, and its results are an excellent starting point for an impactful publication.
- Literature review on RL and weight initialization in computer vision.
- Implementation of cross-embodiment and cross-skill learning setups.
- Integration with ANYmal bridging the sim-to-real gap.
- Extensive comparisons and evaluations to analyze the robustness and downsides of weight reuse in RL.
- Literature review on RL and weight initialization in computer vision. - Implementation of cross-embodiment and cross-skill learning setups. - Integration with ANYmal bridging the sim-to-real gap. - Extensive comparisons and evaluations to analyze the robustness and downsides of weight reuse in RL.
- Programming experience (Python / C++).
- Working experience with large codebases.
- Experience with deep learning projects (preferably with RL).
- Programming experience (Python / C++). - Working experience with large codebases. - Experience with deep learning projects (preferably with RL).
Send your CV and transcript to: tifanny.portela@ai.ethz.ch, crandrei@ethz.ch, rothpa@ethz.ch
Send your CV and transcript to: tifanny.portela@ai.ethz.ch, crandrei@ethz.ch, rothpa@ethz.ch
