University of Zurich

Open Opportunities

Foundation models for vision-based reinforcement learning University of Zurich Robotics and Perception Vision-based reinforcement learning (RL) is more sample inefficient and more complex to train compared to state-based RL because the policy is learned directly from raw image pixels rather than from the robot state. In comparison to state-based RL, vision-based policies need to learn some form of visual perception or image understanding from scratch, which makes them way more complex to learn and to generalise. Foundation models trained on vast datasets have shown promising potential in outputting feature representations that are useful for a large variety of downstream tasks. In this project, we investigate the capabilities of such models to provide robust feature representations for learning control policies. We plan to study how different feature representations affect the exploration behavior of RL policies, the resulting sample complexity and the generalisation and robustness to out-of-distribution samples. Intelligent Robotics Master Thesis, Semester Project

Sim-to-real transfer of event-camera-based RL policies University of Zurich Robotics and Perception This project aims to develop and evaluate drone navigation policies using event-camera inputs, focusing on the challenges of transferring these policies from simulated environments to the real world. Event cameras, known for their high temporal resolution and dynamic range, offer unique advantages over traditional frame-based cameras, particularly in high-speed and low-light conditions. However, the sim-to-real gap—differences between simulated environments and the real world—poses significant challenges for the direct application of learned policies. In this project we will look try to understand the sim-to-real gap for event cameras and how this gap influences downstream control tasks, such as flying in the dark, dynamic obstacle avoidance and, object catching. Intelligent Robotics, Robotics and Mechatronics Master Thesis

Offline-to-Online (model-based) Reinforcement Learning Transfer and Finetuning for Vision-based Robot Control University of Zurich Robotics and Perception Vision-based reinforcement learning (RL) is often sample-inefficient and computationally very expensive. One way to bootstrap the learning process is to leverage offline interaction data. However, this approach faces significant challenges, including out-of-distribution (OOD) generalization and neural network plasticity. The goal of this project is to explore methods for transferring offline policies to the online regime in a way that alleviates the OOD problem. By initially training the robot's policies system offline, the project seeks to leverage the knowledge of existing robot interaction data to bootstrap the learning of new policies. The focus is on overcoming domain shift problems and exploring innovative ways to fine-tune the model and policy using online interactions, effectively bridging the gap between offline and online learning. This advancement would enable us to efficiently leverage offline data (e.g. from human or expert agent demonstrations or previous experiments) for training vision-based robotic policies. Intelligent Robotics, Robotics and Mechatronics Master Thesis

Hierarchical reinforcement learning for 3D object navigation tasks University of Zurich Robotics and Perception This project aims to simplify the learning process for new drone control tasks by leveraging a pre-existing library of skills through reinforcement learning (RL). The primary objective is to define a skill library that includes both established drone controllers and new ones learned from offline data (skill discovery). Instead of teaching a drone to fly from scratch for each new task, the project focuses on bootstrapping the learning process with these pre-existing skills. For instance, if a drone needs to search for objects in a room, it can utilize its already-acquired flying skills. A high-level policy will be trained to determine which low-level skill to deploy and how to parameterize it, thus streamlining the adaptation to new tasks. This approach promises to enhance efficiency and effectiveness in training drones for a variety of complex control tasks by building on foundational skills. In addition it facilitates training multi-task policies for drones. Intelligent Robotics, Robotics and Mechatronics Master Thesis

Incorporating expert data into model-based reinforcement learning University of Zurich Robotics and Perception Model-based reinforcement learning (MBRL) methods have greatly improved sample efficiency compared to model-free approaches. Nonetheless, the amount of samples and compute required to train these methods remains too large for real-world training of robot control policies. Ideally, we should be able to leverage expert data (collected by human or artificial agents) to bootstrap MBRL. The exact way to leverage such data is yet unclear and many options are available. For instance, it is possible to only use such data for training high-accuracy dynamics models (world models) that are useful for multiple tasks. Alternatively, expert data can (also) be used for training the policy. Additionally, pretraining MBRL components can itself be very challenging as offline expert data is typically sampled from a very narrow distribution of behaviors, which makes finetuning non-trivial in out-of-distributions areas of the robot’s state-action space. In this thesis, you will look at different ways of incorporating expert data in MBRL and ideally propose new approaches to best do that. You will test these methods in both simulation (simulated drone, wheeled, legged) and in the real world on our quadrotor platform. You will gain insights into MBRL, sim-to-real transfer, robot control. Intelligent Robotics, Knowledge Representation and Machine Learning, Robotics and Mechatronics Master Thesis, Semester Project

Learn to navigate within scenes by leveraging object representation and interaction. ETH Zurich Institute of Neuroinformatics Other organizations: ETH Competence Center - ETH AI Center Biological intelligence excels in adapting to new tasks by reusing prior knowledge, as seen in landmark-based navigation where animals plan routes using spatial relationships. In contrast, machine learning struggles with navigation due to challenges like understanding the effects of actions on perception (equivariance), estimating distances to objects without supervision (localization), and dealing with occlusion. The proposed method addresses these challenges by learning equivariant object representations and composing them into a coherent map, enabling accurate pose estimation and navigation. Successfully tested in a 3D simulation, the goal is to apply this approach to more realistic environments, such as drone navigation, and to demonstrate its utility in various tasks through reinforcement learning. Artificial Intelligence and Signal and Image Processing Course Project

Mental Workload Detection and Classification Using fNIRS and Machine Learning University of Zurich Institute of Neuroinformatics Background Mental Workload: Mental workload, also known as cognitive workload, refers to the mental effort and resources required to perform a specific task or activity. It encompasses various cognitive processes such as attention, memory, problem-solving, decision-making, and perception. The more demanding a task, the higher the mental workload required to perform it effectively. In the context of Human-Computer Interaction (HCI), mental workload is critical as it affects the user's ability to process information efficiently. When the workload exceeds a user's cognitive capacity, information processing slows down, leading to increased errors and reduced performance. Understanding and measuring mental workload is crucial because it affects our productivity, mental health, and overall well-being. Background Stroke: Stroke is a major cause of motor impairments, leading to an increased demand for effective rehabilitation methods. Technological solutions like VR-based rehabilitation and robotic movement training systems show promise in minimally supervised settings. However, maintaining patient engagement is a challenge. Balancing visual, memory, and attentional load is critical, especially for stroke patients sensitive to excessive task load. This project aims to develop an adaptive neurorehabilitation system using fNIRS to measure and adjust mental workload, enhancing patient engagement and recovery. Behavioural and Cognitive Sciences, Engineering and Technology, Information, Computing and Communication Sciences, Medical and Health Sciences Bachelor Thesis, Collaboration, Internship, Lab Practice, Master Thesis, Semester Project

Investigating Corticomuscular Coherence (CMC) in Cognitive and Muscular Fatigue during Dual Tasks University of Zurich Neuromorphic Cognitive Systems Group - Indiveri, Giacomo This project investigates the relationship between corticomuscular coherence (CMC) and both muscular and cognitive fatigue during dual-task performance. Using synchronized EMG and EEG data collected via the Quattrocento system from OTBioelettronica, we aim to explore CMC variations at the onset of these signals during solo and dual motor-cognitive tasks. By analyzing data from motor (e.g., reach-and-grab) and cognitive (e.g., reading) activities, we will develop algorithms for automatic detection of EMG and EEG signal onsets. The goal is to validate whether CMC coherence increases during dual tasks, contributing to interventions for fatigue management and cognitive-motor training. This research, in collaboration with IIT Genoa and led by Dr. Marianna Semprini, seeks to enhance our understanding of cognitive-muscular interactions through CMC. Engineering and Technology, Medical and Health Sciences Bachelor Thesis, Master Thesis

Functional MRI investigation in the lumbar spinal cord University of Zurich Neurology Recent research has expanded the investigation of spontaneous fluctuations in the BOLD fMRI signal from the brain to the spinal cord. The vast majority of the studies have focused on the cervical cord, neglecting the lumbar cord which is involved in lower limb control as well as bladder, bowel, and sexual function. In a previous project, we demonstrated the presence of resting-state networks in the lumbar cord. Now, we aim to investigate the reliability of these resting-state networks within and across scans. Another goal is to improve the processing of the BOLD fMRI data, which requires an in-depth comparison of different denoising strategies and exploring their impact on reliability. To achieve these goals, the Master's student will have access to existing resting-state BOLD fMRI data and will also have the opportunity to expand the dataset by acquiring additional data. Medical Biotechnology, Neurology and Neuromuscular Diseases, Neurosciences, Radiology and Organ Imaging, Rehabilitation Engineering Master Thesis

Investigating cells mechanical properties via Fluidic Force Microscopy in a 2D autoimmune skin disease model ETH Zurich Biosensors and Bioelectronics (LBB) The remarkable complexity of morphogenesis and tissue regeneration implies the existence of a transcellular communication network in which individual cells sense the environment and coordinate their biological activity in time and space. To understand the fascinating ability of tissue self-organization, comprehensive study of biophysical properties (cellular nanomechanics such as tension forces and bioelectromagnetics) in combination with the analysis of biochemical networks (signaling pathways and genetic circuits) is required. In this framework we are investigating the unacknowledged key role of Desmoglein 3 (Dsg3) as a receptor involved in mechanosensing, capable of initiating a signaling response in the transcellular communication network, which results in stem cell fate conversion, plasticity and tissue repair. Our goal is to apply innovative Fluidic Force Microscopy to measure altered biophysical parameters upon disruption of Dsg3 transadhesion such as cell stiffness, cell-cell adhesion, cell surface charges and electric potentials. Together with the University of Bern and University of Lübeck we are further investigating how these biophysical changes relate to transcriptomic, epigenomic and proteomic response circuits to ultimately infer biophysical and biochemical circuits involved in Dsg3 signaling. Biochemistry and Cell Biology, Biomedical Engineering, Medical and Health Sciences, Physics Bachelor Thesis, ETH Zurich (ETHZ), Master Thesis, Semester Project