EPFL - Ecole Polytechnique Fédérale de Lausanne

Open Opportunities

Reinforcement Learning-Based Robot Design EPFL - Ecole Polytechnique Fédérale de Lausanne ENAC - Environmental Science and Engineering Section Designing robots for different environments and tasks is a complex and time-consuming process that heavily relies on expert knowledge and human interventions. The automation and understanding of robot design, and the interplay between the structure and controller of a robot has long been a key research question. In recent years, reinforcement learning (RL) techniques have demonstrated great potential in solving complex graph generation problems across various fields, such as symbolic regression, AutoML, and chip design. Therefore, applying RL techniques to the robot design process based on the graph grammar[1] offers a promising solution to automating the design process and improving the robot's performance. Additionally, combining RL-based design with control optimization can lead to more efficient and effective robots with superior performance. Artificial Intelligence and Signal and Image Processing Semester Project

Domain Adaptation of Different Dynamics in Reinforcement Learning EPFL - Ecole Polytechnique Fédérale de Lausanne ENAC - Environmental Science and Engineering Section Reinforcement learning (RL) has demonstrated significant advancements in recent years. However, the performance of RL agents can be heavily influenced by domain shift, limiting its real-world applications. To address the challenge of domain shift in different fields, domain adaptation techniques have been proposed. However, while domain adaptation has been extensively studied in computer vision, it remains relatively unexplored in RL. In particular, the unique characteristics of RL environments require a stronger focus on adapting to varying dynamics, rather than simply adjusting to different state spaces. This aspect has received comparatively less attention, and viable solutions remain under investigation. Moreover, the ability to adapt to dynamic environments is especially critical in industrial settings, where systems are diverse and constantly evolving due to degradation. The challenge of domain adaptation in RL, therefore, presents a significant research opportunity with the potential for substantial real-world impact. Artificial Intelligence and Signal and Image Processing Semester Project

Physics-enhanced Graph Neural Networks for Electrical Distribution System State Estimation EPFL - Ecole Polytechnique Fédérale de Lausanne ENAC - Civil Engineering Section Predictive maintenance is critical in complex industrial systems as it aims to reduce maintenance costs and prevent unplanned downtime by detecting anomalies before they lead to system failure. Anomaly detection is a central component of predictive maintenance and involves identifying deviations from normal system behavior. In electrical distribution systems, one avenue to anomaly detection is to derive an anomaly score from state estimation. State estimation is an existing data processing algorithm for converting redundant phasor meter readings and other available information into an estimate of the state of an electric distribution system. Conventional state estimation (CSE) estimates the state with the iterative Newton-Rhapson method based on a set of power flow equations. CSE is built around a numerical solver for a set of equations that are derived from a schematical representation of the industrial asset. Implementing accurate distribution system state estimation (DSSE) faces several challenges, among them is the lack of observability and the high density of the distribution system, convergence issues, and computational time. While data-driven alternatives based on machine learning models show to be a better choice, they suffer from the lack of labeled training data and do not inherently learn the underlying system physics. Because of that reason, in this thesis, we focus on the sim-to-real transfer of physics-enhanced neural state estimation. The goal is to improve DSSE by pre-training a physics-enhanced model on data from extensive existing simulator packages. We aim to use graph neural networks (GNNs) which have emerged as a promising model for DSSE. GNNs encode graph structures by aggregating features of neighboring nodes, which allows them to capture both local and global structures of the graphs. Real-time processing with data-driven state estimators is still a challenge, but GNNs are a promising approach as they perform well with a comparatively small number of neurons and layers. Electrical Engineering Master Thesis, Semester Project

Multi-variate Time Series Anomaly Detection with Graph Neural Networks EPFL - Ecole Polytechnique Fédérale de Lausanne ENAC - Civil Engineering Section Predictive maintenance is critical in complex industrial systems as it aims to reduce maintenance costs and prevent unplanned downtime by detecting anomalies before they lead to system failure. Anomaly detection is a central component of predictive maintenance and involves identifying deviations from normal system behavior. However, constructing the graph from sensor recording data is often not trivial, because it is not clear how an edge between two time series should be defined. Transforming raw data into a graph representation is one of the key challenges faced in graph-based time series anomaly detection tasks. Time series data is often used to monitor the health of machines and equipment, but traditional approaches to predictive maintenance rely on statistical methods or machine learning models that do not capture the complex relationships between different data points. Graph Neural Networks (GNNs) have emerged as a promising approach for anomaly detection in such cases. GNNs encode graph structures by aggregating features of neighboring nodes, which allows them to capture both local and global structures of the graphs. By creating a graph structure from time series data, where each time series is represented as a node and edges capture interactions between the different time series, GNNs can analyze these relationships over time. In addition, GNNs can incorporate additional information about the time series as node and edge attributes, enabling them to capture nonlinear, large-scale, and heterogeneous relationships between time series. In addition, graphs can reveal fundamental insights into the system, which is valuable by itself, independent of an improvement in prediction quality. Furthermore, complex industrial systems are monitored by a variety of different sensors, which collect large amounts of heterogeneous data. The increasing availability and diversity of data resources offer new possibilities for more efficient and reliable system condition monitoring. However, the heterogeneity of sensor data presents several challenges in graph construction and learning from sensory time series. Artificial Intelligence and Signal and Image Processing, Engineering and Technology Bachelor Thesis, Master Thesis, Semester Project

Control of bumper cars EPFL - Ecole Polytechnique Fédérale de Lausanne STI - Mechanical Engineering Section Develop a control system for bumper cars where there are no collisions, no matter how hard kids try! Equip cars with local collision-avoidance controllers, smoothly overtaking control of the wheel/speed when cars are too close to each other. Electrical Engineering, Systems Theory and Control Master Thesis