Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Multi-Agent Deep Reinforcement Learning for Building Control
Buildings are very large energy consumers, and smart control algorithms can help mitigate this issue. Since buildings are often separated in different thermal zones coupled through thermal exchanges, multi-agent reinforcement learning (MARL) algorithms are promising candidates. The objective of this work will thus be to implement MARL agents and investigate their performance in minimizing the energy consumption of a building while maintaining the comfort of the occupants, both in simulation and during the deployment in the actual building.
Keywords: Multi-Agent Reinforcement Learning
Neural Networks
Building Control
Machine Learning
Over the past years, buildings consumed 30% and 40% of the end-use energy in Europe and worldwide, respectively. To decrease the energy intensity of the sector, one possibility is to deploy smart control algorithms. While many solutions exist for the latter, this work shall investigate the potential of Multi-Agent Reinforcement Learning (MARL) to solve the building control problem, i.e. minimize the energy consumption while maintaining the indoor temperature between comfort bounds.
Residential buildings are often separated into different thermal zones with distinct actuators (i.e. you can decide to heat each zone separately, respectively cool them in Summer), but thermal exchanges between the zones naturally couple them together. While it is possible to design a unique centralized controller (e.g. a Deep Reinforcement Learning (DRL) agent) that takes decisions for all the zones simultaneously, it might be more efficient to separate the computational load and train individual agents responsible to maintain the temperature in each zone. To ensure good performance is attained in such a setting, however, the agents need to communicate/coordinate, leading to multi-agent algorithms.
Over the past years, buildings consumed 30% and 40% of the end-use energy in Europe and worldwide, respectively. To decrease the energy intensity of the sector, one possibility is to deploy smart control algorithms. While many solutions exist for the latter, this work shall investigate the potential of Multi-Agent Reinforcement Learning (MARL) to solve the building control problem, i.e. minimize the energy consumption while maintaining the indoor temperature between comfort bounds. Residential buildings are often separated into different thermal zones with distinct actuators (i.e. you can decide to heat each zone separately, respectively cool them in Summer), but thermal exchanges between the zones naturally couple them together. While it is possible to design a unique centralized controller (e.g. a Deep Reinforcement Learning (DRL) agent) that takes decisions for all the zones simultaneously, it might be more efficient to separate the computational load and train individual agents responsible to maintain the temperature in each zone. To ensure good performance is attained in such a setting, however, the agents need to communicate/coordinate, leading to multi-agent algorithms.
This project aims to design, train, and tune DRL agents in simulation before deploying them in the NEST demonstrator at Empa, Duebendorf. Assuming that each thermal zone is controlled by a different agent, the student shall assess how to spark collaboration/competition between them to maximize their overall performance, inspired by the existing literature. These controllers will then be compared to other solutions, typically rule-based controllers and possibly centralized DRL agents, to assess their advantages and disadvantages in simulation. Finally, if time permits, the student might deploy the best controller(s) to control the real building and validate their performance in practice.
To summarize, the main objective of this work is hence to find out which MARL framework is best suited for building control, implement it, compared it to other controllers, and possibly deploy it in NEST.
Tentative list of tasks:
1. Review the literature on the existing MARL frameworks.
2. Implement one or several MARL algorithms in Python, embedded in the existing code, to control buildings.
3. Investigate the performance obtained by these DRL agents compared to classical rule-based controllers.
4. (optional) Compare against the performance of other advanced controllers, such as single DRL agents or MPC schemes.
5. (optional) Deploy one of the best controllers in NEST.
6. Write the final report and prepare the final presentation.
This project aims to design, train, and tune DRL agents in simulation before deploying them in the NEST demonstrator at Empa, Duebendorf. Assuming that each thermal zone is controlled by a different agent, the student shall assess how to spark collaboration/competition between them to maximize their overall performance, inspired by the existing literature. These controllers will then be compared to other solutions, typically rule-based controllers and possibly centralized DRL agents, to assess their advantages and disadvantages in simulation. Finally, if time permits, the student might deploy the best controller(s) to control the real building and validate their performance in practice. To summarize, the main objective of this work is hence to find out which MARL framework is best suited for building control, implement it, compared it to other controllers, and possibly deploy it in NEST.
Tentative list of tasks: 1. Review the literature on the existing MARL frameworks. 2. Implement one or several MARL algorithms in Python, embedded in the existing code, to control buildings. 3. Investigate the performance obtained by these DRL agents compared to classical rule-based controllers. 4. (optional) Compare against the performance of other advanced controllers, such as single DRL agents or MPC schemes. 5. (optional) Deploy one of the best controllers in NEST. 6. Write the final report and prepare the final presentation.
Required qualifications of the eligible student for this MSc thesis include good knowledge of neural networks, machine learning, and reinforcement learning. This work furthermore requires good Python programming skills, to comply with the existing code. The candidate should be proficient in English and ready to hold meetings online (Day-to-day work can be done at Empa).
This project will take place in the ehub team at Empa, Dübendorf, but most of the meetings will be conducted over Zoom. This project is supported by #NCCRAutomation. For further inquiries, please contact Loris Di Natale, at loris.dinatale@empa.ch.
Required qualifications of the eligible student for this MSc thesis include good knowledge of neural networks, machine learning, and reinforcement learning. This work furthermore requires good Python programming skills, to comply with the existing code. The candidate should be proficient in English and ready to hold meetings online (Day-to-day work can be done at Empa).
This project will take place in the ehub team at Empa, Dübendorf, but most of the meetings will be conducted over Zoom. This project is supported by #NCCRAutomation. For further inquiries, please contact Loris Di Natale, at loris.dinatale@empa.ch.