Register now After registration you will be able to apply for this opportunity online.
This opportunity is not published. No applications will be accepted.
Data-driven robust dynamic programming for energy systems management
In this project we aim at apply and extend robust data-driven dynamic programming (DDP) algorithms present in literature. We will test the performance of the developed scheme in the context of energy system management.
Keywords: Stochastic optimal control - Robust dynamic programming - Data-driven control - Energy systems management
Stochastic optimal control has numerous applications in engineering and science, e.g. in sup-
ply chain management, power systems scheduling, behavioral neuroscience, asset allocation,
etc. It refers to a sub field of control theory that deals with the existence of uncertainties
that drive the evolution of the system in an unpredictable manner. Typically the distribution of the exogenous uncertainty is unknown and must be inferred from limited historical
data. In this framework, solving this optimal control program using standard techniques
from dynamic programming is computationally burdensome, and is often complicated by
the difficulty of estimating conditional distributions from sparse data.
To overcome these challenges, some data-driven dynamic programming (DDP) algo-
rithms have recently appeared in literature, leveraging historical data observations to gen-
erate empirical conditional distributions and approximate the cost-to-go function. The re-
sulting data-driven DP scheme is asymptotically consistent and admits an effcient compu-
tational solution when combined with parametric value function approximations. However,
if training data is sparse, the estimated cost-to-go functions display a high variability and
the corresponding control policies perform poorly in out-of-sample tests. To mitigate these
sample effects robust data-driven DP schemes have been proposed, which replace the expectations
in the DP recursions with worst-case expectations over a set of distributions close to the
best estimate according to the Wasserstein distance.
In this project we aim at exploiting the proposed algorithm based on Wasserstein distance to the problem of optimal management of a renewable-based energy hub with storage
facilities. A seminal work in thsi application area considers a PV+battery sys-
tem for residential applications: building upon these results, we aim to extend the framework
to account for longer horizons (up to one year, in order to account for the seasonality effect
which plays a key role in the renewable energy panorama) as well as to include the selection
of device sizes as part of the optimization routine. The algorithm is going to be tested on a
benchmark test case and can be compared with an in-house method to showcase advantages
and disadvantages.
Stochastic optimal control has numerous applications in engineering and science, e.g. in sup- ply chain management, power systems scheduling, behavioral neuroscience, asset allocation, etc. It refers to a sub field of control theory that deals with the existence of uncertainties that drive the evolution of the system in an unpredictable manner. Typically the distribution of the exogenous uncertainty is unknown and must be inferred from limited historical data. In this framework, solving this optimal control program using standard techniques from dynamic programming is computationally burdensome, and is often complicated by the difficulty of estimating conditional distributions from sparse data. To overcome these challenges, some data-driven dynamic programming (DDP) algo- rithms have recently appeared in literature, leveraging historical data observations to gen- erate empirical conditional distributions and approximate the cost-to-go function. The re- sulting data-driven DP scheme is asymptotically consistent and admits an effcient compu- tational solution when combined with parametric value function approximations. However, if training data is sparse, the estimated cost-to-go functions display a high variability and the corresponding control policies perform poorly in out-of-sample tests. To mitigate these sample effects robust data-driven DP schemes have been proposed, which replace the expectations in the DP recursions with worst-case expectations over a set of distributions close to the best estimate according to the Wasserstein distance. In this project we aim at exploiting the proposed algorithm based on Wasserstein distance to the problem of optimal management of a renewable-based energy hub with storage facilities. A seminal work in thsi application area considers a PV+battery sys- tem for residential applications: building upon these results, we aim to extend the framework to account for longer horizons (up to one year, in order to account for the seasonality effect which plays a key role in the renewable energy panorama) as well as to include the selection of device sizes as part of the optimization routine. The algorithm is going to be tested on a benchmark test case and can be compared with an in-house method to showcase advantages and disadvantages.
1. Literature review on Approximate Dynamic Programming (e.g. value function ap-
proximation) and (Distributionally) Robust Optimization to deal with uncertainties
(from selected papers)
2. Working understatement of the data-driven robust dynamic programming approach proposed in selected papers to propose extensions
3. Derive suitable reformulations for the specific problem and implement them
4. Compare results on a benchmark test case.
1. Literature review on Approximate Dynamic Programming (e.g. value function ap- proximation) and (Distributionally) Robust Optimization to deal with uncertainties (from selected papers) 2. Working understatement of the data-driven robust dynamic programming approach proposed in selected papers to propose extensions 3. Derive suitable reformulations for the specific problem and implement them 4. Compare results on a benchmark test case.
Qualifications:
Interested Master students with solid mathematical foundations and good programming
skills (either in Matlab or Python) are encouraged to apply. Familiarity with the main
concepts of (data-driven) dynamic programming is not required, but is certainly a plus.
How to apply:
To apply send CV and updated transcripts of records (BSc and MSc) to:
• Marta Fochesato (mfochesato@ethz.ch)
• Dr. Soroosh Shafieezadeh Abadeh (soroosh.shafiee@gmail.com).
Qualifications: Interested Master students with solid mathematical foundations and good programming skills (either in Matlab or Python) are encouraged to apply. Familiarity with the main concepts of (data-driven) dynamic programming is not required, but is certainly a plus. How to apply: To apply send CV and updated transcripts of records (BSc and MSc) to:
• Marta Fochesato (mfochesato@ethz.ch)
• Dr. Soroosh Shafieezadeh Abadeh (soroosh.shafiee@gmail.com).