Skip to main content

LUP Student Papers

LUND UNIVERSITY LIBRARIES

Deep reinforcement learning for real-time power grid topology optimization

Rothschild, Jacob LU (2021) In Bachelor's Theses in Mathematical Sciences MASK11 20202
Mathematical Statistics
Abstract
In our pursuit of carbon neutrality, drastic changes to generation and consumption of electricity will cause new and complex demands on the power grid and its operators. A cheap, promising, and under-exploited mitigation is real-time power grid topology optimization (RTTO). However, beyond the simplest action of line switching, the combinatorial and non-linear nature of RTTO has made all computational approaches infeasible for grids of interesting scale. At the same time, checking many of its boxes, RTTO may be a task for deep reinforcement learning (DRL).

This thesis starts by providing some further background as to why we care about RTTO. It then covers the deep learning and reinforcement learning concepts that underpin the Deep... (More)
In our pursuit of carbon neutrality, drastic changes to generation and consumption of electricity will cause new and complex demands on the power grid and its operators. A cheap, promising, and under-exploited mitigation is real-time power grid topology optimization (RTTO). However, beyond the simplest action of line switching, the combinatorial and non-linear nature of RTTO has made all computational approaches infeasible for grids of interesting scale. At the same time, checking many of its boxes, RTTO may be a task for deep reinforcement learning (DRL).

This thesis starts by providing some further background as to why we care about RTTO. It then covers the deep learning and reinforcement learning concepts that underpin the Deep Q-Network (DQN). Building on this, it explains the DQN, Double DQN, Dueling DQN, and prioritized experience replay, in some depth. After that, it briefly covers the theory behind line losses, line overloads, and cascading line failures, as well as how bus switching and line switching can help. This is followed by a case study on the winning DRL submission of Geirina to the RTTO competition L2RPN 2019. Finally, the case for DRL for RTTO is made. (Less)
Popular Abstract
As we have come to rely on a constant, uninterrupted electricity supply, the transmission of electricity from generators to consumers has become a vital backbone of modern society. Owing to how safely and reliantly power grids usually perform their task, we tend to take them for granted. Power grids are however some of humanities most complex systems, requiring regular human intervention to function well.

In light of rapid climate change, the sustainable development goals, the Paris climate agreement, and dire outlooks by IPCC, recent years have seen The European Union, Japan, South Korea, and more than 110 other countries pledge carbon neutrality by 2050. Living up to such pledges will require drastic changes to generation and... (More)
As we have come to rely on a constant, uninterrupted electricity supply, the transmission of electricity from generators to consumers has become a vital backbone of modern society. Owing to how safely and reliantly power grids usually perform their task, we tend to take them for granted. Power grids are however some of humanities most complex systems, requiring regular human intervention to function well.

In light of rapid climate change, the sustainable development goals, the Paris climate agreement, and dire outlooks by IPCC, recent years have seen The European Union, Japan, South Korea, and more than 110 other countries pledge carbon neutrality by 2050. Living up to such pledges will require drastic changes to generation and consumption of electricity, which in turn will cause new and complex demands on the power grid and its operators.

Such added demands would traditionally be addressed by physically expanding the grid. However, changing regulations and economics means this will no longer do, making other approaches needed. A cheap, promising, and under-exploited mitigation is real-time topology optimization (RTTO). However, beyond the simplest action of line switching, the combinatorial and non-linear nature of RTTO has made all computational approaches infeasible for grids of interesting scale. Deep reinforcement learning (DRL) may be about to change that.

This thesis starts by providing some further background as to why we care about RTTO. Assuming little previous knowledge, it then covers some relevant parts of deep learning and reinforcement learning, before building on this to give a quite thorough explanation of some central DRL concepts. This is followed by a case study on L2RPN 2019 – an RTTO competition in a simulated environment – and the winning submission by Geirina. Finally, some arguments are given for why DRL is a promising approach to the challenge of RTTO. (Less)
Please use this url to cite or link to this publication:
author
Rothschild, Jacob LU
supervisor
organization
course
MASK11 20202
year
type
M2 - Bachelor Degree
subject
keywords
Deep reinforcement learning, Dueling Deep Q-Network, electricity transmission network, real-time topology optimization, sustainable energy
publication/series
Bachelor's Theses in Mathematical Sciences
report number
LUNFMS-4052-2021
ISSN
1654-6229
other publication id
2021:K9
language
English
id
9042650
date added to LUP
2021-03-31 12:39:30
date last changed
2021-06-03 15:29:26
@misc{9042650,
  abstract     = {{In our pursuit of carbon neutrality, drastic changes to generation and consumption of electricity will cause new and complex demands on the power grid and its operators. A cheap, promising, and under-exploited mitigation is real-time power grid topology optimization (RTTO). However, beyond the simplest action of line switching, the combinatorial and non-linear nature of RTTO has made all computational approaches infeasible for grids of interesting scale. At the same time, checking many of its boxes, RTTO may be a task for deep reinforcement learning (DRL).

This thesis starts by providing some further background as to why we care about RTTO. It then covers the deep learning and reinforcement learning concepts that underpin the Deep Q-Network (DQN). Building on this, it explains the DQN, Double DQN, Dueling DQN, and prioritized experience replay, in some depth. After that, it briefly covers the theory behind line losses, line overloads, and cascading line failures, as well as how bus switching and line switching can help. This is followed by a case study on the winning DRL submission of Geirina to the RTTO competition L2RPN 2019. Finally, the case for DRL for RTTO is made.}},
  author       = {{Rothschild, Jacob}},
  issn         = {{1654-6229}},
  language     = {{eng}},
  note         = {{Student Paper}},
  series       = {{Bachelor's Theses in Mathematical Sciences}},
  title        = {{Deep reinforcement learning for real-time power grid topology optimization}},
  year         = {{2021}},
}