Topology Optimization of Power Systems Combining Deep Reinforcement Learning and Domain Knowledge

Ziming YAN; Yan XU

doi:10.7500/AEPS20210510001

您当前的位置：

首页 >

文章列表页 >

Topology Optimization of Power Systems Combining Deep Reinforcement Learning and Domain Knowledge

Special Topic on Data-Driven Methods for Modernizing Power Systems | 更新时间：2025-06-16

- Topology Optimization of Power Systems Combining Deep Reinforcement Learning and Domain Knowledge
- Automation of Electric Power Systems Vol. 46, Issue 1, Pages: 60-68(2022)
- 作者机构：
  
  南洋理工大学电气与电子工程学院，新加坡 639798，新加坡
- 作者简介：
- 基金信息：
- DOI：10.7500/AEPS20210510001
  CLC：
- Received：10 May 2021，
  
  Revised：2021-09-04，
  
  Published Online：02 December 2021，
  
  Published：10 January 2022
- 稿件说明：
移动端阅览
严梓铭,徐岩.结合深度强化学习与领域知识的电力系统拓扑结构优化[J].电力系统自动化,2022,46(1):60-68.

YAN Ziming,XU Yan.Topology Optimization of Power Systems Combining Deep Reinforcement Learning and Domain Knowledge[J].Automation of Electric Power Systems,2022,46(1):60-68.
严梓铭,徐岩.结合深度强化学习与领域知识的电力系统拓扑结构优化[J].电力系统自动化,2022,46(1):60-68. DOI： 10.7500/AEPS20210510001.

YAN Ziming,XU Yan.Topology Optimization of Power Systems Combining Deep Reinforcement Learning and Domain Knowledge[J].Automation of Electric Power Systems,2022,46(1):60-68. DOI： 10.7500/AEPS20210510001.

摘要

对拓扑结构进行优化可提高电力系统运行灵活性，然而线路开断与变电站母线分裂等系统级的离散决策变量维度极高。该拓扑结构优化问题难以由传统混合整数优化方法求解。针对该问题，提出了一种结合异步优势Actor-Critic（A3C）深度强化学习与电力系统领域知识的运行优化方法，将在线优化的计算负担转移至离线智能体训练阶段。该方法通过同时考虑拓扑结构与发电出力调整的动作空间设计系统运行控制智能体，以最小化约束越限为训练奖励，通过强制约束校验缩减搜索空间并提高强化学习效率，从而实现电力系统运行拓扑结构优化的快速计算，提高电力系统运行的安全性。仿真测试验证了所提方法的有效性。

Abstract

While the flexibility of power system operation can be improved by topology optimization

the dimension of system-level discrete decision variables

including the connections of lines and substation busbars

is prohibitively high. Thus

the topology optimization problem of power systems can hardly be solved by the conventional mixed-integer optimization method. Aiming at this problem

a reinforcement learning based method is proposed combining asynchronous advantage Actor-Critic (A3C) and power system domain knowledge

which transfers the computational burden of online optimization to the offline agent training stage. The defined reward function is adopted to minimize the violations of power transmission line flow limits. Forced constraints verification is employed to reduce the searching space and improve the efficiency of the reinforcement learning. The fast computation of the topology optimization of power system operation is realized

and the operation security of power systems is enhanced. The effectiveness of the proposed method is validated by simulation testing results.

关键词

Keywords

references

杨知方，钟海旺，夏清，等 . 输电网结构优化问题研究综述和展望［J］. 中国电机工程学报， 2016 ， 36 （ 2 ）： 426 - 434 .

YANG Zhifang ， ZHONG Haiwang ， XIA Qing ， et al . Review and prospect of transmission topology optimization ［J］. Proceedings of the CSEE ， 2016 ， 36 （ 2 ）： 426 - 434 .

HAN J ， PAPAVASILIOU A . The impacts of transmission topology control on the European electricity network ［J］. IEEE Transactions on Power Systems ， 2016 ， 31 （ 1 ）： 496 - 507 .

赵博石，胡泽春，宋永华 . 考虑 N -1安全约束的含可再生能源输电网结构鲁棒优化［J］. 电力系统自动化， 2019 ， 43 （ 4 ）： 16 - 24 .

ZHAO Boshi ， HU Zechun ， SONG Yonghua . Robust optimization of transmission topology with renewable energy sources considering N -1 security constraint ［J］. Automation of Electric Power Systems ， 2019 ， 43 （ 4 ）： 16 - 24 .

蒋航，方颖颖，周建英，等 . 基于网络拓扑结构优化的电力系统网损在线综合优化方法［J］. 自动化与仪器仪表， 2020 （ 11 ）： 210 - 215 .

JIANG Hang ， FANG Yingying ， ZHOU Jianying ， et al . Online comprehensive optimization method for network loss of power system based on network topology adjustment ［J］. Automation & Instrumentation ， 2020 （ 11 ）： 210 - 215 .

李少岩，任乙沛，顾雪平，等 . 基于短路电流约束显式线性建模的输电网结构优化［J］. 电工技术学报， 2020 ， 35 （ 15 ）： 3292 - 3302 .

LI Shaoyan ， REN Yipei ， GU Xueping ， et al . Optimization of transmission network structure based on explicit linear modeling of short circuit current constraints ［J］. Transactions of China Electrotechnical Society ， 2020 ， 35 （ 15 ）： 3292 - 3302 .

LAN T ， WANG W Z ， HUANG G M . Transmission grid topology control using critical switching flow based preventive stabilizing redispatch ［J］. IEEE Transactions on Power Systems ， 2018 ， 33 （ 3 ）： 2572 - 2582 .

YANG Z F ， ZHONG H W ， XIA Q ， et al . Optimal transmission switching with short-circuit current limitation constraints ［J］. IEEE Transactions on Power Systems ， 2016 ， 31 （ 2 ）： 1278 - 1288 .

FISHER E B ， O’NEILL R P ， FERRIS M C . Optimal transmission switching ［J］. IEEE Transactions on Power Systems ， 2008 ， 23 （ 3 ）： 1346 - 1355 .

FULLER J D ， RAMASRA R ， CHA A . Fast heuristics for transmission-line switching ［J］. IEEE Transactions on Power Systems ， 2012 ， 27 （ 3 ）： 1377 - 1386 .

SOROUSH M ， FULLER J D . Accuracies of optimal transmission switching heuristics based on DCOPF and ACOPF ［J］. IEEE Transactions on Power Systems ， 2014 ， 29 （ 2 ）： 924 - 932 .

KHANABADI M ， GHASEMI H ， DOOSTIZADEH M . Optimal transmission switching considering voltage security and N -1 contingency analysis ［J］. IEEE Transactions on Power Systems ， 2013 ， 28 （ 1 ）： 542 - 550 .

WU J ， CHEUNG K W . Incorporating optimal transmission switching in day-ahead unit commitment and scheduling ［C］// 2015 IEEE Power & Energy Society General Meeting ， July 26-30 ， 2015 ， Denver， USA ： 1 - 5 .

GOLDIS E A ， RUIZ P A ， CARAMANIS M C ， et al . Shift factor-based SCOPF topology control MIP formulations with substation configurations ［J］. IEEE Transactions on Power Systems ， 2017 ， 32 （ 2 ）： 1179 - 1190 .

HEIDARIFAR M ， GHASEMI H . A network topology optimization model based on substation and node-breaker modeling ［J］. IEEE Transactions on Power Systems ， 2016 ， 31 （ 1 ）： 247 - 255 .

张尚，王涛，顾雪平，等 . 基于输电网结构优化的电网安全均匀调度［J］. 电网技术， 2017 ， 41 （ 9 ）： 2957 - 2963 .

ZHANG Shang ， WANG Tao ， GU Xueping ， et al . Power grid homogeneous dispatch based on transmission topology optimization ［J］. Power System Technology ， 2017 ， 41 （ 9 ）： 2957 - 2963 .

XIAO R S ， XIANG Y M ， WANG L F ， et al . Power system reliability evaluation incorporating dynamic thermal rating and network topology optimization ［J］. IEEE Transactions on Power Systems ， 2018 ， 33 （ 6 ）： 6000 - 6012 .

ZHANG Z D ， ZHANG D X ， QIU R C . Deep reinforcement learning for power system applications： an overview ［J］. CSEE Journal of Power and Energy Systems ， 2019 ， 6 （ 1 ）： 213 - 225 .

ZHAO T Q ， WANG J H . Learning sequential distribution system restoration via graph-reinforcement learning ［J/OL］. IEEE Transactions on Power Systems ［ 2021-03-04 ］. https：//ieeexplore.ieee.org/document/9508140 https://ieeexplore.ieee.org/document/9508140 .

YOON D ， HONG S ， LEE B J ， et al . Winning the L2RPN challenge： power grid management via semi-Markov after state actor-critic ［C］// International Conference on Learning Representations ， May 4-7 ， 2021 ， Vienna， Austria ： 1 - 6 .

HOSSAIN R R ， HUANG Q H ， HUANG R K . Graph convolutional network-based topology embedded deep reinforcement learning for voltage stability control ［J］. IEEE Transactions on Power Systems ， 2021 ， 36 （ 5 ）： 4848 - 4851 .

孙伟卿，王承民，张焰，等 . 电力系统运行均匀性分析与评估［J］. 电工技术学报， 2014 ， 29 （ 4 ）： 173 - 180 .

SUN Weiqing ， WANG Chengmin ， ZHANG Yan ， et al . Analysis and evaluation on power system operation homogeneity ［J］. Transactions of China Electrotechnical Society ， 2014 ， 29 （ 4 ）： 173 - 180 .

A solution for learning to run the power network challenge ［R/OL］. ［ 2020-10-20 ］. https：//github.com/ZM-Learn/L2RPN_ WCCI_a_Solution https://github.com/ZM-Learn/L2RPN_WCCI_a_Solution .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Decision-making Method for Post-disaster Distribution System Restoration Based on Expert Knowledge-embedded Reinforcement Learning

Related Author

HUANG Yu

ZHANG Xiaoxiao

YUE Dong

HU Songlin

WANG Juan

LI Zhukun

SUN Jian

LIAO Kai

Related Institution

Institute of Advanced Technology for Carbon Neutrality, Nanjing University of Posts and Telecommunications

NARI Group Corporation (State Grid Electric Power Research Institute)

School of Electrical Engineering, Southwest Jiaotong University

School of Electric Power Engineering, South China University of Technology

College of Engineering, Shantou University

⁰