Coordinated Optimization of Active Distribution Network Based on Deep Deterministic Policy Gradient Algorithm

Jinxia GONG; Yanmin LIU

doi:10.7500/AEPS20190321010

您当前的位置：

首页 >

文章列表页 >

Coordinated Optimization of Active Distribution Network Based on Deep Deterministic Policy Gradient Algorithm

Basic Research | 更新时间：2020-07-14

- Coordinated Optimization of Active Distribution Network Based on Deep Deterministic Policy Gradient Algorithm
- Automation of Electric Power Systems Issue 6, Pages: 113-120(2020)
- 作者机构：
  
  1. 上海电力大学电气工程学院，上海市 200090
  2. 国网上海市电力公司，上海市 200438
- 作者简介：
- 基金信息：
- DOI：10.7500/AEPS20190321010
  CLC：
- Received：21 March 2019，
  
  Revised：2019-11-13，
  
  Published Online：14 February 2020，
  
  Published：25 March 2020
- 稿件说明：
移动端阅览
龚锦霞,刘艳敏.基于深度确定策略梯度算法的主动配电网协调优化[J].电力系统自动化,2020,44(06):113-120.

GONG Jinxia,LIU Yanmin.Coordinated Optimization of Active Distribution Network Based on Deep Deterministic Policy Gradient Algorithm[J].Automation of Electric Power Systems,2020,44(06):113-120.
龚锦霞,刘艳敏.基于深度确定策略梯度算法的主动配电网协调优化[J].电力系统自动化,2020,44(06):113-120. DOI： 10.7500/AEPS20190321010.

GONG Jinxia,LIU Yanmin.Coordinated Optimization of Active Distribution Network Based on Deep Deterministic Policy Gradient Algorithm[J].Automation of Electric Power Systems,2020,44(06):113-120. DOI： 10.7500/AEPS20190321010.

摘要

将新一代人工智能在智能电网和能源互联网中进行应用，实现高比例可再生能源及时有效接入电网，文中基于深度学习中的深度确定策略梯度(DDPG)算法实现主动配电网的优化运行。首先，构造了含多微电网的主动配电网优化模型的DDPG回报函数，使主动配电网的节点电压总偏差和线损最小，最大限度地降低微电网功率调节量的变化以减小对微电网运行的影响，同时维持联络线功率平衡以减小对配电网的影响。然后，分析了主动配电网优化控制的DDPG样本数据处理、回报函数设计、模型训练和学习过程。最后，通过改进IEEE 14节点算例仿真验证了DDPG算法的有效性。

Abstract

Applying the new generation of artificial intelligence in smart grid and Energy Internet

to achieve high proportion renewable energy access to the power grid in a timely and effective manner

the deep deterministic policy gradient (DDPG) algorithm based on deep learning is applied in the optimized operation of active distribution network (ADN). Firstly， DDPG return function of optimization model for ADN with multiple microgrids is constructed

which can minimize the total node voltage deviation and line loss of ADN. The proposed function can also minimize the variation of the power regulation of microgrid to reduce the impact on operation of the microgrid

and maintain the balance of tie-line power blance to reduce the impact on the distribution network. Secondly

DDPG sample data processing

design of return function

model training and learning process of optimization control for ADN are analyzed. Finally

the effectiveness of the algorithm is verified by the improved IEEE 14-bus example simulation.

关键词

Keywords

references

中国电子学会 . 新一代人工智能发展规划白皮书 [EB/OL].[ 2019-01-22 ]. https://max.book118.com/html/2018/0305/155878388.shtm https://max.book118.com/html/2018/0305/155878388.shtm .

Chinese Institute of Electronics . White paper on development planning of new generation artificial intelligence [EB/OL]. [ 2019-01-22 ]. https://max.book118.com/html/2018/0305/1558783 88.shtm https://max.book118.com/html/2018/0305/1558783 88.shtm .

周念成 , 廖建权 , 王强钢，等 . 深度学习在智能电网中的应用现状分析与展望 [J]. 电力系统自动化 , 2019 , 43 ( 4 ): 180 - 191 . DOI: 10.7500/AEPS20180323002 http://dx.doi.org/10.7500/AEPS20180323002 .

ZHOU Niancheng , LIAO Jianquan , WANG Qianggang , et al . Analysis and prospect of deep learning application in smart grid [J]. Automation of Electric Power Systems , 2019 , 43 ( 4 ): 180 - 191 . DOI: 10.7500/AEPS20180323002 http://dx.doi.org/10.7500/AEPS20180323002 .

唐振韬 , 邵坤 , 赵冬斌 . 深度强化学习进展:从AlphaGo到AlphaGo Zero [J]. 控制理论与应用 , 2017 , 34 ( 12 ): 1529 - 1545 .

TANG Zhentao , SHAO Kun , ZHAO Dongbin . Recent progress of deep reinforcement learning: from AlphaGo to AlphaGo Zero [J]. Control Theory & Applications , 2017 , 34 ( 12 ): 1529 - 1545 ．

GHESU F G , KRUBASIK E , GEORGESCU B . Marginal space deep learning: efficient architecture for volumetric image parsing [J]. IEEE Transactions on Medical Imaging , 2016 , 35 ( 5 ): 1217 - 1228 .

LI H , CAI R , LIU N , et al . Deep reinforcement learning: algorithm, applications, and ultra-low-power implementation [J]. Nano Communication Networks , 2018 , 16 ( 6 ): 81 - 90 .

SUTTON R S , BARTO A G . Reinforcement learning: an introduction [M]. Massachusetts : MIT Press ， 1998 .

YU T , ZHANG X S , ZHOU B , et al . Hierarchical correlated Q-learning for multi-layer optimal generation command dispatch [J]. International Journal of Electrical Power & Energy Systems , 2016 , 78 ( 7 ): 1 - 12 .

赵洪山，刘辉海 . 基于深度学习网络的风电机组主轴承故障检测 [J]. 太阳能学报， 2018 , 39 ( 3 ): 588 - 595 .

ZHAO Hongshan ， LIU Huihai . Fault detection of wind turbine main bear based on deep learning network [J]. Acta Energiae Solaris Sinica , 2018 , 39 ( 3 ): 588 - 595 ．

刘威，张东霞，王新迎，等 . 基于深度强化学习的电网紧急控制策略研究 [J]．中国电机工程学报， 2018 , 38 ( 1 ): 109 - 119 .

LIU Wei , ZHANG Dongxia , WANG Xinying , et al . A decision making strategy for generating unit tripping under emergency circumstances based on deep reinforcement learning [J]. Proceedings of the CSEE , 2018 , 38 ( 1 ): 109 - 119 ．

KOUTSOUKIS C N , GEORGILAKIS S P , HATZIARGYRIOU D N . Multistage coordinated planning of active distribution networks [J]. IEEE Transactions on Power Systems , 2018 , 33 ( 1 ): 32 - 44 .

孟繁星 , 孙英云 , 蒲天骄，等 . 考虑区域自治能力的主动配电网分层优化调度 [J]. 电力系统自动化 , 2018 , 42 ( 15 ): 70 - 76 . DOI: 10.7500/AEPS20170917003 http://dx.doi.org/10.7500/AEPS20170917003 .

MENG Fanxing , SUN Yingyun , PU Tianjiao , et al . Hierarchical optimal scheduling model for active distribution network considering regional autonomy ability [J]. Automation of Electric Power Systems , 2018 , 42 ( 15 ): 70 - 76 . DOI: 10.7500/AEPS20170917003 http://dx.doi.org/10.7500/AEPS20170917003 .

苏小林 , 吴富杰 , 阎晓霞 , 等 . 主动配电网的分层调控体系及区域自治策略 [J]. 电力系统自动化 , 2017 , 41 ( 6 ): 129 - 134 .

SU Xiaolin , WU Fujie , YAN Xiaoxia , et al . Coordinated control system of active distribution network and its regional autonomy strategy [J]. Automation of Electric Power Systems , 2017 , 41 ( 6 ): 129 - 134 ．

LIU Y , ZHAO X , LI G . Research on the optimization plan method based on electric vehicles' scheduling strategy in active distribution network with wind/photovoltaic/energy storage hybrid distribution system [C]// 3rd Asia Conference on Power and Electrical Engineering , June 13 , 2018 , Kitakyushu , Japan: 1 - 7 .

WANG D , MENG K , LUO F , et al . Coordinated dispatch of networked energy storage systems for loading management in active distribution networks [J]. IET Renewable Power Generation , 2016 , 10 ( 9 ): 1374 - 1381 .

李红 . 微网群多智能体协调控制与延时补偿方法研究 [D]. 成都：电子科技大学 , 2018 .

LI Hong . MAS-based cooperative control scheme for microgrid clusters considering time delay compensation [D]. Chengdu ： University of Electronic Science and Technology of China , 2018 .

于文鹏 , 刘东 , 余南华 . 馈线控制误差及其在主动配电网协调控制中的应用 [J]. 中国电机工程学报 , 2013 , 33 ( 13 ): 108 - 115 .

YU Wenpeng , LIU Dong , YU Nanhua . Feeder control error and its application in coordinate control of active distribution network [J]. Proceedings of the CSEE , 2013 , 33 ( 13 ): 108 - 115 ．

刘全 , 翟建伟 , 章宗长，等 . 深度强化学习综述 [J]. 计算机学报 , 2017 , 41 ( 1 ): 1 - 27 .

LIU Quan , ZHAI Jianwei , ZHANG Zongzhang ， et al . A survey on deep reinforcement learning [J]. Chinese Journal of Computers , 2017 , 41 ( 1 ): 1 - 27 ．

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [C]// Proceedings of the International Conference on Learning Representations (ICLR) , May 2 - 4 , 2016 , San Juan, Puerto Rico: 1 - 10 .

SHI H , SUN Y , LI G . Intermittent control with reinforcement leaning [C]// Proceedings of the 2017 International Conference on Progress in Informatics and Computing (PIC) , December 15 - 17 , 2017 , Nanjing, China: 56 - 60 .

YU R , SHI Z , HUANG C . Deep reinforcement learning based optimal trajectory tracking control of autonomous underwater vehicle [C]// Proceedings of the 36th Chinese Control Conference (CCC) , July 26 - 28 , 2017 , Dalian, China: 4958 - 4965 .

TSUNEKAWA K , LEIVA F , SOLAR J R . Visual navigation for biped humanoid robots using deep reinforcement learning [J]. IEEE Robotics and Automation Letters , 2018 , 3 ( 4 ): 3247 - 3254 .

MNIH V , KAVUKCUOGLU K , SILVER D , et al . Human-level control through deep reinforcement learning [J]. Nature , 2015 , 518 ： 529 - 533 .

王锡凡 . 电力系统优化规划 [M]. 北京 : 中国水利电力出版社 , 1990 : 313 - 322 .

WANG Xifan . Power system optimal planning [M]. Beijing : China Water & Power Press , 1990 : 313 - 322 .

吕天光，艾芊，孙树敏，等 . 含多微网的主动配电系统综合优化运行行为分析与建模 [J]. 中国电机工程学报 , 2016 , 36 ( 1 ): 122 - 132 .

LÜ Tianguang , AI Qian , SUN Shumin ， et al . Behavioural analysis and optimal operation of active distribution system with multi-microgrids [J]. Proceedings of the CSEE , 2016 , 36 ( 1 ): 122 - 132 .

王东风 , 孟丽 . 粒子群优化算法的性能分析和参数选择 [J]. 自动化学报 , 2016 , 42 ( 10 ): 1552 - 1561 .

WANG Dongfeng , MENG Li . Performance analysis and parameter selection of PSO algorithms [J]. Acta Automatica Sinica , 2016 , 42 ( 10 ): 1552 - 1561 .

龚锦霞( 1984 —) ，女，通信作者，博士，讲师，硕士生导师，主要研究方向：新能源并网、智能电网调度。E-mail ： jxgong 2015 @ 163 . com

刘艳敏（1983—），女，硕士，高级工程师，主要研究方向：电力系统运行和检修。

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Coordinated Optimal Scheduling of Transmission and Distribution Networks Based on Multiple Agents and Improved Target Cascading Method

Voltage Control for Active Distribution Network Based on Bayesian Deep Reinforcement Learning

Dual-timescale Active and Reactive Power Coordinated Optimization for Active Distribution Network Based on Data-driven and Physical Model

Related Author

JI Xingquan

SUN Chenhao

ZHANG Yumin

YANG Ming

YE Pingfeng

HAN Xueshan

ZHANG Xiao

WU Zhi

Related Institution

College of Electrical Engineering and Automation, Shandong University of Science and Technology

Key Laboratory of Power System Intelligent Dispatch and Control of Ministry of Education (Shandong University)

College of Energy Storage Technology, Shandong University of Science and Technology

School of Electrical Engineering, Southeast University

NARI Group Corporation(State Grid Electric Power Research Institute)

⁰