Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning

Liuyang PENG; Yuanzhang SUN; Jian XU; Siyang LIAO; Li YANG

doi:10.7500/AEPS20190706003

您当前的位置：

首页 >

文章列表页 >

Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning

Basic Research | 更新时间：2025-06-16

- Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning
- Automation of Electric Power Systems Vol. 44, Issue 9, Pages: 33-42(2020)
- 作者机构：
  
  武汉大学电气与自动化学院，湖北省武汉市 430072
- 作者简介：
- 基金信息：
  
  Excellent Youth Foundation of Hubei Province of China;National Natural Science Foundation of China
- DOI：10.7500/AEPS20190706003
  CLC：
- Received：06 July 2019，
  
  Revised：2019-12-20，
  
  Published Online：24 March 2020，
  
  Published：10 May 2020
- 稿件说明：
移动端阅览
彭刘阳,孙元章,徐箭等.基于深度强化学习的自适应不确定性经济调度[J].电力系统自动化,2020,44(9):33-42.

PENG Liuyang,SUN Yuanzhang,XU Jian,et al.Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2020,44(9):33-42.
彭刘阳,孙元章,徐箭等.基于深度强化学习的自适应不确定性经济调度[J].电力系统自动化,2020,44(9):33-42. DOI： 10.7500/AEPS20190706003.

PENG Liuyang,SUN Yuanzhang,XU Jian,et al.Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2020,44(9):33-42. DOI： 10.7500/AEPS20190706003.

摘要

当风电、光伏等间歇性电源大规模接入电力系统时，为应对其出力的不确定性，电力系统经济调度模型需建立在对不确定性建模的基础上，建模精确度将直接影响调度结果的精确度。但当系统同时包含风电、光伏和负荷复杂的不确定性时，对系统整体不确定性进行精确建模显得尤为困难。针对这一问题，引入深度强化学习中深度确定性策略梯度算法，避免对复杂的不确定性进行建模，利用其与环境交互、根据反馈学习改进策略的机制，自适应不确定性的变化。为确保算法适用性，进行了模型泛化方法的设计，针对算法稳定性问题进行了感知-学习比例调整和改进经验回放的机制设计。算例结果表明，所提方法能在自适应系统不确定性的基础上，实现任意场景下的电力系统动态经济调度。

Abstract

When the large-scale intermittent generation sources including wind and photovoltaic generation connects to the power system

in order to cope with their uncertainty

the economic dispatch model for power system needs to be built based on the uncertainty modeling. The accuracy of modeling will directly affect the accuracy of the dispatch results. However

when considering the complex uncertainty of both load and intermittent generation sources such as wind and photovoltaic generation

it is particularly difficult to accurately model the overall uncertainty of the system. In view of this problem

the deep deterministic policy gradient (DDPG) algorithm in the deep reinforcement learning is introduced. The work of uncertainty modeling is avoided. Instead

the uncertainty is adapted by the DDPG algorithm relied on the mechanism of interacting with the environment and improving the strategy based on feedbacks. In order to guarantee the applicability of the algorithm

the generalization method for the DDPG model is proposed. Aiming at the stability problem of the algorithm

two mechanisms are designed

including perception-learning ratio adjustment and experience replay improvement. The result of example shows that the dynamic economic dispatch problem of power systems in any scenario can be solved by the proposed method based on adapting the system uncertainty.

关键词

Keywords

references

HETZER J , YU D C , BHATTARAI K . An economic dispatch model incorporating wind power [J]. IEEE Transactions on Energy Conversion , 2008 , 23 ( 2 ): 603 - 611 .

龚莺飞 , 鲁宗相 , 乔颖 , 等 . 光伏功率预测技术 [J]. 电力系统自动化 , 2016 , 40 ( 4 ): 140 - 151 .

GONG Yingfei , LU Zongxiang , QIAO Ying , et al . An overview of photovoltaic energy system output forecasting technology [J]. Automation of Electric Power Systems , 2016 , 40 ( 4 ): 140 - 151 .

徐迪 , 王洪涛 . 基于随机潮流和风险价值的含大规模风电系统高风险连锁故障评估 [J]. 电网技术 , 2019 , 43 ( 2 ): 400 - 409 .

XU Di , WANG Hongtao . High risk cascading outage assessment in power systems with large-scale wind power based on stochastic power flow and value at risk [J]. Power System Technology , 2019 , 43 ( 2 ): 400 - 409 .

LIANG R H , LIAO Jianhao . A fuzzy-optimization approach for generation scheduling with wind and solar energy systems [J]. IEEE Transactions on Power Systems , 2007 , 22 ( 4 ): 1665 - 1674 .

TANG Chenghui , XU Jian , SUN Yuanzhang , et al . Look-ahead economic dispatch with adjustable confidence interval based on a truncated versatile distribution model for wind power [J]. IEEE Transactions on Power Systems , 2018 , 33 ( 2 ): 1755 - 1767 .

赵唯嘉 , 张宁 , 康重庆 , 等 . 光伏发电出力的条件预测误差概率分布估计方法 [J]. 电力系统自动化 , 2015 , 39 ( 16 ): 8 - 15 .

ZHAO Weijia , ZHANG Ning , KANG Chongqing , et al . A method of probabilistic distribution estimation of conditional forecast error for photovoltaic power generation [J]. Automation of Electric Power Systems , 2015 , 39 ( 16 ): 8 - 15 .

BECKER R . Generation of time-coupled wind power infeed scenarios using Pair-Copula construction [J]. IEEE Transactions on Sustainable Energy , 2018 , 9 ( 3 ): 1298 - 1306 .

潘雄 , 王莉莉 , 徐玉琴 , 等 . 基于混合Copula函数的风电场出力建模方法 [J]. 电力系统自动化 , 2014 , 38 ( 14 ): 17 - 22 .

PAN Xiong , WANG Lili , XU Yuqin , et al . A wind farm power modeling method based on mixed Copula [J]. Automation of Electric Power Systems , 2014 , 38 ( 14 ): 17 - 22 .

刘继 , 徐箭 , 孙元章 , 等 . 考虑风电功率序列时间相关性的电力系统动态经济调度 [J]. 电力系统自动化 , 2019 , 43 ( 3 ): 43 - 55 .

LIU Ji , XU Jian , SUN Yuanzhang , et al . Dynamic economic dispatch of power system considering temporal correlation of wind power sequence [J]. Automation of Electric Power Systems , 2019 , 43 ( 3 ): 43 - 55 .

MISHRA S , MISHRA Y , VIGNESH S . Security constrained economic dispatch considering wind energy conversion systems [C]// IEEE Power & Energy Society General Meeting, July 24-29 , 2011 , Detroit , USA: 8p .

张娜 , 吕泉 , 张显 , 等 . 含风电系统的多级机组组合协调制定策略 [J]. 电力系统自动化 , 2013 , 37 ( 11 ): 39 - 45 .

ZHANG Na , LÜ Quan , ZHANG Xian , et al . A multi-level unit commitment coordinated strategy for power system with wind power [J]. Automation of Electric Power Systems , 2013 , 37 ( 11 ): 39 - 45 .

付一木 , 刘明波 . 求解多目标随机动态经济调度问题的场景解耦方法 [J]. 电力系统自动化 , 2014 , 38 ( 9 ): 34 - 40 .

FU Yimu , LIU Mingbo . Scenario decomposition method for multi-objective stochastic dynamic economical dispatch problem [J]. Automation of Electric Power Systems , 2014 , 38 ( 9 ): 34 - 40 .

WU H , SHAHIDEHPOUR M , LI Z , et al . Chance-constrained day-ahead scheduling in stochastic power system operation [J]. IEEE Transactions on Power Systems , 2014 , 29 ( 4 ): 1583 - 1591 .

左思翔 . 基于深度强化学习的无人驾驶智能决策控制研究 [D]. 哈尔滨 : 哈尔滨工业大学 , 2018 .

ZUO Sixiang . Intelligent control of autonomous driving based on deep reinforcement learning [D]. Harbin : Harbin Institute of Technology , 2018 .

孙润稼 , 刘玉田 . 基于深度学习和蒙特卡洛树搜索的机组恢复在线决策 [J]. 电力系统自动化 , 2018 , 42 ( 14 ): 40 - 47 .

SUN Runjia , LIU Yutian . Online decision-making for generator start-up based on deep learning and Monte Carlo tree search [J]. Automation of Electric Power Systems , 2018 , 42 ( 14 ): 40 - 47 .

孙毅 , 刘迪 , 李彬 , 等 . 深度强化学习在需求响应中的应用 [J]. 电力系统自动化 , 2019 , 43 ( 5 ): 183 - 194 .

SUN Yi , LIU Di , LI Bin , et al . Application of deep reinforcement learning in demand response [J]. Automation of Electric Power Systems , 2019 , 43 ( 5 ): 183 - 194 .

席磊 , 余璐 , 付一木 , 等 . 基于探索感知思维深度强化学习的自动发电控制 [J]. 中国电机工程学报 , 2019 , 39 ( 14 ): 4150 - 4162 .

XI Lei , YU Lu , FU Yimu , et al . Automatic generation control based on deep reinforcement learning with exploration awareness [J]. Proceedings of the CSEE , 2019 , 39 ( 14 ): 4150 - 4162 .

刘洪 , 李吉峰 , 葛少云 , 等 . 基于多主体博弈与强化学习的并网型综合能源微网协调调度 [J]. 电力系统自动化 , 2019 , 43 ( 1 ): 40 - 50 .

LIU Hong , LI Jifeng , GE Shaoyun , et al . Coordinated scheduling of grid-connected integrated energy microgrid based on multi-agent game and reinforcement learning [J]. Automation of Electric Power Systems , 2019 , 43 ( 1 ): 40 - 50 .

刘国静 , 韩学山 , 王尚 , 等 . 基于强化学习方法的风储合作决策 [J]. 电网技术 , 2016 , 40 ( 9 ): 2729 - 2736 .

LIU Guojing , HAN Xueshan , WANG Shang , et al . Optimal decision-making in the cooperation of wind power and energy storage based on reinforcement learning algorithm [J]. Power System Technology , 2016 , 40 ( 9 ): 2729 - 2736 .

JASMIN E A , AHAMED T I , REMANI T . A function approximation approach to reinforcement learning for solving unit commitment problem with photovoltaic sources [C]// IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES), December 14-17 , 2016 , Trivandrum , India .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [EB/OL]. [ 2019-07-05 ]. https://arxiv.org/abs/1509.02971v2 https://arxiv.org/abs/1509.02971v2 .

Elia Group . Transparency on grid data [DB/OL]. [ 2019-06-03 ]. http://www.elia.be/en/grid-data http://www.elia.be/en/grid-data .

孙元章 , 吴俊 , 李国杰 , 等 . 基于风速预测和随机规划的含风电场电力系统动态经济调度 [J]. 中国电机工程学报 , 2009 , 29 ( 4 ): 41 - 47 .

SUN Yuanzhang , WU Jun , LI Guojie , et al . Dynamic economic dispatch considering wind power penetration based on wind speed forecasting and stochastic programming [J]. Proceedings of the CSEE , 2009 , 29 ( 4 ): 41 - 47 .

付一木 . 考虑随机风电接入的动态经济调度问题研究 [D]. 广州 : 华南理工大学 , 2016 .

FU Yimu . Study on dynamic economic dispatch considering stochastic wind power integration [D]. Guangzhou : South China University of Technology , 2016 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Low-carbon Optimal Dispatch of Smart Building Based on Interval Multi-objective Optimization with Deep Reinforcement Learning

Real-time Dispatch Strategy for Electric Vehicles Based on Deep Reinforcement Learning

Related Author

HOU Hui

HE Ziyin

CHEN Yue

HOU Tingting

TANG Jinrui

WU Xixiu

Hang LI

Guojie LI

Related Institution

School of Automation, Wuhan University of Technology

Shiyan Power Supply Company, State Grid Hubei Electric Power Company

Economics and Technology Research Institute, State Grid Hubei Electric Power Company

Shenzhen Research Institute, Wuhan University of Technology

Key Laboratory of Control of Power Transmission and Conversion, Ministry of Education (Shanghai Jiao Tong University)

⁰