基于深度强化学习的自适应不确定性经济调度

彭刘阳; 孙元章; 徐箭; 廖思阳; 杨丽

doi:10.7500/AEPS20190706003

您当前的位置：

首页 >

文章列表页 >

基于深度强化学习的自适应不确定性经济调度

学术研究 | 更新时间：2025-06-16

- 基于深度强化学习的自适应不确定性经济调度
- Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning
- 电力系统自动化-方正平台 2020年44卷第9期页码：33-42
- 作者机构：
  
  武汉大学电气与自动化学院，湖北省武汉市 430072
- 作者简介：
  
  彭刘阳(1995—)，男，硕士研究生，主要研究方向：人工智能在电力系统中的应用。E-mail：leftimepelly@whu.edu.cn
  孙元章(1953—)，男，通信作者，教授，博士生导师，主要研究方向：电力系统分析与控制。E-mail：yzsun@mail.tsinghua.edu.cn
  徐箭(1980—)，男，教授，博士生导师，主要研究方向：电力系统稳定与控制。E-mail：xujian@whu.edu.cn
- 基金信息：
  
  国家重点研发计划资助项目(2016YFB0900100);湖北省杰出青年基金资助项目(2018CFA080);国家自然科学基金资助项目(51707136)
- DOI：10.7500/AEPS20190706003
  中图分类号：
- 收稿日期：2019-07-06，
  
  修回日期：2019-12-20，
  
  网络出版日期：2020-03-24，
  
  纸质出版日期：2020-05-10
- 稿件说明：
移动端阅览
彭刘阳,孙元章,徐箭等.基于深度强化学习的自适应不确定性经济调度[J].电力系统自动化,2020,44(9):33-42.

PENG Liuyang,SUN Yuanzhang,XU Jian,et al.Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2020,44(9):33-42.
彭刘阳,孙元章,徐箭等.基于深度强化学习的自适应不确定性经济调度[J].电力系统自动化,2020,44(9):33-42. DOI： 10.7500/AEPS20190706003.

PENG Liuyang,SUN Yuanzhang,XU Jian,et al.Self-adaptive Uncertainty Economic Dispatch Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2020,44(9):33-42. DOI： 10.7500/AEPS20190706003.

摘要

当风电、光伏等间歇性电源大规模接入电力系统时，为应对其出力的不确定性，电力系统经济调度模型需建立在对不确定性建模的基础上，建模精确度将直接影响调度结果的精确度。但当系统同时包含风电、光伏和负荷复杂的不确定性时，对系统整体不确定性进行精确建模显得尤为困难。针对这一问题，引入深度强化学习中深度确定性策略梯度算法，避免对复杂的不确定性进行建模，利用其与环境交互、根据反馈学习改进策略的机制，自适应不确定性的变化。为确保算法适用性，进行了模型泛化方法的设计，针对算法稳定性问题进行了感知-学习比例调整和改进经验回放的机制设计。算例结果表明，所提方法能在自适应系统不确定性的基础上，实现任意场景下的电力系统动态经济调度。

Abstract

When the large-scale intermittent generation sources including wind and photovoltaic generation connects to the power system

in order to cope with their uncertainty

the economic dispatch model for power system needs to be built based on the uncertainty modeling. The accuracy of modeling will directly affect the accuracy of the dispatch results. However

when considering the complex uncertainty of both load and intermittent generation sources such as wind and photovoltaic generation

it is particularly difficult to accurately model the overall uncertainty of the system. In view of this problem

the deep deterministic policy gradient (DDPG) algorithm in the deep reinforcement learning is introduced. The work of uncertainty modeling is avoided. Instead

the uncertainty is adapted by the DDPG algorithm relied on the mechanism of interacting with the environment and improving the strategy based on feedbacks. In order to guarantee the applicability of the algorithm

the generalization method for the DDPG model is proposed. Aiming at the stability problem of the algorithm

two mechanisms are designed

including perception-learning ratio adjustment and experience replay improvement. The result of example shows that the dynamic economic dispatch problem of power systems in any scenario can be solved by the proposed method based on adapting the system uncertainty.

关键词

Keywords

references

HETZER J , YU D C , BHATTARAI K . An economic dispatch model incorporating wind power [J]. IEEE Transactions on Energy Conversion , 2008 , 23 ( 2 ): 603 - 611 .

龚莺飞 , 鲁宗相 , 乔颖 , 等 . 光伏功率预测技术 [J]. 电力系统自动化 , 2016 , 40 ( 4 ): 140 - 151 .

GONG Yingfei , LU Zongxiang , QIAO Ying , et al . An overview of photovoltaic energy system output forecasting technology [J]. Automation of Electric Power Systems , 2016 , 40 ( 4 ): 140 - 151 .

徐迪 , 王洪涛 . 基于随机潮流和风险价值的含大规模风电系统高风险连锁故障评估 [J]. 电网技术 , 2019 , 43 ( 2 ): 400 - 409 .

XU Di , WANG Hongtao . High risk cascading outage assessment in power systems with large-scale wind power based on stochastic power flow and value at risk [J]. Power System Technology , 2019 , 43 ( 2 ): 400 - 409 .

LIANG R H , LIAO Jianhao . A fuzzy-optimization approach for generation scheduling with wind and solar energy systems [J]. IEEE Transactions on Power Systems , 2007 , 22 ( 4 ): 1665 - 1674 .

TANG Chenghui , XU Jian , SUN Yuanzhang , et al . Look-ahead economic dispatch with adjustable confidence interval based on a truncated versatile distribution model for wind power [J]. IEEE Transactions on Power Systems , 2018 , 33 ( 2 ): 1755 - 1767 .

赵唯嘉 , 张宁 , 康重庆 , 等 . 光伏发电出力的条件预测误差概率分布估计方法 [J]. 电力系统自动化 , 2015 , 39 ( 16 ): 8 - 15 .

ZHAO Weijia , ZHANG Ning , KANG Chongqing , et al . A method of probabilistic distribution estimation of conditional forecast error for photovoltaic power generation [J]. Automation of Electric Power Systems , 2015 , 39 ( 16 ): 8 - 15 .

BECKER R . Generation of time-coupled wind power infeed scenarios using Pair-Copula construction [J]. IEEE Transactions on Sustainable Energy , 2018 , 9 ( 3 ): 1298 - 1306 .

潘雄 , 王莉莉 , 徐玉琴 , 等 . 基于混合Copula函数的风电场出力建模方法 [J]. 电力系统自动化 , 2014 , 38 ( 14 ): 17 - 22 .

PAN Xiong , WANG Lili , XU Yuqin , et al . A wind farm power modeling method based on mixed Copula [J]. Automation of Electric Power Systems , 2014 , 38 ( 14 ): 17 - 22 .

刘继 , 徐箭 , 孙元章 , 等 . 考虑风电功率序列时间相关性的电力系统动态经济调度 [J]. 电力系统自动化 , 2019 , 43 ( 3 ): 43 - 55 .

LIU Ji , XU Jian , SUN Yuanzhang , et al . Dynamic economic dispatch of power system considering temporal correlation of wind power sequence [J]. Automation of Electric Power Systems , 2019 , 43 ( 3 ): 43 - 55 .

MISHRA S , MISHRA Y , VIGNESH S . Security constrained economic dispatch considering wind energy conversion systems [C]// IEEE Power & Energy Society General Meeting, July 24-29 , 2011 , Detroit , USA: 8p .

张娜 , 吕泉 , 张显 , 等 . 含风电系统的多级机组组合协调制定策略 [J]. 电力系统自动化 , 2013 , 37 ( 11 ): 39 - 45 .

ZHANG Na , LÜ Quan , ZHANG Xian , et al . A multi-level unit commitment coordinated strategy for power system with wind power [J]. Automation of Electric Power Systems , 2013 , 37 ( 11 ): 39 - 45 .

付一木 , 刘明波 . 求解多目标随机动态经济调度问题的场景解耦方法 [J]. 电力系统自动化 , 2014 , 38 ( 9 ): 34 - 40 .

FU Yimu , LIU Mingbo . Scenario decomposition method for multi-objective stochastic dynamic economical dispatch problem [J]. Automation of Electric Power Systems , 2014 , 38 ( 9 ): 34 - 40 .

WU H , SHAHIDEHPOUR M , LI Z , et al . Chance-constrained day-ahead scheduling in stochastic power system operation [J]. IEEE Transactions on Power Systems , 2014 , 29 ( 4 ): 1583 - 1591 .

左思翔 . 基于深度强化学习的无人驾驶智能决策控制研究 [D]. 哈尔滨 : 哈尔滨工业大学 , 2018 .

ZUO Sixiang . Intelligent control of autonomous driving based on deep reinforcement learning [D]. Harbin : Harbin Institute of Technology , 2018 .

孙润稼 , 刘玉田 . 基于深度学习和蒙特卡洛树搜索的机组恢复在线决策 [J]. 电力系统自动化 , 2018 , 42 ( 14 ): 40 - 47 .

SUN Runjia , LIU Yutian . Online decision-making for generator start-up based on deep learning and Monte Carlo tree search [J]. Automation of Electric Power Systems , 2018 , 42 ( 14 ): 40 - 47 .

孙毅 , 刘迪 , 李彬 , 等 . 深度强化学习在需求响应中的应用 [J]. 电力系统自动化 , 2019 , 43 ( 5 ): 183 - 194 .

SUN Yi , LIU Di , LI Bin , et al . Application of deep reinforcement learning in demand response [J]. Automation of Electric Power Systems , 2019 , 43 ( 5 ): 183 - 194 .

席磊 , 余璐 , 付一木 , 等 . 基于探索感知思维深度强化学习的自动发电控制 [J]. 中国电机工程学报 , 2019 , 39 ( 14 ): 4150 - 4162 .

XI Lei , YU Lu , FU Yimu , et al . Automatic generation control based on deep reinforcement learning with exploration awareness [J]. Proceedings of the CSEE , 2019 , 39 ( 14 ): 4150 - 4162 .

刘洪 , 李吉峰 , 葛少云 , 等 . 基于多主体博弈与强化学习的并网型综合能源微网协调调度 [J]. 电力系统自动化 , 2019 , 43 ( 1 ): 40 - 50 .

LIU Hong , LI Jifeng , GE Shaoyun , et al . Coordinated scheduling of grid-connected integrated energy microgrid based on multi-agent game and reinforcement learning [J]. Automation of Electric Power Systems , 2019 , 43 ( 1 ): 40 - 50 .

刘国静 , 韩学山 , 王尚 , 等 . 基于强化学习方法的风储合作决策 [J]. 电网技术 , 2016 , 40 ( 9 ): 2729 - 2736 .

LIU Guojing , HAN Xueshan , WANG Shang , et al . Optimal decision-making in the cooperation of wind power and energy storage based on reinforcement learning algorithm [J]. Power System Technology , 2016 , 40 ( 9 ): 2729 - 2736 .

JASMIN E A , AHAMED T I , REMANI T . A function approximation approach to reinforcement learning for solving unit commitment problem with photovoltaic sources [C]// IEEE International Conference on Power Electronics, Drives and Energy Systems (PEDES), December 14-17 , 2016 , Trivandrum , India .

LILLICRAP T P , HUNT J J , PRITZEL A , et al . Continuous control with deep reinforcement learning [EB/OL]. [ 2019-07-05 ]. https://arxiv.org/abs/1509.02971v2 https://arxiv.org/abs/1509.02971v2 .

Elia Group . Transparency on grid data [DB/OL]. [ 2019-06-03 ]. http://www.elia.be/en/grid-data http://www.elia.be/en/grid-data .

孙元章 , 吴俊 , 李国杰 , 等 . 基于风速预测和随机规划的含风电场电力系统动态经济调度 [J]. 中国电机工程学报 , 2009 , 29 ( 4 ): 41 - 47 .

SUN Yuanzhang , WU Jun , LI Guojie , et al . Dynamic economic dispatch considering wind power penetration based on wind speed forecasting and stochastic programming [J]. Proceedings of the CSEE , 2009 , 29 ( 4 ): 41 - 47 .

付一木 . 考虑随机风电接入的动态经济调度问题研究 [D]. 广州 : 华南理工大学 , 2016 .

FU Yimu . Study on dynamic economic dispatch considering stochastic wind power integration [D]. Guangzhou : South China University of Technology , 2016 .

浏览量

下载量

CSCD

文章被引用时，请邮件提醒。

提交

工具集

关联资源

基于深度强化学习区间多目标优化的智能建筑低碳优化调度

基于深度强化学习的电动汽车实时调度策略