Prediction and Decision Integrated Scheduling of Energy Storage System in Wind Farm Based on Deep Reinforcement Learning

Yixiao YU; Jiajun YANG; Ming YANG; Yuan GAO

doi:10.7500/AEPS20200226003

您当前的位置：

首页 >

文章列表页 >

Prediction and Decision Integrated Scheduling of Energy Storage System in Wind Farm Based on Deep Reinforcement Learning

Special Topic on Forecasting Technology and Its Application in Power Systems with Renewable Energy Sources | 更新时间：2025-06-16

- Prediction and Decision Integrated Scheduling of Energy Storage System in Wind Farm Based on Deep Reinforcement Learning
- Automation of Electric Power Systems Vol. 45, Issue 1, Pages: 132-140(2021)
- 作者机构：
  
  1.山东大学电网智能化调度与控制教育部重点实验室，山东省济南市 250061
  2.国网陕西省电力公司电力科学研究院，陕西省西安市 710054
- 作者简介：
- 基金信息：
  
  State Grid Corporation of China
- DOI：10.7500/AEPS20200226003
  CLC：
- Received：26 February 2020，
  
  Revised：2020-06-01，
  
  Published Online：04 September 2020，
  
  Published：10 January 2021
- 稿件说明：
移动端阅览
于一潇,杨佳峻,杨明等.基于深度强化学习的风电场储能系统预测决策一体化调度[J].电力系统自动化,2021,45(1):132-140.

YU Yixiao,YANG Jiajun,YANG Ming,et al.Prediction and Decision Integrated Scheduling of Energy Storage System in Wind Farm Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2021,45(1):132-140.
于一潇,杨佳峻,杨明等.基于深度强化学习的风电场储能系统预测决策一体化调度[J].电力系统自动化,2021,45(1):132-140. DOI： 10.7500/AEPS20200226003.

YU Yixiao,YANG Jiajun,YANG Ming,et al.Prediction and Decision Integrated Scheduling of Energy Storage System in Wind Farm Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2021,45(1):132-140. DOI： 10.7500/AEPS20200226003.

摘要

风电场储能系统的优化控制可以提高风电场作为发电商在电力市场中的竞争力。文中提出基于深度强化学习的储能系统预测决策一体化调度方法，令高维度的风电场原始测量数据直接驱动储能系统。与预测、决策相分离的传统调度模式相比，预测决策一体化调度模式将风电功率预测与储能系统动作决策相融合，避免了预测阶段中有效决策信息的损失，使风电的随机规律无须通过数学假设等方式被人为刻画，避免了建模误差。其次，引入深度强化学习Rainbow算法优化风电场测量数据与储能系统动作指令之间端到端的控制策略，该策略具备动态统筹多时段系统收益的能力。最后，基于风电场历史数据的算例分析，验证了所提一体化调度模式的优越性和深度强化学习应对不确定性问题的有效性。

Abstract

The optimal control of the energy storage system (ESS) in the wind farm can improve the competitiveness of the wind farm

which is a power producer in the electricity market. In this paper

a prediction and decision integrated scheduling method of ESS of the wind farm is proposed based on deep reinforcement learning

in order to make the original high-dimensional measurement data of the wind farm directly drive the ESS. Compared with the traditional scheduling model that separates prediction and decision

the prediction and decision integrated scheduling model combines the wind power prediction and ESS operation decision together

to avoid the loss of effective decision-making information in the prediction period. Stochastic laws of wind power do not need to be artificially portrayed through mathematical assumptions and modeling errors are avoided. Then

Rainbow algorithm

a deep reinforcement learning algorithm

is introduced to optimize the end-to-end control strategy between the wind farm measurement data and the ESS operation instruction

which can dynamically coordinate multi-period system profits. Finally

by analyzing case studies based on the wind farm historical data

the superiority of the proposed integrated scheduling mode and the effectiveness of deep reinforcement learning in dealing with the uncertainty problems are verified.

关键词

Keywords

references

YU Yixiao ， HAN Xueshan ， YANG Ming ， et al . Probabilistic prediction of regional wind power based on spatiotemporal quantile regression ［C］// IEEE Industry Applications Society Annual Meeting， September 29-October 2 ， 2019 ， Baltimore ， USA .

HUI Wang ， SHEN Quntai ， ZHEN Taoliu ， et al . Improved control scheme of stand-alone DFIG-based wind energy conversion system with battery energy storage system ［J］. International Journal of Computer Applications in Technology ， 2016 ， 54 （ 3 ）： 211 .

ANTÓNIO C ， SÍLVIO J P S M ， PEDRO M S C ， et al . Hydro-wind optimal operation for joint bidding in day-ahead market： storage efficiency and impact of wind forecasting uncertainty ［J］. Journal of Modern Power Systems and Clean Energy ， 2020 ， 8 （ 1 ）： 142 - 149 .

DING Huajie ， HU Zechun ， SONG Yonghua . Stochastic optimization of the daily operation of wind farm and pumped-hydro-storage plant ［J］. Renewable Energy ， 2012 ， 48 （ 6 ）： 571 - 578 .

BITARAF H ， RAHMAN S ， PIPATTANASOMPORN M . Sizing energy storage to mitigate wind power forecast error impacts by signal processing techniques ［J］. IEEE Transactions on Sustainable Energy ， 2015 ， 6 （ 4 ）： 1457 - 1465 .

ZHANG Z S ， SUN Y Z ， GAO D W ， et al . A versatile probability distribution model for wind power forecast errors and its application in economic dispatch ［J］. IEEE Transactions on Power Systems ， 2013 ， 28 （ 3 ）： 3114 - 3125 .

刘国静，韩学山，王尚，等 . 基于强化学习方法的风储合作决策［J］. 电网技术， 2016 ， 40 （ 9 ）： 2729 - 2736 .

LIU Guojing ， HAN Xueshan ， WANG Shang ， et al . Optimal decision-making in the cooperation of wind power and energy storage based on reinforcement learning algorithm ［J］. Power System Technology ， 2016 ， 40 （ 9 ）： 2729 - 2736 .

MA Jiayi ， MING Yang ， HAN Xueshan ， et al . Ultra-short-term wind generation forecast based on multivariate empirical dynamic modeling ［J］. IEEE Transactions on Industry Applications ， 2018 ， 54 （ 2 ）： 1029 - 1038 .

周玮，钟佳成，孙辉，等 . 基于隐马尔可夫模型的日内风电功率预测误差区间滚动估计［J］. 电力系统自动化， 2018 ， 42 （ 21 ）： 90 - 95 .

ZHOU Wei ， ZHONG Jiacheng ， SUN Hui ， et al . An interval rolling estimation method for daily wind power forecast errors based on hidden Markov model ［J］. Automation of Electric Power Systems ， 2018 ， 42 （ 21 ）： 90 - 95 .

YOU Lin ， MING Yang ， WAN Can ， et al . A multi-model combination approach for probabilistic wind power forecasting ［J］. IEEE Transactions on Sustainable Energy ， 2019 ， 10 （ 1 ）： 226 - 237 .

杨锡运，张艳峰，叶天泽，等 . 基于朴素贝叶斯的风电功率组合概率区间预测［J］. 高电压技术， 2020 ， 46 （ 3 ）： 1099 - 1108 .

YANG Xiyun ， ZHANG Yanfeng ， YE Tianze ， et al . Prediction of combination probability interval of wind power based on naive Bayes ［J］. High Voltage Engineering ， 2020 ， 46 （ 3 ）： 1099 - 1108 .

LI Jiayong ， WAN Can ， ZHAO Xu . Robust offering strategy for a wind power producer under uncertainties ［C］// IEEE International Conference on Smart Grid Communications， November 6-9 ， 2016 ， Sydney ， Australia .

BARTO A G . Reinforcement learning： an introduction ［M］. Cambridge， UK ： MIT Press ， 1998 .

徐龙博，王伟，张滔，等 . 基于神经网络平均影响值的超短期风电功率预测［J］. 电力系统自动化， 2017 ， 41 （ 21 ）： 40 - 45 .

XU Longbo ， WANG Wei ， ZHANG Tao ， et al . Ultra-short-term wind power prediction based on neural network and mean impact value ［J］. Automation of Electric Power Systems ， 2017 ， 41 （ 21 ）： 40 - 45 .

WAN Can ， SONG Yonghua ， ZHAO Xu ， et al . Probabilistic wind power forecasting with hybrid artificial neural networks ［J］. Electric Power Components and Systems ， 2016 ， 44 （ 15 ）： 1656 - 1668 .

叶林，路朋，滕景竹，等 . 考虑风电功率爬坡的功率预测-校正模型［J］. 电力系统自动化， 2019 ， 43 （ 6 ）： 49 - 56 .

YE Lin ， LU Peng ， TENG Jingzhu ， et al . Power prediction and correction model considering wind power ramping events ［J］. Automation of Electric Power Systems ， 2019 ， 43 （ 6 ）： 49 - 56 .

LEVINE S ， FINN C ， DARRELL T ， et al . End-to-end training of deep visuomotor policies ［J］. Journal of Machine Learning Research ， 2015 ， 17 （ 1 ）： 1334 - 1373 .

冯恒，杨争林，郑亚先，等 . 发电商多输入决策因子竞价的智能代理模拟方法［J］. 电力系统自动化， 2018 ， 42 （ 23 ）： 72 - 77 .

FENG Heng ， YANG Zhenglin ， ZHENG Yaxian ， et al . Intelligent agent based bidding simulation method for multi-input decision factors of power suppliers ［J］. Automation of Electric Power Systems ， 2018 ， 42 （ 23 ）： 72 - 77 .

DENG Li ， YU Dong . Deep learning： methods and applications ［J］. Foundations and Trends in Signal Processing ， 2014 ， 7 （ 3/4 ）： 197 - 387 .

MNIH V ， KAVUKCUOGLU K ， SILVER D ， et al . Human-level control through deep reinforcement learning ［J］. Nature ， 2015 ， 518 （ 7540 ）： 529 - 533 .

HESSEL M ， MODAYIL J ， VAN HASSELT H ， et al ． Rainbow： combining improvements in deep reinforcement learning ［EB/OL］. ［ 2020-01-01 ］. https：//arxiv.org/abs/1710.02298 https://arxiv.org/abs/1710.02298 .

SCHAUL T ， QUAN J ， ANTONOGLOU I ， et al . Prioritized experience replay ［EB/OL］. ［ 2020-01-01 ］. https：//arxiv.org/abs/1511.05952 https://arxiv.org/abs/1511.05952 .

VAN HASSELT H G A ， SLIVER D . Deep reinforcement learning with double Q -learning ［EB/OL］. ［ 2020-01-01 ］. https：//arxiv.org/abs/1509.06461 https://arxiv.org/abs/1509.06461 .

WANG Z ， SCHAUL T ， HESSEL M ， et al . Dueling network architectures for deep reinforcement learning ［EB/OL］. ［ 2020-01-01 ］. https：//arxiv.org/abs/1511.06581 https://arxiv.org/abs/1511.06581 .

HINTON G E ， SRIVASTAVA N ， KRIZHEVSKY A ， et al . Improving neural networks by preventing co-adaptation of feature detectors ［J］. Computer Science ， 2012 ， 3 （ 4 ）： 212 - 223 .

YAO Fang ， DONG Zhaoyang ， MENG Ke ， et al . Quantum-Inspired particle swarm optimization for power system operations considering wind power uncertainty and Carbon tax in Australia ［J］. IEEE Transactions on Industrial Informatics ， 2012 ， 8 （ 4 ）： 880 - 888 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Multi-objective Optimal Scheduling Strategy for Energy Storage System Based on Improved Multi-agent Deep Deterministic Policy Gradient Algorithm

Multi-period Equilibrium Analysis Method for Electricity Market Considering Risk Management

Hierarchical Optimization Strategy for Home Energy Management Based on Deep Reinforcement Learning

考虑低碳效益的风电两部复合竞价方法

储能参与的AGC市场出清优化建模新方法

Related Author

WANG Yufei

WANG Yishun

XUE Hua

TU Yiyun

LIN Shunfu

Chengpeng TANG

Lizi ZHANG

Hui DENG

Related Institution

College of Electrical Engineering, Shanghai University of Electric Power

School of Electrical & Electronic Engineering, North China Electric Power University

Electric Power Research Institute, State Grid Zhejiang Electric Power Co., Ltd.

Power Market Simulation Laboratory, State Grid Zhejiang Electric Power Co., Ltd.

State Grid Zhejiang Electric Power Co., Ltd.

⁰