Optimal Scheduling of Electric Vehicle Load for Large-scale Battery Charging and Swapping Based on Deep Reinforcement Learning

Dunnan LIU; Lingxiang WANG; Weiye WANG; Hua LI; Wen WANG; Mingguang LIU

doi:10.7500/AEPS20210523003

您当前的位置：

首页 >

文章列表页 >

Optimal Scheduling of Electric Vehicle Load for Large-scale Battery Charging and Swapping Based on Deep Reinforcement Learning

Basic Research | 更新时间：2025-06-16

- Optimal Scheduling of Electric Vehicle Load for Large-scale Battery Charging and Swapping Based on Deep Reinforcement Learning
- Automation of Electric Power Systems Vol. 46, Issue 4, Pages: 36-46(2022)
- 作者机构：
  
  1.华北电力大学经济与管理学院，北京市 102206
  2.国网电动汽车服务有限公司，北京市 100053
- 作者简介：
- 基金信息：
  
  National Social Science Foundation of China(19ZDA081);State Grid Corporation of China(5418-202018247A-0-0-00)
- DOI：10.7500/AEPS20210523003
  CLC：
- Received：23 May 2021，
  
  Revised：2021-06-22，
  
  Published Online：30 August 2021，
  
  Published：25 February 2022
- 稿件说明：
移动端阅览
刘敦楠,王玲湘,汪伟业等.基于深度强化学习的大规模电动汽车充换电负荷优化调度[J].电力系统自动化,2022,46(4):36-46.

LIU Dunnan,WANG Lingxiang,WANG Weiye,et al.Optimal Scheduling of Electric Vehicle Load for Large-scale Battery Charging and Swapping Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2022,46(4):36-46.
刘敦楠,王玲湘,汪伟业等.基于深度强化学习的大规模电动汽车充换电负荷优化调度[J].电力系统自动化,2022,46(4):36-46. DOI： 10.7500/AEPS20210523003.

LIU Dunnan,WANG Lingxiang,WANG Weiye,et al.Optimal Scheduling of Electric Vehicle Load for Large-scale Battery Charging and Swapping Based on Deep Reinforcement Learning[J].Automation of Electric Power Systems,2022,46(4):36-46. DOI： 10.7500/AEPS20210523003.

摘要

针对大规模充换电站的聚合优化调度问题，提出一种基于SAC深度强化学习的充换电负荷实时优化调度策略。该策略充分考虑了负荷调控过程中的用户因素、系统因素和市场因素，能够实现大规模电动汽车与各类电力系统主体的友好互动。首先，考虑充换电站的发展规模和调度性能建立联合运行框架；其次，提出考虑多重用户特征的可调性识别模型对电动汽车的实际可调性进行判断；进而，考虑充换电站优化调度的多重时空特征，构建不同场景下可调充换电负荷的优化调度模型；然后，基于SAC算法求解并网充换电负荷的实时调度方案；最后，以电动汽车聚合优化虚拟电厂负荷为例，验证了SAC算法应用于大规模电动汽车充换电负荷实时优化调度的经济性和高效性。

Abstract

For the optimal scheduling problem of the aggregation in large-scale battery charging and swapping stations

a real-time optimal scheduling strategy is proposed for battery charging and swapping loads based on soft actor-critic (SAC) deep reinforcement learning. The strategy fully considers the user

system and market factors in the load control process

and can realize the friendly interaction between large-scale electric vehicles and various types of power system subjects. Firstly

considering the development scale and scheduling performance of battery charging and swapping stations

a joint operation framework is established. Secondly

an adjustable identification model considering multi-user characteristics is proposed to judge the actual adjustable performance of electric vehicles. Furthermore

considering the multiple spatio-temporal characteristics of the optimal scheduling of battery charging and swapping stations

the optimal scheduling models of adjustable battery charging and swapping loads in different scenarios are constructed. Then

the real-time scheduling scheme of battery charging and swapping loads is solved based on SAC algorithm. Finally

the case of virtual power plant load for electric vehicle load aggregation optimization verifies the economy and high efficiency of the SAC algorithm applied in real-time optimal scheduling of electric vehicle loads for large-scale battery charging and swapping.

关键词

Keywords

references

国家发展改革委 . 关于扩大战略性新兴产业投资培育壮大新增长点增长极的指导意见［EB/OL］.［ 2020-10-05 ］. http：//www.gov.cn/zhengce/2020-09/24/content_5546618.htm http://www.gov.cn/zhengce/2020-09/24/content_5546618.htm .

National Development and Reform Commission . Guidelines on expanding investment in strategic emerging industries to foster and strengthen new growth points and poles ［EB/OL］. ［ 2020-10-05 ］. http：//www.gov.cn/zhengce/2020-09/24/content_5546618.htm http://www.gov.cn/zhengce/2020-09/24/content_5546618.htm .

黄淼，陈鑫，文旭在，等 . 计及市场风险的含电动汽车风光互补微电网随机规划调度模型［J］. 电网技术， 2020 ， 44 （ 8 ）： 2883 - 2890 .

HUANG Miao ， CHEN Xin ， WEN Xuzai ， et al . Stochastic planning and scheduling model of electric vehicles with wind-solar complementary microgrid considering market risks ［J］. Power System Technology ， 2020 ， 44 （ 8 ）： 2883 - 2890 .

ABBASI M H ， TAKI M ， RAJABI A ， et al . Coordinated operation of electric vehicle charging and wind power generation as a virtual power plant： a multi-stage risk constrained approach ［J］. Applied Energy ， 2019 ， 239 ： 1294 - 1307 .

MAHMUD K ， HOSSAIN M J ， RAVISHANKAR J . Peak-load management in commercial systems with electric vehicles ［J］. IEEE Systems Journal ， 2019 ， 13 （ 2 ）： 1872 - 1882 .

张新城，刘志珍，侯延进，等 . 计及冬季预热需求的居民区电动汽车负荷调度策略［J］. 电力自动化设备， 2020 ， 40 （ 11 ）： 115 - 121 .

ZHANG Xincheng ， LIU Zhizhen ， HOU Yanjin ， et al . Electric vehicle load scheduling strategy considering winter preheating demand in residential area ［J］. Electric Power Automation Equipment ， 2020 ， 40 （ 11 ）： 115 - 121 .

SOVACOOL B K ， KESTER J ， NOEL L ， et al . Actors， business models， and innovation activity systems for vehicle-to-grid （V2G） technology： a comprehensive review ［J］. Renewable and Sustainable Energy Reviews ， 2020 ， 131 ： 109963 .

REHMAN U . A decentralized dynamic marketing-based demand response using electric vehicles in smart grid ［J］. Arabian Journal for Science and Engineering ， 2020 ， 45 （ 8 ）： 6475 - 6488 .

HUANG X ， ZHANG Y ， LI D ， et al . An optimal scheduling algorithm for hybrid EV charging scenario using consortium blockchains ［J］. Future Generation Computer Systems ， 2019 ， 91 ： 555 - 562 .

王毅，陈进，麻秀，等 . 采用分群优化的电动汽车与电网互动调度策略［J］. 电力自动化设备， 2020 ， 40 （ 5 ）： 77 - 84 .

WANG Yi ， CHEN Jin ， MA Xiu ， et al . An interactive dispatching strategy for electric vehicles and power grid based on clustering optimization ［J］. Electric Power Automation Equipment ， 2020 ， 40 （ 5 ）： 77 - 84 .

刘东奇，王耀南，袁小芳 . 电动汽车充放电与风力/火力发电系统的协同优化运行［J］. 电工技术学报， 2017 ， 32 （ 3 ）： 18 - 26 .

LIU Dongqi ， WANG Yaonan ， YUAN Xiaofang . Coordinated optimization operation of electric vehicle charge-discharge and wind/thermal power system ［J］. Transactions of China Electrotechnical Society ， 2017 ， 32 （ 3 ）： 18 - 26 .

张高，王旭，蒋传文 . 基于主从博弈的含电动汽车虚拟电厂协调调度［J］. 电力系统自动化， 2018 ， 42 （ 11 ）： 48 - 55 .

ZHANG Gao ， WANG Xu ， JIANG Chuanwen . Coordinated dispatch of virtual power plants with electric vehicles based on master-slave game ［J］. Automation of Electric Power Systems ， 2018 ， 42 （ 11 ）： 48 - 55 .

LIU J ， ZHANG T ， ZHU J ， et al . Allocation optimization of electric vehicle charging station （EVCS） considering with charging satisfaction and distributed renewables integration ［J］. Energy ， 2019 ， 164 ： 560 - 574 .

实地探访新能源汽车换电站：换电模式风口来临政策持续“力挺” ［EB/OL］.［ 2020-07-28 ］. https：//xw.qq.com/cmsid/20200728A0127I00 https://xw.qq.com/cmsid/20200728A0127I00 .

On-site visit to new energy vehicle electrical changing station： the arrival of electrical changing mode tuyere policy continued to “strongly support” ［EB/OL］. ［ 2020-07-28 ］. https：//xw.qq.com/cmsid/20200728A0127I00 https://xw.qq.com/cmsid/20200728A0127I00 .

田文奇，和敬涵，姜久春，等 . 电动汽车换电站有序充电调度策略研究［J］. 电力系统保护与控制， 2012 ， 40 （ 21 ）： 114 - 119 .

TIAN Wenqi ， HE Jinghan ， JIANG Jiuchun ， et al . Research on orderly charging scheduling strategy for electric vehicle electrical changing station ［J］. Power System Protection and Control ， 2012 ， 40 （ 21 ）： 114 - 119 .

刘佳佳 . 电动汽车换电调度及换电站充电策略研究［D］. 武汉：华中科技大学， 2019 .

LIU Jiajia . Research on electric vehicle changing scheduling and charging strategy of electric changing station ［D］. Wuhan ： Huazhong University of Science and Technology ， 2019 .

陈吕鹏，潘振宁，余涛，等 . 基于动态非合作博弈的大规模电动汽车实时优化调度［J］. 电力系统自动化， 2019 ， 43 （ 24 ）： 32 - 40 .

CHEN Lüpeng ， PAN Zhenning ， YU Tao ， et al . Real-time optimal scheduling of large-scale electric vehicles based on dynamic non-cooperative game ［J］. Automation of Electric Power Systems ， 2019 ， 43 （ 24 ）： 32 - 40 .

许刚，张丙旭，张广超 . 电动汽车集群并网的分布式鲁棒优化调度模型［J］. 电工技术学报， 2021 ， 36 （ 3 ）： 565 - 578 .

XU Gang ， ZHANG Bingxu ， ZHANG Guangchao . Distributed and robust optimal scheduling model for large-scale electric vehicles connected to grid ［J］. Transactions of China Electrotechnical Society ， 2021 ， 36 （ 3 ）： 565 - 578 .

王惠中，黄金凤，杨家颜 . 基于MOPSO的电动汽车有序充电调度控制策略［J］. 兰州理工大学学报， 2019 ， 45 （ 5 ）： 85 - 90 .

WANG Huizhong ， HUANG Jinfeng ， YANG Jiayan . Control policy of ordered charging scheduling of eletric automobiles based on MOPSO ［J］. Journal of Lanzhou University of Technology ， 2019 ， 45 （ 5 ）： 85 - 90 .

李咸善，陈敏睿，程杉，等 . 基于双重激励协同博弈的含电动汽车微电网优化调度策略［J］. 高电压技术， 2020 ， 46 （ 7 ）： 2286 - 2296 .

LI Xianshan ， CHEN Minrui ， CHENG Shan ， et al . Optimal scheduling strategy of micro-grid with electric vehicle based on dual excitation cooperative game ［J］. High Voltage Engineering ， 2020 ， 46 （ 7 ）： 2286 - 2296 .

金国彬，潘狄，陈庆，等 . 考虑源荷不确定性的直流配电网模糊随机日前优化调度［J/OL］. 电工技术学报： 1 - 12 ［ 2021-05-20 ］. https：//doi.org/10.19595/j.cnki.1000-6753.tces.201295 https://doi.org/10.19595/j.cnki.1000-6753.tces.201295 .

JIN Guobin ， PAN Di ， CHEN Qing ， et al . Fuzzy stochastic day-ahead optimal scheduling of DC distribution network considering source-charge uncertainty ［J/OL］. Transactions of China Electrotechnical Society ： 1 - 12 ［ 2021-05-20 ］. https：//doi.org/10.19595/j.cnki.1000-6753.tces.201295 https://doi.org/10.19595/j.cnki.1000-6753.tces.201295 .

周华嫣然，周羿宏，胡俊杰，等 . 人工智能技术支撑的集群电动汽车实时优化调度策略［J］. 电网技术， 2021 ， 45 （ 4 ）： 1446 - 1459 .

ZHOU Huayanran ， ZHOU Yihong ， HU Junjie ， et al . Real-time optimal scheduling strategy for clustered electric vehicles supported by artificial intelligence technology ［J］. Power System Technology ， 2021 ， 45 （ 4 ）： 1446 - 1459 .

李航，李国杰，汪可友 . 基于深度强化学习的电动汽车实时调度策略［J］. 电力系统自动化， 2020 ， 44 （ 22 ）： 161 - 167 .

LI Hang ， LI Guojie ， WANG Keyou . Electric vehicle real-time scheduling strategy based on deep reinforcement learning ［J］. Automation of Electric Power Systems ， 2020 ， 44 （ 22 ）： 161 - 167 .

李向杰，张向文 . 基于人工蜂群和支持向量机的电动汽车制动意图识别方法［J/OL］. 中国机械工程： 1 - 12 ［ 2021-06-18 ］. http：//kns.cnki.net/kcms/detail/42.1294.TH.20210329.1711.010.html http://kns.cnki.net/kcms/detail/42.1294.TH.20210329.1711.010.html .

LI Xiangjie ， ZHANG Xiangwen . Braking intention identification method for electric vehicles based on artificial bee colony and support vector machine ［J/OL］. The Chinese Mechanical Engineering ： 1 - 12 ［ 2021-06-18 ］. http：//kns.cnki.net/kcms/detail/42.1294.TH.20210329.1711.010.html http://kns.cnki.net/kcms/detail/42.1294.TH.20210329.1711.010.html .

姚春晓 . 广域可调节负荷的辨识方法研究［D］. 北京：华北电力大学， 2019 .

YAO Chunxiao . Research on identification method of wide-area adjustable load ［D］. Beijing ： North China Electric Power University ， 2019 .

ZHAO T ， LI Y ， PAN X ， et al . Real-time optimal energy and reserve management of electric vehicle fast charging station： hierarchical game approach ［J］. IEEE Transactions on Smart Grid ， 2018 ， 9 （ 5 ）： 5357 - 5370 .

PEVEC D ， BABIC J ， KAYSER M A ， et al . A data-driven statistical approach for extending electric vehicle charging infrastructure ［J］. International Journal of Energy Research ， 2018 ， 42 （ 9 ）： 3102 - 3120 .

黄煜梵，彭诺蘅，林艳，等 . 基于SAC强化学习的多智能体车联网频谱资源动态分配［J/OL］. 计算机工程： 1 - 12 ［ 2020-12-27 ］. http：//kns.cnki.net/kcms/detail/31.1289.TP.20201210.1648.003.html http://kns.cnki.net/kcms/detail/31.1289.TP.20201210.1648.003.html .

HUANG Yufan ， PENG Nuoheng ， LIN Yan ， et al . Dynamic spectrum resource allocation of multi-agent network of vehicles based on SAC reinforcement learning ［J/OL］. Computer Engineering ： 1 - 12 ［ 2020-12-27 ］. http：//kns.cnki.net/kcms/detail/31.1289.TP.20201210.1648.003.html http://kns.cnki.net/kcms/detail/31.1289.TP.20201210.1648.003.html .

HAARNOJA T ， ZHOU A ， ABBEEL P ， et al . Soft actor-critic： off-policy maximum entropy deep reinforcement learning with a stochastic actor ［C］// 35th International Conference on Machine Learning ， July 10-15 ， 2018 ， Stockholm， Sweden ： 2976 - 2989 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

Bi-layer Coordinated Control Strategy of Distribution Network Considering Participation of Electric Vehicles and Microgrid

Optimal Scheduling of Electric Vehicle Clusters Considering Characteristics and Adjustability of Charging Behavior

Coordinated Charging Strategy Applicable to Large-scale Charging Stations Based on Deep Reinforcement Learning

Long-time-scale Charging and Discharging Scheduling of Electric Vehicles Under Joint Price and Incentive Demand Response

Related Author

FAN Peixiao

YANG Jun

WEN Yuxin

KE Song

LIU Xuecheng

DING Leyan

XU Junjun

CHENG Yiling

Related Institution

Hubei Engineering and Technology Research Center for AC/DC Intelligent Distribution Network

School of Electrical Engineering and Automation, Wuhan University

Electric Power Research Institute of China Southern Power Grid Company Limited

College of Automation & College of Artificial Intelligence, Nanjing University of Posts and Telecommunications

School of Electrical Engineering, Southeast University

⁰