Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning

LYU Kai; SHAO Zhenguo; CHEN Changming; CHEN Feixiong

doi:10.7500/AEPS20251228001

您当前的位置：

首页 >

文章列表页 >

Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning

Basic Research | 更新时间：2026-11-04

- Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning
- Automation of Electric Power Systems Vol. 50, Issue 21, Pages: 151-163(2026)
- 作者机构：
  
  数字能源福建省高校重点实验室（福州大学），福建省福州市 350108
- 作者简介：
- 基金信息：
  
  National Natural Science Foundation of China(72501064);Fujian Provincial Natural Science Foundation of China(2025J08055)
- DOI：10.7500/AEPS20251228001
  CLC：
- Received：28 December 2025，
  
  Revised：2026-03-24，
  
  Online First：03 August 2026，
  
  Published：10 November 2026
- 稿件说明：
移动端阅览
吕凯,邵振国,陈昌铭等.基于多智能体安全强化学习的城市综合能源站双层鲁棒规划[J].电力系统自动化,2026,50(21):151-163.

LYU Kai,SHAO Zhenguo,CHEN Changming,et al.Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning[J].Automation of Electric Power Systems,2026,50(21):151-163.
吕凯,邵振国,陈昌铭等.基于多智能体安全强化学习的城市综合能源站双层鲁棒规划[J].电力系统自动化,2026,50(21):151-163. DOI： 10.7500/AEPS20251228001.

LYU Kai,SHAO Zhenguo,CHEN Changming,et al.Two-layer Robust Planning of Urban Integrated Energy Station Based on Multi-agent Safe Reinforcement Learning[J].Automation of Electric Power Systems,2026,50(21):151-163. DOI： 10.7500/AEPS20251228001.

摘要

针对城市综合能源站选址定容规划中面临的负荷不确定性、地理信息复杂等挑战，提出一种基于多智能体安全强化学习的双层鲁棒规划方法。首先，构建城市综合能源站选址与定容的双层规划框架：上层基于位置集合覆盖模型，融合多能负荷时空特性与地理信息系统数据，以经济性和覆盖效率最优为目标优化城市综合能源站的选址策略；下层考虑多阶段建设时序，以全生命周期成本最小为目标优化设备的多阶段定容与运行策略。然后，提出多智能体安全近端策略优化算法，处理规划过程中的复杂高维不确定性问题。所提算法通过引入价值约束网络，将优化策略引导至安全域内，并采用增广拉格朗日法将约束优化问题转化为无约束优化问题，通过智能体自适应学习不确定环境下的鲁棒优化策略。最后，以某实际城市为例进行算例分析。仿真结果表明，所提模型与算法在经济性、计算效率及应对长期负荷增长不确定性的鲁棒性方面均优于传统方法，可为城市能源系统规划提供有效的决策支持。

Abstract

To address challenges including load uncertainty and complex geographic information faced in the siting and sizing planning of urban integrated energy stations

this paper proposes a two-layer robust planning method based on multi-agent safe reinforcement learning. Firstly

a two-layer planning framework for the siting and sizing of urban integrated energy stations is constructed. At the upper level

based on the location set-covering model

the spatio-temporal characteristics of multi-energy loads and geographic information system data are integrated to optimize the siting strategy of urban integrated energy stations with the objectives of optimal economy and coverage efficiency. At the lower level

considering the multi-stage construction sequence

the multi-stage sizing and operation strategies of equipment are optimized to minimize the full-life-cycle cost. Secondly

a multi-agent safe proximal policy optimization algorithm is proposed to tackle complex high-dimensional uncertainty problems in the planning process. By introducing a value constraint network

the proposed algorithm guides the optimization policy into the safe feasible region. The augmented Lagrangian method is adopted to transform the constrained optimization problem into an unconstrained one

where agents adaptively learn robust optimization policies in uncertain environments. Finally

numerical cases are carried out based on a real-world city. Simulation results demonstrate that the proposed model and algorithm outperform conventional methods in terms of economic performance

computational efficiency

and robustness against long‑term load-growth uncertainty

and can provide effective decision-making support for urban energy system planning.

关键词

Keywords

references

刘硕，滕云，陈哲 . 融合减碳型多能源微网的城市能源系统环境-经济协调优化模型［J］. 电工技术学报， 2025 ， 40 （ 23 ）： 7532 - 7553 .

Liu Shuo ， Teng Yun ， Chen Zhe . An environment-economic coordination optimization model for urban energy systems integrating carbon-reducing multi-energy microgrids ［J］. Transactions of China Electrotechnical Society ， 2025 ， 40 （ 23 ）： 7532 - 7553 .

李争博，刘友波，任鹏哲，等 . 考虑地理信息聚类分区的配电网多阶段动态规划方法［J］. 电力系统自动化， 2022 ， 46 （ 14 ）： 38 - 45 .

Li Zhengbo ， Liu Youbo ， Ren Pengzhe ， et al . Multi-stage dynamic planning method for distribution network considering clustering and partitioning of geographic information ［J］. Automation of Electric Power Systems ， 2022 ， 46 （ 14 ）： 38 - 45 .

朱继忠，骆腾燕，吴皖莉，等 . 综合能源系统运行可靠性评估评述Ⅰ：模型驱动法［J］. 电工技术学报， 2022 ， 37 （ 11 ）： 2761 - 2776 .

Zhu Jizhong ， Luo Tengyan ， Wu Wanli ， et al . A review of operational reliability assessment of integrated energy systems Ⅰ： model-driven method ［J］. Transactions of China Electrotechnical Society ， 2022 ， 37 （ 11 ）： 2761 - 2776 .

王兆强，刘青荣，阮应君，等 . 区域能源规划中站网布置优化设计方法研究与应用［J］. 热能动力工程， 2019 ， 34 （ 8 ）： 25 - 30 .

Wang Zhaoqiang ， Liu Qingrong ， Ruan Yingjun ， et al . Research and application of optimal layout method for station network layout in regional energy planning ［J］. Journal of Engineering for Thermal Energy and Power ， 2019 ， 34 （ 8 ）： 25 - 30 .

Maroufmashat A ， Elkamel A ， Fowler M ， et al . Modeling and optimization of a network of energy hubs to improve economic and emission considerations ［J］. Energy ， 2015 ， 93 ： 2546 - 2558 .

米阳，孔令飞，王育飞，等 . 面向配电网的多综合能源系统容量规划和能源交易优化［J］. 电力自动化设备， 2025 ， 45 （ 9 ）： 38 - 45 .

Mi Yang ， Kong Lingfei ， Wang Yufei ， et al . Capacity planning and energy trading optimization of multiple integrated energy systems oriented to distribution network ［J］. Electric Power Automation Equipment ， 2025 ， 45 （ 9 ）： 38 - 45 .

江城，王浩，徐辰冠，等 . 考虑互联协同的分布式能源站低碳多目标选址规划研究［J］. 全球能源互联网， 2024 ， 7 （ 3 ）： 256 - 270 .

Jiang Cheng ， Wang Hao ， Xu Chenguan ， et al . Low-carbon multi-objective site selection planning for distributed energy stations considering interconnection synergy ［J］. Journal of Global Energy Interconnection ， 2024 ， 7 （ 3 ）： 256 - 270 .

徐成司，董树锋，吴金城，等 . 考虑区域综合能源系统拓扑特性的能源站和管线规划［J］. 电力系统自动化， 2020 ， 44 （ 3 ）： 74 - 82 .

Xu Chengsi ， Dong Shufeng ， Wu Jincheng ， et al . Planning of energy station and pipeline considering topological characteristics of regional integrated energy system ［J］. Automation of Electric Power Systems ， 2020 ， 44 （ 3 ）： 74 - 82 .

毕皓淳，祁琪，刘向军，等 . 计及5G基站运行协同的分布式储能优化配置方法［J］. 电工技术学报， 2024 ， 39 （ 21 ）： 6819 - 6833 .

Bi Haochun ， Qi Qi ， Liu Xiangjun ， et al . An optimal configuration method for distributed energy storage considering coordinated operation of 5G base station ［J］. Transactions of China Electrotechnical Society ， 2024 ， 39 （ 21 ）： 6819 - 6833 .

孟政吉，陈铄鑫，胡雪凯，等 . 多区域分布式能源站选址及供能管线布局规划研究［J］. 河北电力技术， 2022 ， 41 （ 1 ）： 23 - 27 .

Meng Zhengji ， Chen Shuoxin ， Hu Xuekai ， et al . Research on location selection and pipeline layout planning of multi regional distributed energy stations ［J］. Hebei Electric Power ， 2022 ， 41 （ 1 ）： 23 - 27 .

周苏洋，关奥博，顾伟，等 . 工业综合能源系统的多工序统一建模与规划［J］. 电力系统自动化， 2025 ， 49 （ 3 ）： 31 - 41 .

Zhou Suyang ， Guan Aobo ， Gu Wei ， et al . Unified modeling and planning of industrial integrated energy system with multiple processes ［J］. Automation of Electric Power Systems ， 2025 ， 49 （ 3 ）： 31 - 41 .

Cunden T S M ， Doorga J ， Lollchund M R ， et al . Multi-level constraints wind farms siting for a complex terrain in a tropical region using MCDM approach coupled with GIS ［J］. Energy ， 2020 ， 211 ： 118533 .

Jayarathna L ， Kent G ， O’Hara I ， et al . A geographical information system based framework to identify optimal location and size of biomass energy plants using single or multiple biomass types ［J］. Applied Energy ， 2020 ， 275 ： 115398 .

Marques-Perez I ， Guaita-Pradas I ， Gallego A ， et al . Territorial planning for photovoltaic power plants using an outranking approach and GIS ［J］. Journal of Cleaner Production ， 2020 ， 257 ： 120602 .

Unternährer J ， Moret S ， Joost S ， et al . Spatial clustering for district heating integration in urban energy systems： Application to geothermal energy ［J］. Applied Energy ， 2017 ， 190 ： 749 - 763 .

Finney K N ， Sharifi V N ， Swithenbank J ， et al . Developments to an existing city-wide district energy network： Part Ⅰidentification of potential expansions using heat mapping ［J］. Energy Conversion and Management ， 2012 ， 62 ： 165 - 175 .

Molina Bacca E J ， Knight A ， Trifkovic M . Optimal land use and distributed generation technology selection via geographic-based multicriteria decision analysis and mixed-integer programming ［J］. Sustainable Cities and Society ， 2020 ， 55 ： 102055 .

窦嘉铭，王小君，刘曌，等 . 面向综合能源系统调度多场景泛化提升的扩散策略学习［J］. 电力系统自动化， 2026 ， 50 （ 3 ）： 145 - 155 .

Dou Jiaming ， Wang Xiaojun ， Liu Zhao ， et al . Diffusion-policy learning for multi-scenario generalized boosting of integrated energy system scheduling ［J］. Automation of Electric Power Systems ， 2026 ， 50 （ 3 ）： 145 - 155 .

张薇，王浚宇，杨茂，等 . 基于分布式双层强化学习的区域综合能源系统多时间尺度优化调度［J］. 电工技术学报， 2025 ， 40 （ 11 ）： 3529 - 3544 .

Zhang Wei ， Wang Junyu ， Yang Mao ， et al . The multi-time-scale optimal scheduling for regional integrated energy system based on the distributed bi-layer reinforcement learning ［J］. Transactions of China Electrotechnical Society ， 2025 ， 40 （ 11 ）： 3529 - 3544 .

https：//github.com/cityflow-project/CityFlow https://github.com/cityflow-project/CityFlow .

Tang Z ， Naphade M ， Liu M Y ， et al . CityFlow： a city-scale benchmark for multi-target multi-camera vehicle tracking and re-identification ［C］// IEEE/CVF Conference on Computer Vision and Pattern Recognition （CVPR）， 2019 ： 8789 - 8798 .

Views

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

考虑多能负荷不确定性的区域综合能源系统鲁棒规划

Dual-level Planning of Renewable Energy Considering Electricity-Carbon Integrated Demand Response Under Carbon Tax Mechanism

Stochastic Scheduling of New Distribution System Considering Uncertainty of Residential Flexible Load Response

Hierarchical Robust Reinforcement Learning Method for User-side Integrated Energy System Considering Flexibility Resources

Suppression Strategy of High-frequency Oscillation for DFIG Grid-connected System Based on Stator Voltage Feedforward Compensation

Related Author

SHEN Xinwei

GUO Chuangxin

KONG Shuyi

LIN Yujie

XIONG Houbo

GUO Mingyu

DU Liwei

WANG Runzhu

Related Institution

Tsinghua-Berkeley Shenzhen Institute, Tsinghua University

College of Electrical Engineering, Zhejiang University

China Southern Power Grid Dispatch and Control Center

Fuzhou Power Supply Company of State Grid Fujian Electric Power Company

College of Electrical Engineering, Chongqing University

⁰