基于Hadoop的广域测量系统数据处理

QU Zhaoyang; ZHU Li; ZHANG Shilin

您当前的位置：

首页 >

文章列表页 >

基于Hadoop的广域测量系统数据处理

更新时间：2025-08-13

- 基于Hadoop的广域测量系统数据处理
- Automation of Electric Power Systems Vol. 37, Issue 4, Pages: 92-97(2013)
- 作者机构：
  
  东北电力大学信息工程学院,吉林市,吉林省,132012
- 作者简介：
- 基金信息：
- DOI：
  CLC：
- Published：2013
- 稿件说明：
移动端阅览
QU Zhaoyang, ZHU Li, ZHANG Shilin. [J]. Automation of electric power systems, 2013, 37(4): 92-97.
DOI：

QU Zhaoyang, ZHU Li, ZHANG Shilin. [J]. Automation of electric power systems, 2013, 37(4): 92-97. DOI：

摘要

为解决目前广域测量系统（WAMS）海量数据处理中存在的数据冗余、处理效率低等问题，设计并实现了一个基于Hadoop的WAMS数据处理云计算平台。首先，给出了平台的体系结构。其次，设计了基于Hadoop分布式文件系统（HDFS）存储的WAMS海量数据加载方法和利用 MapReduce模型实现多个文件数据的并行抽取、转换和加载（ETL）操作流程。提出了结合 MapReduce的MPApriori数据挖掘算法，用于高效地挖掘出连锁故障时各站点之间的相互影响。最后，通过对区域电网WAMS实际数据进行处理，验证了Hadoop处理海量数据的高效性。所述平台适用于高性能局域网络连接的计算机集群对海量电网数据进行文件数据处理。

Abstract

To solve the wide area measurement system（WAMS）mass data processing problems such as data redundancy and low processing efficiency，a cloud computing platform based on Hadoop is designed and implemented. The structure of this platform is described first. Then，a WAMS mass data loading method is designed based on Hadoop distributed file system （HDFS）and parallel data extraction-transformation-loading（ETL）for multiple file processing by using MapReduce. MPApriori data mining algorithm combined with MapReduce is proposed to discover the interplay of power sites when cascading failures occurred. Finally，through the regional network WAMS actual data processing，the effectiveness of mass data processing on Hadoop is proven. This platform is suitable for mass power grid files data mining by high performance local area network connection of a computer cluster.

关键词

Keywords

references

Views

2537

下载量

CSCD

Alert me when the article has been cited

提交

Tools

Publicity Resources

基于MapReduce模型的间歇性能源海量数据处理技术

Deception-based Privacy Preservation Method of Dispatch Decision-making Model for Distribution Network in Cloud Computing Environment

Key Technologies and Prospects of Modeling and Simulation of New Power System

基于云计算的电网调度控制培训仿真系统设计

基于功率曲线的风电机组数据清洗算法

Related Author

MEI Huawei

LI Zhiyi

YUAN Cen

YANG Fu

YAO Gang

XIN Huanhai

WANG Wenhai

Qi GUO

Related Institution

School of Electrical and Electronic Engineering，North China Electric Power University

NGICS Platform, Zhejiang University

College of Electrical Engineering, Zhejiang University

Power Dispatch & Control Center, Guizhou Power Grid Company

National Energy Power Grid Technology R&D Centre

⁰