A hierarchical correlated Q-learning (HCEQ) approach is presented to solve the dynamic optimization of generation command dispatch (GCD) for automatic generation control (AGC). In order to decrease the dimensions of GCD

the AGC units are classified into different clusters according to their time delay during frequency control. Compared with single-agent reinforcement learning

the HCEQ method introduces the solution of equilibrium objective function

which effectively improves the optimization speed. The generating error

hydropower capacity margin and AGC regulating cost are turned into the Markov decision process reward function via the linearly weighted aggregate algorithm. The application of the hierarchical correlated Q-learning algorithm in China southern power grid (CSG) model shows that the method proposed is capable of reducing the converging time in the pre-learning process and the AGC regulating cost while improving the control performance of AGC systems in a complicated environment of random perturbation. This work is supported by National Basic Research Program of China (973 Program) (No. 2013CB228205) and National Natural Science Foundation of China (No. 51177051

No. 51477055).