In view of the energy storage control problems of smart grid with multi-type service demands

an analytical model and strategy optimization framework based on semi-Markov control processes are developed by capturing the characteristics of stochastic distributions of service demands

behaviors of users and charging/discharging of energy storage devices. On this basis

with the minimization of long-run average cost of the operation in smart grids as the objective

and by combining with the performance potentials estimate on sample paths

a simulation based policy iteration optimization algorithm is presented. This algorithm alleviates the curse of dimensions induced by the large system state space

and is able to find a near-optimal policy with rapid convergence rate. Simulation results demonstrate the effectiveness of the presented method. This work is supported by National Natural Science Foundation of China (No. 61374158

No. 6123303

No. 61174186) and International Science and Technology Cooperation Program of China (No. 2011DFA10440).