To ensure the successful execution of the new electricity market reform

it is significant to conduct the research on the interest conflict under the new structure market among different entities especially the exactly conflicting interest between the electricity sales company and some other entities. Based on the proposed relationship establishment of electricity market structure under the new environment

a multi-agent technology is adopted to develop the game framework of multiple market entities including the power generation companies

electricity sales companies

and market end-users(compared with non-market users). Then considering on each entity's characteristic and transaction rules and the interaction relationship among multiple entities

a market game model with the aim of entity revenue function is designed. Finally

the electricity trading price and electricity quantity of power generation companies and electricity sales companies under the bilateral contract market and the centralized trading market are optimally solved by the reinforcement learning method. Simulation results show that the market profits and share of the power generation companies and the electricity sales companies mainly depend on the rational distribution of the selling and purchasing power in the bilateral contract market and the centralized trading market. With the advance of the game process and the increasing transparency of the market information

the power price and the profit will gradually tend to be stable.