Mode switching control strategy of voltage source converter based high voltage direct current(VSC-HVDC) transmission system under AC system fault of DC voltage control side is analyzed. A mode switching control based on hysteresis loop and local DC voltage detection is proposed and implementation of this method is given. Relationship between DC power and voltages of both sides under steady state is then derived and a method for calculating DC voltage working range of active power control converter is given. Based on these approaches, a method for determining DC voltage thresholds and references in the mode switching control strategy is proposed. Finally, simulations in PSCAD/EMTDC are conducted to verify the validity of the mode switching control strategy under various faults and operating state. Results show that this determination method can provide mode switching control strategy of VSC-HVDC system with a reliable and accurate reference.