In the series hybrid active power filter based on the fundamental magnetic flux compensation， the pulse width modulation（PWM） inverter acts as a controlled fundamental current source. At the primary side of the series transformer， the system power supply， system impedance， passive power filter and nonlinear loads affect together to lead the terminals at the secondary side of the series transformer to produce a disturbance harmonic voltage， so the inverter output current contains harmonic components， which influences the filtering effect. The mathematical model of the active power filter based on the PWM inverter is established. Then， the filtering characteristics of the active power filter are analyzed. Three measures， i.e. augmenting the filtering inductance properly， selecting the transformation ratio of the series transformer reasonably， and adopting the voltage feed forward control are proposed to improve the filtering effect. The validity of the principle analysis is verified by the experiment on a 10 kVA active power filter prototype.