Comprehensive evaluation of short-term wind power prediction results is essential for the improvement of prediction accuracy and for the application of prediction results. Most of present prediction error evaluation criteria are based on the mean-value indices and cannot be applied to analyze the level of the prediction system in all aspects. The paper summarizes the existence forms of prediction error， and proposes a set of comprehensive evaluation indices， including level errors， phase errors， correlation factor and extreme conditions. Furthermore， based on the actual data of a wind farm in Inner Mongolia， the guidance values of the suggested evaluation methods are verified through the comprehensive assessment of various prediction methods and different error produced sections.