Abstract: In recent years, the third-party evaluation has been regarded as a means to ensure objective and fair results. However, it is not satisfactory in practice, and is even criticized as pseudo-independent, pseudo-professional, and pseudo-objective. The existing research mostly attribute the phenomenon to the adversarial action of the clients and the targets, and rarely analyze the phenomenon from the perspective of the third party. Taking the third-party evaluation agency S as an example, this paper attempts to display the micro process of social organization grade evaluation. It is found that the self-interest action of the third party is the main reason for the failure. Due to the fuzziness of indicators, the limitation of expert selection and the lack of process supervision, the third-party agencies have great discretion in evaluation, which then gives rise to a series of profit-seeking strategies, and results in negative consequences such as "collusion", "bad money drives out good" and "profit chains". At a deeper level, the lack of constraints is the root cause of the failure of third-party evaluation. The extensive development and formal supervision of the past provide chance for third-party evaluation agencies to pursue profits. What is more, the public sector's need for legitimacy and formal objectivity further reinforces the negative consequences of the profit-seeking behavior. In the context of the new era, Government should pay more attention to the quality of social organization development and its social benefits, and adopt more and pragmatic supervision strategies.