The words "racist", "bigot", "Xphobe", and so on are not rational arguments, they are slurs (whether directed against a person or against an argument) designed to make people be quiet. They have the same value and purpose as the n word.
If you make a rational argument, and someone counters by calling you or the argument some variant of ist, ism, or ic, it is counter-productive to try to respond to such slurs with further argument (imagine the absurdity of a black person getting called the n word, and proceeding to argue that that slur is inaccurate as applied to them). Depending on the particular person you're dealing with, the correct response will either be to stop responding to them, or to try to calmly explain that such terms are irrelevant to the question of what's true. In no case does arguing that you aren't a racist (or what have you) have any value, it only encourages further use of such slurs by allowing the conversation to be derailed from the original point.
