Bottom line is that it's my money, my business, and my property - I'm perfectly within my rights to discriminate in the dispensation of my possessions. Am I violating someone's "civil rights" because I only give money to white homeless and not black ones? No? Then why does the same logic apply to hiring practices which is the same general concept?
Tell me, how successful do you think A & F would be if they refused to hire black people? My guess is that they would go out of business, and rightfully so.