The problem is that the term "alt right" has been used against a wide swath of people, whenever it's convenient.
I think it was CNN who called Ben Shapiro "alt right" last week, over the James Damore thing and if you're not familiar he's a practicing Jewish conservative.
"Alt right" has replaced Nazi, gamergater, etc, as the new catch all slur against right leaning people.
They're still a fringe and still largely internet.
I will defend anyone, who is having their civil rights threatened, who is being doxxed or threatened to be doxxed, who is being wrongly assaulted.
I can't vacate my core beliefs because supremacists suck, they still have rights, whether we like it or not.
I nor anyone else, who happen to/may happen to have some overlap of my ideological vin diagram, am obligated to condemn what is already self evidently wrong.
This is bad faith politics, people should assume, unless you state otherwise, that you think it is wrong.
People who do assume that you align with these wrong beliefs, will not be satisfied by your condemnation.
They'll simply move on to the next thing to rail at you about.