The problem with the "silent (white and mostly male) majority" is that they don't really understand the level of racism being discussed. They always want to dumb it down to the most common denominator. That being how "racism" affects them in their personal lives as if it really made a different in the societal scale of things. You ever notice that when you discuss racism on DP, you have about 10-15 people, most right wing or closeted right wingers crying about that one time they got passed over for some black guy at work? A story which can't be verified in any manner, shape or form is the information they want us to take into consideration when discussing racism. That's it. When you show them the mountains of independent sociological studies, government studies, demonstrating executive level discrimination against minorities, women, etc - they simply brush it off with a standard idiotic response of "slavery is over!", "Women get paid!" and "Obama got elected!" - then they parade their tokens - even the ones who pretend to be black on the internet - to show off how slavery is somehow a "societal problem" that affects us all equally. It really isn't. Not in the American context anyways.
White Protestant males have simply had it better in US history. There is not a SINGLE person with any kind of historiographical studies under their belt who'd deny this on purpose. That's where "white privilege" comes from. It's not really speaking about "white women" privilege, though they have benefited from it. The label targets a pretty specific demographic who for one reason of another wants to deny it actually exists. For every rich African-American, Chinese American and Hispanic-American family there exist dozens of rich white protestant families many of whom wouldn't have reached their level if it weren't for the unique accident that they were born white in a country with 200+ years of social, governmental and cultural policies that benefited protestant whites. Sure, there exist some white groups within the US who have been discriminated against. However, they're few, far in between and noticeable exceptions to the general racism experienced by visible minorities in this country.
The day the silent white mostly male majority recognizes this, we'll be able to discuss racism. Until then, we'll have to live with their personal unverifiable tales of racism and purposeful ignorance of the subject.