I just don't believe this. I doubt very much that many people would actually simply be calm about accepting someone's refusal to serve them, especially for something that like sex or race or their religion/beliefs. Especially if you took the time to actually go to that place to begin with, having no idea that they would refuse to serve you simply because you were a part of some group of people that they didn't agree with, not because you were asking for something they felt was offensive or hateful.
Imagine driving on a road, and you need gas. The first place you come to says "can't give you gas, your car is blue or foreign". Okay, so you accept that reasoning even if it is stupid and your car takes the gas the person sells. You drive on a little, getting lower on gas as you go. There is another station. You pull over definitely worried now that you might run out of gas despite having filled up at the beginning of your drive. Now, it is you and your girlfriend (not romantic, just a friend) on the road. The woman at the counter misinterprets your hug as romantic and says she can't provide you with service because you are obviously a lesbian couple (each of you have a wedding band on your finger). You try to explain that you aren't but she doesn't want to hear it, saying you would lie to get her service. So you drive away hoping there is another gas station close. Right as you are getting desperate, light's been on, gas gage has been sitting on E, you see another station. You go to get gas, and the attendant stops you, saying he can't sell you gas because the sticker on the back of your car declares you are Catholic. He feels Catholics are wrongly interpreting the scriptures and worship the Pope rather than God/Jesus. Now what do you do? Simply say "well that is their rights" or recognize that all these businesses had different reasons for turning you away, reasons that didn't have anything to do with protecting their business, only their personal biases. And these things put you in danger. Would it be different if the one that refused due to the color or type of car you had was the last one, the one to leave you desperately sitting at E, unable to start your car, and having to call AAA or some other service to get you?