Yes indeed the entire concept of white privilege is subjective enough to make me skeptical of its relevance and the need to have the term when other words already cover such social issues. Questioning that relevance in no way makes the claim that one is denying anything like what you mentioned. Another red flag that I noticed is that the term white privilege seems to be used as a derogative slam on white people in general. It makes the claim that white people are unfair and that we do not even know how unfair that we are being. I am always skeptical of something that people say that a entire race is doing but they are unable to see it. Such claims have been made about blacks time and time again. But everyone always points out how that assertion is prejudice yet when the same logic is applied to whites everyone ignores it.
The reason that people like me want the facts and the proof the methodology of how white privilege is defined is because we want things to be straight up. If there is something that can be done to avoid making people feel unwanted or worse then we need to get to the bottom of this. Not just claiming that something exists while being unable to say that its anything more than subjective.