Like I said, the "privilege" is at its core not having had to overcome any externally imposed barriers on your success, the kind of barriers that those of other races, religions, and women in many cases DO have to overcome.
Or to put it another way, I can't speak for you, but I know in my personal or professional life I've never for one second had to worry about my race or my gender or my religion. It has never occurred to me that any of those might be a disadvantage in any way. That's not true for blacks, women, or for other races or religions, for whom who they are IS often/frequently/constantly a barrier to their success.
Finally, if I was worried number 1 about career success and could pick, I'd pick what I am - white male Christian. In most areas of the country, in the vast majority of jobs, that's still true and it's not close (except obviously race/gender/religion specific - I'd make a poor lingerie model...).
It doesn't mean you were GIVEN a thing, or that your road was easy, or that your success illegitimate or in any way unearned. You're misunderstanding what the term is supposed to imply.