Any time you find yourself supporting a sexist, racist, homophobic, etc, position because of you own personal observations, that's something you need to rethink. Personal observations are far too small of a sample size to accurately reflect anything at all about society as a whole, and a person is extremely likely to be seeing the situations in their personal life through a heavy lens of their own subjective opinions and whatnot. That's true for anything, not just issues like sexism, but for issues like sexism it's especially important not to rely on such unreliable data because people get screwed over all the time by common misconceptions like this. You absolutely need to base your opinions on topics like that on statistical data, scientific studies, etc.
The factual data does not support what you are saying. There is clear cut, thoroughly documented, statistics that show that women recieve less pay for the exact same jobs. If you believe that men are more productive workers, certainly you need to be able to back that up with solid, credible, studies documenting it. Speculating on something like that off the cuff is just wrong.