I've always felt that some were born that way and some "ended up" that way for one reason or other.
I've always pointed to a phenomenon most have encountered. You walk into a room and unconsciously count the sexes. Then it turns out your count was wrong because one of the women was actually a man or vice versa. Not cross dressing, just bearing, mannerisms, etc. Not even the gender "overlay" one often sees. Perfectly, completely naturally effeminate men and masculine women.
This always inferred an "inborn" element to me.