The whole “transgender” movement is about denying basic differences between male and female, and suggesting a much higher degree of interchangeability and indistinction between them.
But the truth is that men are different from women, in fundamental, unalterable ways. Among these is that men tend to be bigger, stronger, more aggressive, and more sexually-driven. Aside from basic issues of modesty and propriety, the separation of restrooms, dressing rooms, and such by sex is to protect women from sexual aggression by men, in the situations where women are left most vulnerable. There is much less need to protect men from such aggression from women.
Aside from that, the prominent incidents that have come to public attention involving inappropriate invasions of “transgenders” into opposite-sex facilities have nearly all been men pretending to be women, invading women's-only facilities; and not women pretending to be men and invading men's facilities. Either the latter is happening much less frequently, or they are doing so more discretely.