@January, we aren't talking about "bathrooms".
There is a big difference between the public lavatories built in the old days for men and for women.
I personally would have no problem with a trans woman using the ladies' facilities. After all, there is always a door for every cubicle. Nor would I object if a trans person chose to use the handicapped toilet, but that is a far from ideal solution.
I can see a very big problem for a trans woman who has to use the men's where there are a few cubicles and many urinals. I think it is spiteful to insist that that is where they should relieve themselves.