If we're talking about first hand experience..........
What's REALLY difficult is hiring men and women of a certain age. I found that hiring a woman into a male team at say, 20 years of age, can be problematic because - and I stress this is first hand experience - they have no issue dating other team members. This can be disruptive, and it's hard to stay on top of, especially if there's a break up. In another instance in my career I was working for a software company who had a Christmas party, and two of the sales team were younger women. One of them skipped most of the party because they were "entertaining" in their hotel room. This stuff goes on, and it can be consensual.
This has never been an issue for me because I've never been "the catch".

I strived to be equal to all my staff, which means putting up a bit of a wall. Staying professional demands you're a bit like a bank manager at all times, often you can't be yourself. Such is life.
Harrasment happens when someones approach is unwanted. But in my experience it's never as easy as simply stating things shouldn't ever happen between people. In fact, you can't stop it!