In the recent shitstorm that came to be known as "Elevatorgate" (I surely don't need to put links here, you all know where to go), a lot of the arguments mostly men made went along the line of "for all we know he was perfectly benevolent. He accepted no as an answer. It's sexist to assume that just because he was a guy he wanted to rape her".
Well, apart from the fact that neither of us can know what would have happened in the room had Rebecca Watson accepted his proposition, I think you're getting this the wrong way.
You sympathise with the Elevator Guy (and after being dragged all over the internet I sure do, too), you identify with him.
After all you're a good guy. You'd never hurt a woman, you'd never drug her drink, you'd always accept her no. Therefore a woman would be perfectly safe with you in an elevator at 4 am, in a dark alley, in a hotel room.
Therefore you think it's totally ok for you to make such a proposition.
Therefore you're offended at the suggestion that she might think you to be anything else than a great good guy after you worked hard on becoming one. It's an allegation you take serious. I understand you, believe me I do*.
The problem is: we don't know you. We don't know all those wonderful things about you. We don't know that we'd be perfectly safe with you.
So let's try and shift the perspective a little.
Let's say the woman isn't Rebecca Watson or any other woman you might find interesting and want to get to know better. Let's say the woman is your 18 year old daughter who comes to you for advice.
The guy in question isn't you for whom you can vouchsafe, but a stranger of whom you have no more information than we do about EG.
Now, would you tell your daughter to accept the proposition or would you tell her to stay away because accepting might bring her into danger?
If your reaction would be the latter, you have reached the exact same conclusion you were offended by before.
If you wouldn't want your daughter to accept a proposition from a man she has no more information about than a woman has about you, don't make it.
*I'm German. I'm an anti-fascist, I come from a family who suffered greatly under Hitler. I work hard to fight racists and (neo-)fascists. Still, usually when being abroad, some people will treat me with caution, with a certain amount of dislike by default for my nationality. Yes, it hurts. Yes, it offends. But I understand why they do so. And I try to make them see that I really am one of the nice Germans.