Is it rude for me to ask people NOT to stand behind my desk?

I just had a guy come in to get a quote, and as I was working up a price in my computer he kept coming around to look at what I was typing. I asked him twice if he wouldn't mind staying in front. Then he came up again and rested his elbow in right next to me to look at my computer. I asked him one last time "Please stay in front of the desk." and he took his papers told me to have a "good fucking day", and left.

I don't like people being close to me. I do not like hands being near me. Hands seriously gross me out. Plus this guy can't see some of my price lists, and I think that's what he was trying to do.

I'm feeling really frustrated and panicky now. I feel like maybe I was the asshole? How many times should I have to ask him to step away?