When I used to date, I always pursued the hot girls. Despite being an 5'8" 145 lbs nerdy guy, I usually got the girl.
I don't like being shot down, but if you don't ask, you don't get.
You're right, most guys are scared of you. Don't be fooled by the really hot guys, assuming that because they are hot they are confident...most of them are not. In fact, the hot guys are usually more scared of getting shot down that me--Mr. Average Looks.