Perhaps you are portraying yourself as too subservient? If you lack the confidence in yourself and certain guys pick up on that, they really think they can muscle you into getting what they want. Just a thought. The whole "painfully shy" thing really got me thinking in that direction.
Don't beat yourself up too much about it. While it's true that you may be doing some things wrong and have areas you can improve, your self worth isn't in what other people think of you. While I don't want you to be cocky, you have many good qualities and you should really believe them about yourself. If you don't believe in yourself, how will others? Something I learned the hard way.
Waking up next to a beautiful girl,
Step outside and say hello to my beautiful world.