No I did not read completely to the end because it's 1 am here so I read half way and skimmed your questions. I think partially you are just attracting the wrong girls because of your first impression not matching your full personality very well. There's not a whole lot you can do to stop this and all I can suggest is to keeping waiting for the right girl to come along. You are still very young and women are going to mature and understand more what they want as you get older as well. With experience some will learn to judge your personality better so your odds only go up the older you get.
The 2nd problem I'm seeing is that maybe you are being a little too much of a push over when claiming to be yourself. If you don't like a movie say you don't like it and why and then say you are still willing to watch it since they are interested in it. If you don't like a food say you don't like it but you'll eat it sometimes since she does like it. Most girls and the ones you probably want a relationship with don't want you to happily follow along keeping your opinion to yourself. They want you to state your opinion but then you get points if you go ahead and do what they want sometimes anyway. Not all the time or it gets annoying. You have to occasionally say no I'm tired of that I want to do/eat this. From experience agreeable slaves are boring and I lose interest in them quickly. You can be a nice guy while still having an opinion and contributing to plans being made.
Girls are subtle and guys are clueless. It's a fact of life. Girls drop hints and guys fail to pick them up majority of the time. Number one suggestion I have for girls that are having trouble communicating with guys is stop using hints and just state everything you feel and want in simple statements. It's the only way guys understand. Especially when it comes to starting and ending relationships. Some guys you just want to whack alongside the head cause no matter how many opportunities and flirtatious comments you make they still don't realize you are interested. Then you end up with his best friend who you aren't as attracted to because he was too clueless to realize you were interested in him. Then you have to break up with said best friend who you were never that attracted to in the first place but after months of trying to drop hints and let him down easy he still has no clue so finally you get the guy you've been talking to and would like to date to tell the other guy it's over because apparently they can only understand guy language and girl language is too subtle. Yes that is a real situation. I can name another 2 dozen similar ones.