Sweet, talk to him and get your answers from him. There is little to no sense in just telling you to leave him. When you love someone it's often hard to see the forest for the trees so you (the general you) need to figure out that you're doing the right thing on your own. No matter how many people who don't even know you tell you to just leave him, you won't until you've satisfied yourself that you're better off without him, then with him. You are'nt someone who has ignored things for half a lifetime just to come here and vent but stay. You are someone who knows what they want and if you're not going to get it then I think you're smart enough to not settle for crumbs the rest of your life. If you are, well best learn to deal then because if after you talk to him and he doesnt' make an effort to change it up and be more attentive then he never will.
Think smart and logically and you'll be looking after your own interests (for a change).
“The willingness to accept responsibility for one’s own life is the source from which self-respect springs.” ~Joan Didion