So confused..

We met back in 2009 and we're off and on then got back together in 2011 and found out we were expecting 2012 with a little girl. From the start we haven't always had the healthiest relationship but I was so head over heels for him I pushed the big picture out of the way and was to concerned about the little picture. We rented out first home Memorial 12' weekend. You could say it was sort of a rough start again but once again so head over heels and at that time you could say I didn't have a back bone because I just let him get away with everything. It was cheating it was the drinking and him not putting my pregnant feelings in perspective at all. I cried Alot and he knew but nothing ever changed. I just always feel he's so rude and it's not just me who thinks that, his mom agrees too. I feel put down and belittled. Ive tried leaving him twice, once last year and the second time was last February. The second time I was done, I was ready to start a new life with my daughter and I. I just want to say I would never be that mom to take my kid away from their dad so I told him I wanted a schedule for when who has her and everything else. At first he was resentful to me when I left being rude and telling me that he will tell our daughter one day that it was all my fault that we didn't stay together then he 100% thought I found somebody else because "nobody just leaves for a reason". I never would cheat on him but this is what stumps me.. A month before I left him we got into our big fight they we always get into and I told him I wasn't happy and that if I continue down this path I will leave him. Of course it didn't hit home till he realized I was really leaving him. Then the water works from him came in and he wanted me back of course. Well I was pretty damn devasted myself but I was trying to be strong. Ive ways wanted a complete family but sometimes it's not worth it anymore. We don't have sex and I'm getting to the point where I don't even care. We fight all the time and it's usually because the way he talks to me & after so long I found my back bone so now our fights are even worse because I stick up for myself. Todays fight was because I was cleaning around the house and put his work belt with his other belts.. Well he couldn't find it so I showed him and he comes out with "this is the type of stuff I would like you to think about" ...like how rude?? Theres a million long list of things that urk me that's never changed but do I make him feel like shit about it? No because people don't deserved to feel like that. I just feel like a broken record.. I'm getting to the point where I don't care anymore. I ask him why did you want me back so bad? Why didnt you just let us go our separate ways? I'm mind blown because he seems like I am so annoying and Bitchy but he wants me?? Makes no since. I stay at home so I do nothing but clean and take care of our daughter all day. That's fine but I feel like I'm being taken advantage of. He can't even throw anything away or pick up after himself. He feels good about him self if he's done it 1 out of 10times..that's not how it should be. Even when I did work he never helped with cleaning. I'm at my Witt's end. I need an opinion.. Anything... I know this is all kind of rambled but I have so much frustration from 3 years no and don't know if it's worth it anymore. Even if I love him it's more because he's a perfect daddy to our daughter but not the best man for me...