Unsure Boyfriend - male advice please

Hello
I have been in a relationship with my boyfriend for 2 and a half years. He is 36 and i am 32, we have lived together in the past but currently are not officially living together as he tends to freak out after a few weeks and flees.
During our relationship there have been a few rough patches due to not communicating well and also because he has a lot of female admirerers which drive me mad. We have broken up once for 3 months in which time he slept with a few other girls, i found out through friends. He came back to me after some time away by his choice and we worked through our differences and were really cool and happy.
Latley things have been really strained we seem to argue over little stupid things, we get annoyed for no real reason and blow up about slight insignificant subjects. We have both expressed that we love each other and want to be together, but the little fights keep happening. Communication seems almost impossible as if i say anything that may offend him he packs up, turns his phone off tells me its over and wont talk to me at all. This has happened about 4 times in the last 6 weeks, we have an argument mostly about him not telling me about something like who was texting at 130am or what his plans are on a sunday and why am i not allowed to come along. I feel pushed away by him especially after times when we get really close and everything seems great. it is really confusing and i feel like im always waiting for him to make the call on if he wants to come back to our relationship. We have once again been in "negotiations" around the future of our relationship over the last 2 days, he tells me he doesnt want to see me but i end up seeing him and talking.
The last message he sent me today was " i want to be on my own this weekend. i need some quiet time alone to think about things and our future"
What does this really mean? Am i wasting my time?

OP, please read your post again. It's very clear to me that this guy just isn't the man you want him to be. You definitely are wasting your time with him. You're just an option for him. If he was so much into you during your relationship he'd be proud to make it known to everyone that he's taken. You feel pushed away by him because he really is pushing you away.

This guy won't bother looking at you again once he finds someone he thinks is better for him. Sorry but I don't see you and him having a future together.

He is using you and is probably starting the fights as an excuse to leave. Whether the reason is he's insecure, or he really doesn't care for you, does not matter. He just doesn't value you for a person. I think you can find someone who cares for you more.