Boyfriend lost his sex drive, feeling rejected

Hi sorry this is long, I'm having problems with the intimacy side of things with my boyfriend and I'm worried I'm pushing him away. We have been together just under a year and 5 months ago he started taking very strong medication for his mental illness, this affected his libido massively. We went from having sex 4-5 times a week to once a week but only if I initiate it.

When I first met him he wanted sex ALL the time, morning and night, he told me that when I'm not around he masturbated 4 times a day to his favourite porn sites. This never bothered me before as we had regular sex but now it really does because that part of his drive doesn't seem to of been affected. He seems to still have the ability to masturbate. Now it's not me and he wants children one day and never wants to lose me and he finds me very attractive because he never stops touching me as kissing me so just to clear that up. But i know he still masturbates, dare I say more than we have sex now. I know this because of his internet history, porn sites pop up on his phone and I've argued with him about it each time.

I know the medication has affected him and we do still get intimate but it's only once or twice a week (twice very rarely) when we do it's mind blowing and adventurous and he said he finds me incredibly sexy, he said he has never had this problem before and feels less of a man for not being as into sex anymore.

But it seems that porn is the only thing that can still get him going now and it's making me insanely jealous and depressed and I can't go on like this. It's got to a point where I hate working opposite shifts and I never want to leave him alone in the house and that's not healthy!
I find myself trying it on with him at very bad times, when he's done a really long shift at work, when he's feeling down from his illness etc. I seem to make matters worse, when all I want is to feel closer to him and make love, he doesn't see it like this and keeps accusing me of treating him "like a piece of meat" he wants me to just hold him and hold hands and when I'm am wanting to get intimate I'm looking like I only want one thing in this relationship when that's not true at all, I love him so much but my sex drive is just very high, I want sex 4-5 times a week and I only try initiate it 3 times a week and it's making me look like a horny school girl with a one track mind.

If he rejects me I get upset and feel bad about myself, which in turn makes HIM feel bad, I know it's just his medication but I know he can masturbate so it all confuses me. He sometimes plays with me and it's almost like he's teasing me and I'll think that he's up for making love so I'll kiss his neck, then he will turn around and say "later" as in later on in bed but then it doesn't lead to anything later on and he will just go to bed to sleep which I find isn't fair considering the whole situation.

He recently confided in our very close friend about it and told him that he fancies me like mad but his meds have ruined his life and is considering lowering the dosage but he's worried that if does so he will go crazy and have manic moments again and I'll not be able to live with him. Our friend had a chat with me abut it and advised me to back off a little and just cuddle him, he asked me to promise I wouldn't tell my boyfriend that he told me that they spoke about it so I did. What bothers me is he won't talk to me about it because if he did I would feel a whole lot better!

The other night we were in the living room and he kept playing with me, what you would consider light foreplay and he knows that gets me going and he said "we will go to bed early" then I saw him grab his phone and go to the toilet, he came back and his phone was on silent and I instantly presumed that he had masturbated in there, so I tried to kiss his neck but he brushed me away asking for me to just hug him. I told him I thought we was making love tonight and he got mad and said I'm making him feel like a bad boyfriend and I keep doing it even though I say I'm sorry every time, he admitted that he dreads coming in from work because he feels I will just pounce on him again and expect sex and I'm doing it several times a week now and it's getting too much, then he told me to go away. This morning he didn't tell me he loved me and I'm so worried.

Now I'm only presuming that he masturbated in the toilet because I just have that gut feeling. This subject is making me miserable, it's actually consuming me and I've become obsessed. I'm worried I'm pushing him away with me being so obsessed over the subject, that I'm turning his idea of sex off even more with being so pushy a couple of nights a week.

What should I do to make things better? I know I should of backed off ages ago when I first knew this was going on but I feel we won't have sex at all if I don't make a move. I feel so unattractive and I've lost my confidence, I've worn things that he has suggested that I wear that turns him on and he just drools at me and then doesn't do anything, it's so frustrating. Every time I bring the subject up he gets annoyed at me and it's starting to become something he expects which I don't want because I just want to sort things once and for all! What can I do to change things? When he won't bring this up with me and goes and tells our friend instead, I feel I'm really really annoying him every time I speak of it.