Yeah, he's not just joking. There would be no point in making a joke if he knows you're going to be hurt and offended by it. He's jealous of your friendships, and this is his way of expressing it. The thing that I find most problematic is that you've told him how much it bothers you, and he won't own up to it. One of the signs of verbal abuse is actually when somebody tries to convince you that they're just joking when they're really attempting to hurt you. You may want to do an internet search to look up other signs of a verbally abusive relationship. The problem here is with him and not with you.
“This planet has - or rather had - a problem, which was this: most of the people living on it were unhappy for pretty much of the time. Many solutions were suggested for this problem, but most of these were largely concerned with the movement of small green pieces of paper, which was odd because on the whole it wasn't the small green pieces of paper that were unhappy.” ― Douglas Adams, The Hitchhiker's Guide to the Galaxy