If everything you say is true, I'm inclined to agree with Lite. I would really consider you caught this disease from him and he is being an ass from his own guilt and, probably b/c he knows its all over for him.
Please try to leave asap before he hurts you further. Don't be confused by your feelings about how you would LIKE things to be vs. how they ARE. You'll be fine, the sooner you get away from him the better.
Good luck. Post here for support if it helps. Find the No Contact with Ex thread.
Second thoughts can generally be amended with judicious action; injudicious actions can seldom be recovered with second thoughts.
--Cyteen by C.J.Cherryh