I don't know whether to leave and try to build a new life or carry on as we were on the surface, but with no sex - effectively a marriage of convenience. I hate him and love him at the same time and can't imagine my life without him.
Of course you mean YOU have a life without sex while he continues on with his extra-curricular sexual trysts. He's not going to give that up for you.
Why leave now? Just stay and allow him to have sex with others while you do the same. An open relationship. Google it (to get some idea of the rules and regs) and find a forum that caters to those who have marriages of convenience while supplementing the sex elsewhere. That or councelling to help you to get over your simple fear of being alone and then leave him once you've garnered the self-worth and strength to go. Leave him to his extra curricular debauchery.
“The willingness to accept responsibility for one’s own life is the source from which self-respect springs.” ~Joan Didion