I cannot get over the thought that so many other women have known him, and that he had a casual, promiscuous approach to sex that I don’t really share. I feel it cheapens the contact I have with him now.
You're with someone you're not compatible with on a very fundamental personal boundary level. Unless you know how to change what has been ingrained into you as you grew up, then don't stay with him because if you haven't been able to stifle your retrograde jealousy and the negative thoughts on your own, then anything we say here to tell you how silly some of us may think you're being will NOT change your mindset.
Even if we say it's how he is treating YOU now that you should be looking at and not how he was before. Does he give you any reason to think he's not capable of monogamy?
“The willingness to accept responsibility for one’s own life is the source from which self-respect springs.” ~Joan Didion