You're codependent beyond reason. Have an abortion and then get councelling for your low self-esteem. Contrary to what the "good" doctor above has to say, people who are not codependent LEAVE A MARRIAGE they are not happy in.. they don't have affairs. Nor do they blame their partner for their affair by inforcing ideas like he wasn't doing something right so I did something wrong. People with good self-esteem, good personal boundaries in place simply just leave an relationship that is not fulfilling. At best, they get couples councelling before they resort to an affair and if the therapy doesn't help then they leave. Those are the people with good self-worth, personal boundaries in place and a good sense of romantic boundaries that are in place to avoid such goings on.
If you're looking for permission to terminate the pregnancy then you have mine. BUT you need the therapy that will help you through that and get you in control of your codependency, shame, heart ache (from him not leaving his wife) and all the other crap that is consisting of this shit-storm you've created for yourself.
Google "Codependency" and "The Importance of Personal Boundaries" and start reading there... Call your doctor first and make the arrangements.
“The willingness to accept responsibility for one’s own life is the source from which self-respect springs.” ~Joan Didion