Ya i get what your saying but just because he feels bad doesnt mean we should be nice as pie. A lot of cheaters feel bad at first but it doesnt stop them going back for more. Most people who start an affair feel the xact same as the OP and try to stop it a few times but eventually it gets to the point where they cant deal with the guilt and start blaming their partner, or the OW/OM. Anything to justify in their own head what they are doing until they eventually forget that they ever loved their partner and walk out the door.
Or even if they do stop it straight away-it doesnt stop them thinking about the other person a lot and wondering what if. And when your thoughts are somewhere else thats enough to cause many many issues.
He doesnt get a free pass for feeling guilt coz most cheats feel guilty at first and it doesnt stop him wanting more.