No one can tell you what to do in this situation. All I can say is if you take him back you had better not plan on holding this against him and you better be prepared to genuinely forgive him otherwise it will be a very rough road from here on in and there will be no improvements.
Think about it hard before you make any kind of decision.
To love is to suffer. To avoid suffering, one must not love; but then one suffers from not loving. Therefore, to love is to suffer, not to love is to suffer, to suffer is to suffer. To be happy is to love; to be happy then is to suffer, but suffering makes one unhappy; therefore to be unhappy one must love or love to suffer or suffer from too much happiness. I hope you're getting this down.