Sounds like he feels guilty about having bad thoughts about you. If you want to keep trying to tell him you understand and are ok with his "bad thoughts", but I also think he's not over his ex....