bonbiazon, he's been very clear about not wanting to try again with you, so you need to work on accepting what he says. My best advice is to move on and drop him from your life as far as possible. There are a couple of reasons to do this:
First of all, it's disrespectful to his girlfriend to have you hanging around as a supposed friend who wants more.
Second, you won't get over him while he's still in your life. Seeing him with another girl will only add more torture to how you already feel.
Never regret anything that has happened in your life. It cannot be changed, forgotten or undone. So, take it as a lesson learned and move on.