Feelings for my FWB.. Please read and help

Okay well this is a long intense story but I will make it quick and you can ask for details of needed.

I meet this guy about three years ago at my job. We both felt a strong sexual connection between us instantly. So we made a hook up type of relationship. But it seemed more than that pretty quickly. He would say I love you and beg me to be his even though we both already had people.. We stayed like that constantly seeing each other for about five months.. Then it ended because people found out

Fast forward to now. He showed up at my new job and obviously the spark was still there. We instantly went and hooked up in his car on the dead end road by his house. He asked me to be with him and that he missed me and told me to leave the other person I have been with for three years. I told him I couldn't because it was a tough decision for me. He kept begging me and saying he loved me. He also was with someone currently...(yes horrible situation I know).

I just texted him and he told me that he can't do this anymore. He isn't even with the girl anymore he left her. And he said he needs to think about his life and he doesn't want to hurt me... But i told him I'm sorry I can't date him... He is much older.. My family disapproves.. He isn't stable financially... He is a sketchy chesty person and I don't want that in my husband. Ironic because yes just recently we both cheated but our old flames took over...

Then guy I am currently with I love him so much and he means so much to me. He is my best friend and he does everything for me.. But I've been with him so long the sexual flame is completely gone and sex with him grosses me out and is depressing. He isn't experiences since I was his first lover.. And I can't see myself leaving him because I know it'll hurt him so bad. So I'm staying with him forever...
I was trying to secure a open relationship FwB with the other guy with everyone's consent obviously. But the guy just dropped tonight two days after we had Greatbsex a great emotionally filled night together.?

I just don't understand what could have made him just say no more all of a sudden. He was totally into the secret relationship thing I am so hurt because he was so important in my life and kept me alive... I tried to text him and explain my feelings for him and he just say goodbye... My hearts breaking.

- - - Updated - - -

I do remember one more detail of the other night in his car. While we were making love he looked into my eyes and said I looked sad. And I said I wasn't and that I was having the best night I've had Ina a while. And I was consciously thinking I felt bad about doing this because I remembered how I loved him and it was bringing it all back and I knew we couldn't be together. And maybe he knew that or sensed it idk. Do you think he is just wanting to settle down finally in his life? He just didn't seem like that kind of guy to me. He has three kids and didn't stay with their parents. Why now?