A couple of months ago I hooked up with my colleague and we have flirted back and forth at work, and he has been at my place a couple of times. He's the easiest person to talk to, and it seems as though he really likes talking to me, both in a flirty way, personal way and professional way. He's a very "selfish" person though, he's said himself that he prefers to "let the people you love go" rather than fight to be with a person (he got really hurt a long time ago). The last time we hooked up he told me to not fall in love with him and I promised him not to.
Lately, I feel like everything has been different between us. He's always greeting our co-workers but with me he always comes into my office, sits down and smiles and asks me how I'm feeling and how I'm doing. He's calling me almost every day at work to chat and then ask a work related question which he could've easily figures out himself. We haven't seen each other that much lately because he's traveling a lot but he's always telling me about everything he's going to do in the next few days and how his trips has been.
Today we were the only ones at the office and he knows I've been having a hard time with my family lately so he asked tons of questions and was genuinely (or so it seemed) interested in everything I told him. He then asked me how my sex life was, and he has never asked anything like that before so I simply said "maybe" indicating that I've slept with other guys after he and I hooked up. He got a bit distressed and asked me what I meant by maybe and I just looked at him and asked him why he wanted to know to which he replied "well, I need to know this kind of stuff". He got up and left to do some work and after a while I came into his office to hand him something and he said "so this maybe sex... Is this a boyfriend or what?" Again, I just laughed and didn't say anything. Later on I told him that he's starting to get grey hair and he replied that it's my fault because I exhaust him (with a light, flirty tone). Right before he was leaving he came into my office and told me that he is starting to get grey hair, and out of nowhere came over to me and kissed me (not in a passionate way, just a quick kiss on the lips) which he has never done before.
He's the most diffucult person to read in the world, and I don't know if he's doing all this stuff to have sex with me again, which doesn't make sense because I texted him the last time and he told me he didn't have time (he is extremely busy though, he works 14 hours a day and has a lot of hobbies and friends). So any guys out there who can help me, pleeease reply because I don't know what to make of the whole situation and how careful I have to be.