Had great connection, then he withdrew..

Here's my situation. Met a great guy, we had an amazing mutual connection. He lives in another state and travels for his work. He told me he had never met anyone like me in his life. We immediately hit it off the night we met and spent the whole time together. He immediately texted me afterwards and said he had never met such an amazing, well rounded person and that he smiled the whole way home. He called me two days later and we talked on the phone several times for 3 or 4 hours at a time. It was amazing. 2 weeks later I went out to visit him for 5 days. We had a great time and it went perfectly. Spent a lot of time together and got to bond. On the second night I was there, he asked "what are we going to do about this cosmic connection" I didn't really know what to say, so I said "I don't know, what do you want to do?" I know, not the best answer. He said "I think we should have fun" Not sure if that's what he meant or if it was a defense answer because I didn't sound too assuring. Regardless the rest of the trip was extremely affectionate and wonderful. So I get back and he starts to call/text less. He comes into my area 4 days later and we spend one night together and he asks me "so what are we doing? what are you thinking right now?" Again, I have a lame answer "well I'm thinking about you and the situation. Well, what are you thinking?" Again I turn it on him. He says "we I'm in a hectic time in my life, traveling all over the place. I don't want to drag you around emotionally. I've made the mistake of rushing into things and I don't want to do that. But I'm optimistic about us" That was a month ago, and we've slowly communicated less and less. I've initiated most of it, but he always responds immediately and will always pick up the phone no matter what. Also, I have not had sex with him. He wanted to on the last trip and I told him, no, not yet. Now I'm not sure what to do. I know he's reeeally stressed with work. I'm in love with him but I don't how to communicate and I feel like I sounded uncertain when he tried to bring it up. Advice??