I text him saying that I didn't think that we really clicked, but that might have been due to us being rushed as he had to be at work soon. He replied that he wasn't sure if we really had much in common... I said I wasn't sure either but was up for meeting up again and giving it another shot
Guys know that most girls often sound more positive than they actually feel, so guys have a translation filter that takes down optimism by 2 notches. You said "I didn't think we really clicked" which, when translated really means "I don't want to see you again".
IMO he's not that into you, except if you show serious, sustained interest on your own. If you make an effort he might come around, but since guys do 99% of the work in dating, he's not going to waste his time with someone who isn't interested.
I.e. If you're not going to put work into the date, why should he?
I have a long time interest in psychology, specializing in relationship dynamics for 30 years.
(Please note, we give the best advice we can based on the information given in a post. For better advice, please include the age of all romantic partners.)