If he wanted to get to know you as a person sooner, he could have just asked your mutual friend for your number.
BTW - I'm not "jumping on his case". I don't necessarily have an issue with casual sex, unless one or the other party is actually hoping for more, which sounds likethe case in this situation.