We've had sex a few times, and now..

Long story short, a long time guy friend expressed his interest in me after my break up with my ex. He told me he really liked me and that he wanted me to be his girlfriend after things died down with my ex. He would invite me to dinner, to the movies, to go bowling, etc, but my parents, being extremely strict and wanting me to "take a break from guys for a while" after seeing my ex hurt me, wouldnt let me go out with this guy. I didnt want to lie and say I was going out with a girlfriend or something, because I'm a TERRIBLE liar and with my luck would get caught the very first time i would try haha... so, I resorted to sneaking out at night to see him.

Now, he knows the predicament Im in with my parents, and one night we were together, he even made a joking comment like "hopefully we wont always have to do this". Now, since its late at night and we're alone at his place, things usually get a little heated and frisky, and the first time, he even stopped himself because he said he really liked me and wanted more than just sex. Since then though, we've been getting more and more caught up in the moment and we go all the way.

Im kind of scared that Im giving him the wrong impression by only hanging out with him late at night, even though i cant really help it. I feel like I hear from him less than I used to, which worries me that it could mean that hes losing interest. I never get ignored if I initiate conversation, but still.

Did I put myself in a position where he's losing interest ?