I hear you Mojo. If I'm not going to get there, then I'm not going to get there. I'm not going to fake it because faking it is just a lie - but by the same token, I don't want him spending ages down there if nothing is gonna happen because it just gets boring. Is this what you're talking about? And yes, I too have said that I'm not going to get there.
I suggest you talk to him outside the bedroom and find a solution which suits you both.
Never regret anything that has happened in your life. It cannot be changed, forgotten or undone. So, take it as a lesson learned and move on.