Never, ever ask a question you don't want to hear the answer to. Honestly, what was the point of asking? Were you hoping he would say something to reinforce your self-esteem? Because if you were, you should have told him you were feeling insecure and needed a confidence boost. Men (generally) don't want to hurt their partner's feelings, but they also want to be honest. Men have also been conditioned over time to NEVER answer that question because it always gets them in trouble, regardless of how they answer it. If you want to ask a guy something and have them give you the specific answer you want, don't ask them in the first place. He didn't say anything off-side or rude at all, but you took it that way because you're feeling sensitive and you wanted a confidence boost.
"Caring is not an advantage."