This is an interesting approach I never would have thought of... lol *files away in the brain for if I ever have a serious boyfriend*
To be honest, OP, ask yourself why you're with him. Are you with him because of how you feel around him or how he looks in clothes? If it's the former that's the answer, then I believe it shouldn't really matter what you think of his clothes. But otherwise, all I can really say is that you need to really assess your situation and relationship if it matters to you so much and you're focused on his looks...no offense. That's just one person's view- whatever you do or think is up to you. Best of luck