No, it's not normal. I bet he probably tells you you're being "jealous" and "insecure" when you bring it up too, right?
That's classic guilt. He knows he's wrong so he's going to project it back on you to justify it to himself and make you doubt your position on this. Don't doubt it for one second. If he has a closer relationship with this other girl than you, or even an equal amount of closeness, he's gone too far and has, as you've said, a 'crush'. Not appropriate in a marriage.
Tell him his behaviour is inappropriate and it's affecting your relationship. If he gets angry at you or tries to turn it back on you, you need to take the hard stance and leave. Seriously. My ex-husband cheated on me with a few women, he did this with every one of them.
Live as if you were to die tomorrow. Learn as if you were to live forever. - Mohandas Gandhi