Over time, the little things that bother you now will become gigantic. Don't discount them. The things you view as imperfect will become intolerable over time.
Also, you don't mention how old you are, but if you are still in your 20s, you can look forward to outgrowing him emotionally within another 10 years, because you will rise in strength and character, and your personality is still under development, while he is probably pretty much peaked out.
Relax... I'll need some information first. Just the basic facts - can you show me where it hurts?