Well orcabel it sounds like you know what you need to do. That's good, that's half the battle.
I would have to say it sounds like he is either on his way to cheating or already has. if he had any respect for you he would have told you up front about his secret meetings. Since he never did, there's a reason he kept it to himself. I think if this is really want you want, and you don't want to fix anything and you feel like you have out grown him then you do need to get out fo the relationship. You deserve someone who can be totally honest with you!
If you can't handle the thorns, don't crave the rose!!