If you were childless, I'd say just pack up and go. But you're not childless and I think you owe it to your daughter to give this everything you can. Yes, you may have made more sacrifices than him, but this does not warrant a divorce. It was a choice you made by your own free will and you have no right to hold it against him.
Get some marriage counselling and do your best to work through it.
Oh, and lust doesn't last. It belongs in the honeymoon stage of when you first meet a person. That's not to say that you should have zero attraction either, but after a few years the lust fades to a more balanced feeling.
Never regret anything that has happened in your life. It cannot be changed, forgotten or undone. So, take it as a lesson learned and move on.