When you meet someone who ticks all the boxes, someone that you are compatible with and have been happy with for four years, someone that your living with and planning to marry-it is very very hard to find that again with someone else. Your bf has set your standards and expectations and you said he is "fantastic". If you leave him-you will likely end up looking for someone just like him and every guy you meet will be compared to him.
You would be an idiot to throw that away for a "what if". Your friend is just a fantasy and if you leave your bf for him he will likely just be a rebound and once the infatuation wears off and you start to realize it was a fantasy-you will hate yourself for throwing something so good away.
We all have options. Theres plenty of people who may be interested in you-loads of "what ifs" everywhere but that does not mean that we throw something great away for every opportunity that comes along.