I know first hand how much this predicament sucks!
IMHO it comes down to this. If it went to hell could you still work with this person? i.e, proximity matters! If you have to work really closely with them it's different to crossing them in the halls once a day.
Loads of relationships start at work. There's no reason why it couldn't be great. Particularly if you take things slowly, it gives you the opportunity to suss eachother out and end it if it looks like a 'no go' before things get too involved.
You never know............................. until you ask!