Is this an affair? Or is it even anything?

Ok... There is this guy that I've been hanging out with for about 1 1/2 month. He recently got out of a serious relationship and so did I. We mainly just cried about that to each other. But latley its gotten a little different. We talked about it and that we arent in love with each other, but its kinda more than friendship. So we agreed to just kind wait & see how things develop.
We're both old enough (23 & 28) and I guess we should know better than to pretend that NOTHING is goin on. We talk & text daily and spend every day of the past weekend together -havent kissed, but hugged & walked holding hands at a concert.
The thing is that his ex is really nosey and kinda wants him to stick around in case her new relationship doesnt hang out. He's not over her, but says he'd never go back to her... I dont want to start something too soon after our past realtionships, but its pretty dumb to say that nothing is happening... What I'm most afraid of is to become friends with benefits and I dont know how to clear the situation up.. .I guess I dont even know what I want -I want to be close with him, but I am not in love with him.
What is the right thing to do? Does it sound real obivious to everybody but me???

You answered yourself, everything that you are worried about is exactly how you should deal with it. Don't rush.

It isn't truly over until contact is cut for some time. It sounds like he can still go back to her if he keeps in touch with her. Especially if she is looking back to him as a safety net.

Just be careful and keep your guard up. You're both in vulnerable spots...don't let it progress any further until it's clear you have BOTH (and I say this to you as well...BOTH) have made sure you moved on fully.