No, not being prudish. if you are not into that he shouldn't push you into it just because he wants it.
That isn't a partnership to only do what he likes or wants, if you wanted to bring another man in would he have agreed?
I say, no he wouldn't. Most men wouldn't unless Bi because two woman is a straight guys fantasy with him.
Is it about him watching you with another woman or the fact he wants another woman brought in?
I don't think it's prudish, I wouldn't do it either.
It's not what you have, it's what you do with what you have that matters.