I don't know that he sounds crazy exactly, but he is coming on strong for someone you've only been dating a couple weeks. Not everyone is bashful about their fantasies/fetishes and that's fine if it works for all involved...clearly this does not work for you. I've had women I've never met and women I've only been on a single date with tell me about numerous sexual fantasies...clearly trying to gauge my reaction/interest level. That's how you get to know prospective partners. I was happy I made them feel comfortable enough to share their intimate thoughts...even when I didn't see myself taking part.
You can tell this guy you're not interested and it's not going to work out without getting into all the details. Just tell him you don't feel you're a match and it's not going to work out...which certainly sounds accurate. If he persists then block him from contacting you further.
Pain is what the world does to you, suffering is what you do to yourself. Pain is inevitable, suffering is optional.