The truth is that people have different sexual appetites. Some people have high libidos (like yourself) and some don't (like your bf). That is a possibility and if that is the case then maybe that is a dealbreaker in this relationship.
But also consider this - there may be more going on than you know. At his age, he is probably dealing with multiple stresses, including his job, his house/apartment, any societal pressure on him for dating someone younger, etc. Stress lowers the sexual ability of people, and he may actually be having some difficulties performing. He may be too embarrassed to say anything, or may even not know what is going on (which is just as scary). The fact that you two fight whenever you bring up sex leads me to believe that he is being overly defensive and possibly going through something you aren't aware of.
The best thing to do, in my opinion, is to try to eliminate whatever stresses you can for him. Let him know you want to help however you can with things (without bringing up the reasons why). Stuff like that. Maybe take a vacation where he can relax and get away from the everyday things and just focus on you and he.
Good luck.
Brought to you by Dating With Devon!