[Fey]

Let's start simple.
This is abuse.
And no one, even someone you love, has the right to abuse you. Ever. For any reason.

You are NOT giving up on him, but you are giving up on an abusive relationship. There is NOTHING wrong with that. You deserve better than that, and you need to recognize it.

What he is doing is most definitely manipulative, whether it's intentional or not. But no matter what HE is going through it doesn't give him the right to abuse you, even unintentionally. It's fine to love him, but you can't stay in a relationship just for someone else when it's hurting you mentally.

Inform him that you are there for him, when he can treat you like a fellow human being. Until that time you aren't going to be in contact with him because you deserve the same respect and care that he does, and right now you're not getting it.