[Vanitas]

Sounds like your ex is just trying to ruin everything with your new boyfriend. I'd stay away from bringing it up to your current boyfriend, he may think you still have feelings for your ex and you definitely don't want that to happen or see your current boyfriend get into a fight with your ex. :/
I know ignoring isn't helping you and you mentioned that changing your number would bother you, but what bothers you more? Changing your number? Or your ex continuing to contact you and cause your anxiety to rise?
I definitely think you should get your number changed if it is causing you emotional pain like that...
Don't even respond to his texts and try to laugh them off, maybe even change his name in your phone to something funny or stupid to help ease the anxiety and not answer him.
You definitely don't need to let him ruin your life and don't stay stuck in the past, don't let him ruin everything for you. You're in control and you DON'T have to respond back to him.