Nora

I can totally understand your problem with the situation, but it's not unusual to keep in touch with your former relationship. So maybe you should talk to him and tell him why you have such a big problem with his wife and him meeting each other. If you only think about it and just being jealous and petty he might think you're clinging too hard.
You have to show him, that his relationship is okay for you and you only don't want to lose him. I guess if you'll do that he is able to understand you and may have less contact to his former wife.