No, you are not crazy to stay with him. In fact, I admire you. Clearly, your husband has some emotional issues and they are being worked out through having affairs. He must be a terribly insecure man.
The reasons that men have affairs are varied. Many wives blame themselves but this is misguided because no matter how strained a marriage is, having an affair is never justified. I think that you are a great mother. Your children may think that you are crazy to put up with your husband’s affairs, but as they mature they will understand that you did the best thing for them. Think about it like this; it is better for kids to stay in a home with married parents where one parent is unfaithful than to live with divorced parents. The harm that they endure from an unfaithful father is far less severe than what they would experience from divorce.
I would strongly encourage you to get some help for yourself so that you can stay emotionally healthy and strong. He may continue to act out because he knows that he can and perhaps he knows that you won’t demand anything better from him. I encourage you to toughen up. No, you do not want to threaten to leave, but you need to demand some respect for yourself and the marriage.
I’m sure that friends will tell you that you are crazy but also think about this: if you do leave him, then you are putting your children in a position where they will have to deal with their father’s habits on their own. They would have to go to his house and see him with other women and this could be devastating for them. And there would be nothing that you could do about it!
You are a good woman. Keep getting stronger and try not to listen to the negative banter out there. You are doing the right thing for your family.