(((Hugs)))
If you feel this is the onset of dementia, you need try to get DH to be checked out by his doctor. If it is dementia, be prepared for it to get worse/DH to get more belligerent. And you will clearly need to keep him away from your family, especially your GC, even if that means no more babysitting. But you'll also have to protect yourself by arranging for nurses or nurses aides to come to the house and so on. Though it's hard to leave someone with an illness, if there's no love left, you might want to get out, anyhow.
If it's just that he lacks patience with kids, it may be enough to keep him away from your grands/see your family without him. As PPs have said, the parents may not leave you much choice, anyway.
"... and doesn't care that this upsets and offends me."
But this part makes me feel that it's more than mere impatience. If he's not suffering from dementia, then it definitely sounds as if he has a bullying, abusive side to his nature. This, too, is likely to get worse with time. In that case, IMO, you need to leave, ASAP, no discussing it with him.
I hope you figure it all out soon.