How distressing it all must have been. Right now, try not to worry about it too much.
The baby's birth and then serious illness has been upsetting for you all, so perhaps your son's anger was mainly due to that. I do hope the baby is progressing well.
Was he only drinking to cope in this stressful situation, or is he a habitual drinker? If the former, let it go, if the latter, you might later need to talk to him about it.
Don't make any decisions about moving nearer to them right now. If you do decide to do so, I suggest you have a serious, quiet talk to your son about guidelines. If you are to help with the children you need to know his and his wife's ground rules, but you have the right not to be yelled at if you make mistakes, and certainly not in front of the grandchildren.
If you do talk to him, try to make him understand that whatever happened when he was a child may have been the wrong decision to make, but presumably you found it, if not the right one then, the only one. He needs to move on now.
In hindsight, ten days is perhaps too long a visit.