Your son and his wife might want their child's party treat at the zoo to be focused on the birthday girl with just her parents, siblings I she has any and the birthday girl's grandparents. If they are a bit jealous of your relationship with your daughter and her kids they might think that with your other grand children there, as you have a great relationship with them , the focus will be taken off the birthday child. We were always the more the merrier, but not everybody is like that and so often on Mumsnet you read of issues like that with dils comparing their children's relationship with that of the daughter's children.
I would go along to the zoo, have a good time, make a fuss of said grandchild and say nothing about your daughter's thoughts/wishes on the subject.
I would avoid as much as possible saying anything to your son about your feelings on wanting everybody there. Inevitably everything you say will be mentioned to your Dil who will put her own interpretation on what you are saying. I used to tell my own mother this when tensions occurred with her sons. You can say anything you like to me, but not to your sons. They will inevitably talk over everything with their own wives. Not necessarily to make trouble, sometimes to try to get said wife to try to please mum...but fireworks can result.
Incidentally are there other cousins on dils side of the family? If they are not invited then it would be difficult to invite your grandchildren and daughter.
I would leave well alone with relationship with sister. Invite them separately, enjoy them separately.
You can't make them come together for Sunday lunch and even if they did to 'keep the peace' inevitably there would be tensions and might result in a perfectly avoidable row.
Sorry this is long. Made me think of how my own mother felt similarly to you, thought she could openly say so ...but everything was far better when she didn't and has maintained good relationships all round. 💐