I was moved twice in primary, my last move in the january of yr 6, (not first day back after Christmas either), I was very shy and did not have lots of friends, but a couple.
I was the only child from that school to go to a large comp, although I passed the 11+, my parents believed in the New Comp ideology. My point is that when we go to college or uni and work, we don't go with friends, we need to become confident in our ability to make friends. I am aware of his issues of anxiety, our GS has them, but talking to school pastoral tutor, hopefully will help. Other good suggestions have been made too.
We do try to wrap them up in cotton wool, not sure that is the best way.
My son went on holiday with his dad every year, as we were divorced recently he told me he hated it...... so there is no easy solution, but keeping on good terms with exs when there are children is the best way IMO.