I expect your daughters know what he is like and what a d...head he is to speak of you like that to them and don't appreciate how he behaves to you. For your older daughter - arrange to see her and your grandchild when he is not there. The younger ones must be at least 10 and there is no need for you both to be at the school so you need to take it in turns. Take a friend or arrange to be with another parent in case he shows up on a day that isn't his day. I feel for you. DD2 has PTSD from her abuse and spent a day in a cupboard as she thought she heard him breaking in. She now has padlocked gates and a video doorbell and had help from the local police over what to do. It is horrible and so sorry you are in this situation.
New house and a sloping garden