Well there’s a lot of hesitancy (35 miles is almost next door to be honest) so like others I don’t think you are ready to move you did add at the end I m also struggling with the idea of selling my house, so your daughter and baby is only part of it
Ten years is a long time and if you have not felt the urgency to be totally together before now, maybe to stay as you are is the answer
He doesn’t want to move to you and you don’t want to move to him, simples, stay, but do be aware that the grandchild growing will pull you in more as the relationship between you will develop and your daughter may well have another child (what of the other daughter?) so I think you need to be honest with the chap, and in the same way he’s not prepared to move to you, you may never be able to move to him either