Hello Lizzie, after reading your most recent reply I must say that the main blockade for visiting appears to be your husband. Yes the distance is hard but it would be much easier if your husband decided to keep quiet and have faith in your driving, so much easier that, who knows, maybe you wouldn’t mind driving there more often.
I do think you moving closer will actually help quite a bit regarding what I mentioned above, however your husband’s behavior will always make it difficult. I mean no offense to you or him, however it is in his locus of control to stop acting as a blockade to your happiness. He really should put whatever it is that leads him to actively and frequently comment and nitpick your driving aside for your sake, his own wife for crying out loud. It isn’t to much to ask, not even close.