You need to talk to your husband and tell him you’re going.
Can he understand that his behaviour is spoiling this for you?
He needs to show that he accepts you’re going and not make your return miserable. You mustn’t waste time, on your precious break, worrying about your reception when you return home.
Grandmabatty may be right, there could be awkward phone calls during your stay and you should be ready for that.
Can you make it clear that you’d rather you went as a couple, but if he doesn’t want to, you will still go. Be firm with him, it’s a fait acompli.
My partner and I have occasionally had separate breaks/holidays with family members and it has worked out fine.
I hope all goes well. ?