Oh dear! Like so many here have said, the most troubling aspect about your post, is how your husband has banned your own daughter/granddaughter from visiting, against your will.
Your H is not your D's father or gd's grandfather, so more than likely there could be some ambiguous, negative feelings lurking about (jealousy?). Your gd on the other hand sounds both responsible and trustworthy, so why the ban?.
No matter the motive, your H's ability to control you is unacceptable and his demands are completely unreasonable! This type of behavior is not driven by love, but by selfishness and insecurity
How dare he presume to separate you from your own children? For your own happiness and self-preservation, you need to put a stop to this. Call your daughter ASAP and invite them over. Remind him they are your family too and he has no right to keep them away.
If he doesn't give in, you may have to think about what kind of man you're married to and whether or not he's worth keeping.