Clearly your husband didn't stop to think before he spoke to your nanny. The situation has obviously upset you or you wouldn't have posted here.
Some of the replies, I feel, go further into the realms of unfounded accusations than your husband's clumsy words.
Firstly, well done on caring for your grandchildren - I'm sure that can't be easy. We're exhausted just having one of them overnight!
I'm sure working while looking after grandchildren, and also having had to deal with whatever circumstances led to you having custody of them will have taken it's toll.
The idea of asking her to take the keys while you're away was probably well intentioned but not the best way to demonstrate trust. She may have found the suggestion a bit odd or stressful, in fact.
Anyway - if the relationship with her has been good in the past and, more importantly, the kids like her and are safe with her - try to slowly rebuild the relationship if she stays in the role.
Little things like wishing her a happy birthday if you know when it is, asking after her family members if she's discussed them in the past, expressing genuine thanks and praise for specific things she's done with the kids and so on might slowly rebuild the relationship (more than offering extra duties). Maybe a Christmas card ( with a more special gift than usual if you usually give one) expressing thanks for all she does, and gently saying sorry for upsetting her, might help as well.
Good luck.