We often drop in unannounced at our DS & DDiL's house, as they do with us.
Everyone knows that if it's not convenient, we'll just say so, and the visitors will leave immediately. We would never dream of walking in using our keys, unless it had been particularly requested that we do.
However, as they live less than 300 yards from us, (or 60 yards as the crow flies) it's not a problem.
As we look after our DGS three days a week, we often buy ’big toys' which live at either house, so we all have keys to both houses, to enable us to pop in and get stuff whenever we want.
We would never go 'nosing about' in their house, and we know they'd never do so in ours either.
We're lucky in that we have a very, very close relationship with our DS, and DDiL is very 'family minded' too, but lives an hours drive from her parents, so is very much 'ours'!
I really feel for people who are estranged from their children, I don't know how I'd cope, probably wouldn't!