Obviously I'm in a minority here, but I have allowed folk to use my drive in the past, and know several people who do. We don't at present have anyone who does so regularly, but a number who do on occasions.
As VioletSky says, it looks as if someone is in, anyone eyeing up the property can see folk coming and going.
It is doing a favour for someone who might at some point do one for you. To me it is part of being in a community. If parking is tight in the street it helps everybody.
I only do it with people I feel comfortable with (and maybe biglouis, you don't). We are clear about the conditions, and would withdraw if there were any problems.
My parents were very happy, for these reasons, to have a neighbour park on their drive most of the time. The day after my dad died, when I went to the house, her (the neighbour) car was on the street. She had been told the news, and to her, permission had ceased. I asked her to put her car back, both to keep an eye on the house and help ease congestion in the street.
In fairness, I think we operate on gut feeling, so I understand anyone who says 'no'. I just don't think it's daft to allow it.