My parents had a caravan on a farm for many many years. They always let others use it when they weren't there. They said it kept it aired and loved, and built happy memories for others who wouldn't otherwise get away.
They were, however, generous to a fault (!)
When, decades later, they parted with the caravan, they did have numerous invites from their friends to visit or use their holiday places - and they did so - in their holiday homes, caravans and cottages.
I think kindness generally come back to you.
If no damage is done by letting others use it, why not be proud of your ability to spread a little joy.