I had a couple of nasty calls a very long time ago.
In one they were asking questions about, I thought, my Bro-in-law who was living with us. The questions seemed to relate to work and, as he'd just been for an interview, I started to answer them. After a while they seemed to go off the point and to get more personal to me so I stopped answering, getting a mouthful of abuse as a result.
The second purported to come from the head/deputy head of my son's school, saying that he'd had an accident. I don't know what they were going to ask but, since I'd collected him and brought him home for his lunch, it was clearly a hoax and again I got abuse.
It's not at all nice, Applegran but, as you suggested, more often than not these people don't know you. The calls are just as much as scam as the ones, e.g., that tell there's a problem with your internet. If you listen to what's actually said it's convincing but not quite accurate.