Why do the elderly people in my area complain all the time I am not entitled to the house I inherited from my parents?.

I am now 63 and had early retirement in 2008 and my mother passed away in 2007. I was widowed in 2004.
I worked in London and inherited half of my parents house and joined my father in our jointly owned house in Birmingham.
As it happened my 2 daughters went to Birmingham University and settled in Birmingham.
I found a part time job later that year and my father joined a bereaved club and befriended a woman who was also a member.
They went out together for about 18 months but my father did explain to her at the start there was no prospect of her getting any money as I owned half the house and his half was willed to me.
She suddenly came out with a statement at the end of the 18 months saying her family should now inherit the house and when my father said this was not going to happen she left slamming the door behind her.
Sadly my father passed away earlier this year and the probate is complete and the house is now fully in my name.
Every time I go shopping locally most of the time I run into somebody saying the house belongs to this lady and it should have gone to her.
Two people have even said I have stolen the house from her.
Most of the people are 75+ but why do they never think that the house was willed to me which legally makes it mine, it is now my home and I have spent a lot of money on it.