Cats facing up to each other, should I worry?

I had an Alpha tom cat who had a huge territory which it took him all his time to control and as he was run over when he was 6 years old, he never reached the stage when he had to concede it to the next Bosscat.

Now I have a neutered Tomcat who had trouble as a kitten just looking after our garden which is his territory. We decided early on to help him and everytime the new Bosscat called round we threw a bit of earth and clapped our hands. This was started 16 years ago and after this help in the beginning, our cat has prevented another cats from coming in through the cat flap and so into the house.
We acted on the advice of our 'Know your Cat' manual.

Because our cat is so gentlemanly and polite he has a large fan club of single ladies who feed him tit bits ( forbidden by me but ignored). We celebrate his birthday every 5 years with garden party and I can thank them for looking after him when we are away.

My daughter's next door neighbour shouted over to her when she was hanging washing out. 'Do you know Eric (Cantona - don't ask!) often comes through our cat flap when he can smell bacon?' My daughter said 'sorry - just shoo him out - he has plenty of food here.' 'Oh, we don't mind - I just wanted to give you this photo.' Neighbour produced a photo of Eric fast asleep in her bed, head on pillow and covered up!!