chocolatepudding I am a great believer in basic good manners, and I think from what you have written, your goodwill has been taken for granted.
I think there are two issues here, namely:
1) Your neighbour is being paid, for your goodwill.
2) YOUR land is not being maintained in a proper manner.
I think that the fact that the first informal arrangement now seems to have been passed over to the next generation, needs to be addressed. Perhaps contacting them and arranging a meeting?
However, although I don't know much about the legal side of things, it might be that if they have had unrestricted use of the land for such a long period, they can claim some sort of right of usage.
Perhaps you should seek legal advice, but of course it is very sad that a "gentleman's agreement" should have to resort to this.
Good luck, and every good wish to you.