Trespass is a difficult kind of crime to prove legally, and I think you will only have a lot of trouble if you try to stop a neighbour from entering your garden.
I agree that it is impolite of her to do so, and very odd that she asked you to take fence panels down, so she could get into someone else's garden from yours. I would have refused to countenance that either.
However, I do understand that the lady could be worrying about a missing pet, but the long and the short of it is that if you don't want her in your garden when you are out she ought to respect that.
Whether you have a legal right to try and keep her out, I don't know. I believe it depends on where you live. In Scotland there is no law of trespass, as far as I know, but England is a different matter.
Has she done any damage to your property?