Either of you are allowed to erect and maintain a fence, it just has to be your side of the border. If your neighbor bought the materials and erected a fence on your side, even by an inch, then you have the right to ask them to remove it so you can put up your own. (Or you can remove it and return the materials to them) But even if it is on your side, you don't have the right to risk damage to it by fixing things to it without permission. So basically if it's a fence you paid for, then you can fix what you like to it. But if you accidentally put it even slightly over their side, they have the right to ask you to move it. That's why mostly there is the agreement in the deeds about who does what. However that is all worthless in the situation the neighbor who's meant to have the responsibility doesn't keep their fence in good repair. Thats why the fence on all sides of my garden is mine, but I put it up where the old fence was which was probably within the boundaries of the neighbor on at least one side! However, they've not complained, and I have not attempted to stop them fixing stuff to it. If later they want to claim it's theirs, then that's fine by me.