It's very hard because on the one hand one always wants to have the right balance with neighbours. Such as being on the right side of friendly, looking out for each other. My immediate neighbour telephones us if she is going away and likewise we do too, I also take in any parcels for her whilst she is at work, at one time quite numerous but nevertheless she is always grateful and we don't mind if we are there, making sure to help out in an emergency. This to me is being a good neighbour.
There have been occasions, not in the house I live now, a previous one, when I was a lot younger and unencumbered by children. I had a day off and was enjoying a read in my garden on a sunny day. My neighbour who I didn't know that well as I was usually out at work all day, leaned over the fence to ask if I'd like to come and have a look at her new dinner service
I found the invitation so random and bizarre, had no interest in viewing a dinner service, stuck into a really good book, enjoying the sun, time on my own. However, had no bloody idea how to extricate myself from a very unwanted invite that turned into a dreary couple of hours discussing, or rather me listening to her running through a series of purchases and then showing them to me. Worse still being invited to something called a "Pippa Dee Event" one of those home sell affairs (clothes) in her home and buying something I didn't want under duress. Think I'd just say no! (politely) to both these days, life's too short.
I have a friend who lives on her own happily in a lovely cottage she's done up. Nice neighbours, however, one side keep inviting her to dinner, she works all week up in London, out visiting quite often at weekend so genuine excuse as to why she couldn't go on the date they suggested, thought no more of it, because frankly she doesn't want the relationship to go beyond "just good neighbours" Since then she's been bombarded with alternative dates, sometimes she's made excuses, sometimes she's felt she was hiding indoors to escape. In the end she texted with a "thank you so much for your invitation, but the dates just don't work for me, so best leave it, but thanks anyway" I think the penny finally dropped.
Goodluck OP follow your instincts.