I never worry about doing things on my own, although I rarely have the chance! If I do get a day out by myself I am fine with going for lunch, museums or whatever. I don't feel anyone is looking at me or wondering why I am alone, and even if they did I wouldn't care one jot, if I even noticed.
I have always been more invlined to do things on my own without a second thought, whereas DH would never go and do something without me. Perhaps because I was an only Child, and had to amuse myself most of the time, maybe that has stuck with me, who knows.
Just enjoy your coffee/tea/cakes or whatever and don't give it a thought