No you're NOT being unreasonable if the smoke was blowing directly at you. And I say that as a smoker. I'm always very aware of being a social pariah and am so careful now about where or if I smoke if I am not at home. For instance, even if I go outside a bar or restaurant to have a "breath of fresh air" as I call it, I make sure to walk a fair distance to find a side road off the main thoroughfare so as not to inflict my addiction on others.
At a garden cente, I certainly wouldn't light up as there are so many others around, including young children very often! It doesn't take a lot of willpower to deny yourself for a couple of hours, surely? I was at an RHS garden show for the day recently, and found a very quiet spot behind a large trailer with no others in sight to have my fresh air quota. I still felt guilty. A bit like behind the bike sheds at school.
I am trying to quit, honest. I've downloaded Alan Carr's book and will try my best!