In the case of telling you to go away, I think that kind of goes beyond bad manners. I’d like a child to be aware of how their actions can hurt or upset others. Anyone can say please or thank you just so they will get what they want! To me, the same goes for saying sorry. It’s very easy to give the child the message that they can do what they like, as long as they say sorry afterwards.
Mind you, I do agree it’s not your place to criticise, but if it really bothers you, I think it would be okay to set a ground rule for your house, something like ,’ we speak nicely to each other, and say please and thank you’. If the child ‘ forgets’, ask them to tell you the rule, then let you hear them speaking nicely, and give lots of praise when they do.