Vintagejazz
Except that wasn't what was said. Again, writing your own narrative.
... are you slowly losing the will to live Vintagejazz? 
Leaving aside any criticism of the father, I think the point you're now having to defend is that you were reluctant to say anything because you weren't sure of the reaction you might get. Which some seem to think unreasonable, because if you're polite - they'll probably respond in kind.
I've no idea of any statistical evidence on this but going from my own personal experience (highlighted in a post somewhere above) being 'polite' will not necessarily prevent an over-reaction from some individuals, who may become verbally abusive or (as in my case) physically violent. It may not happen often, but it does happen - and that's why you were cautious.
As for the father - he might also be 'on the spectrum' (never really understood this phrase, but my own grandson has been officially described as such, so I assume it's OK to use it) in which case, some understanding is needed. But, how are you supposed to know this? Or he might simply have been one of those men (it's often men) who engage with children by jocularly teasing them / winding them up to fever pitch with clownish behaviour - to keep them amused and entertained. We have (had) one in my family - he'd come round for Sunday lunch, excite the children into a frenzy of excitement by acting the clown (not meant in a derogatory way) and then, toddle off to the pub with the rest of the menfolk leaving the women to finish cooking Sunday lunch and deal with hyped up kids looking for further excitement
.
I don't think you were being unreasonable.