I agree about less/fewer, different from/to etc, and can add to the list
'Meet with' and 'work colleague' are irritating me just now, as a friend of mine uses both a lot, yet criticises the mistakes of others.
Times change, and language evolves, though. The examples above don't reduce the number of concepts we can discuss (do they? I don't think they do), or cause confusion about what is being said.
Mixing up 'infer' and 'imply', on the other hand, or 'uninterested' and 'disinterested' genuinely alter the meaning of what is being said, and are more likely to impoverish the language than choosing the wrong way of expressing something, such as 'would of' instead of 'would have'.