It's not clear whether he did fool almost everyone though - did they believe he was innocent or did they turn a blind eye because of the money he raised and his connections? Who knows?
Everyone I know says they knew he was a 'wrong 'un', but he still got away with it. Maybe it's a case of hindsight being a wonderful thing, but I agree that I felt there was something creepy about him. I never met him, and don't know anyone who did, so that was just based on his appearance and demeanour, as well as rumours.
I also agree that saying anything about people like Savile is fraught with danger, as there is a good chance that you will meet the response you describe, Primrose. People leap to the defence of men (possibly women too, but I can't think of an example of that) who have been accused by multiple people of the same crimes - often with very similar experiences, and anyone saying 'hang on - there's a pattern forming here' is expected to have fingerprints, DNA and CCTV evidence to back up what they say. Of course we don't want to see anyone convicted in Salem-style witch hunts, but equally, this attitude creates the atmosphere in which people like Savile can get away with their behaviour. It's difficult to know how to get that balance right.