I have never, ever, heard anyone suggest that all leave voters were racist. That is a slur bandied about by the liars of the leave campaign. I am pretty sure we all know folk who voted leave who are definitely not racists. So lets not keep repeating that slur.
What has become pretty evident, however, and has been shown in many surveys, is that all, or very nearly all, of the racists who voted in the fraudulent referendum, voted leave.
Just yesterday, in our local pub, I heard a leave voter, who I would not accuse of being a racist, say as much and he pointed to the topic of immigration and the effect of liar Farage's infamous "breaking point" poster as the deciding factor.