I've offered him anti bac hand wash while out
This is the only instance given in the OP that the man refuses to wash his hands. It's possible to wash ones hands perfectly thoroughly enough without ever using anti-bacterial hand wash. Soap and water.
I'm wondering if the man doesn't wash his hands at all (in which case, yuck!) or if he just finds the current fad about anti-bac hand-wash tiresome*. If he washes his hands with soap and water after going to the loo, before preparing food, and after travelling on public transport, I'm not sure you need to worry.
* I hate hand gels. They are sticky so one's hands don't feel clean after using them. Because they are sticky they leave deposits on anything one touches thus increasing the likelihood, I feel, of germs sticking to those places as well, rather like having dirty, sweaty hands. They often contain perfumes too, which make me sneeze.
I'm sticking with good old fashioned soap and water. Can't be beaten.