I've been wearing a sun hat ever since I burnt my hair and scalp while living in Sydney for two years. Not much use when it's windy but one of them rolls up so I can put it in my bag if I want to. After wearing one for years I don't even stop to think about it now. They protect your ears, nose, face, neck, scalp and hair and protect your eyes from glare. I wear them when I am gardening as well as out and about and often forget I am wearing it.
Mine both have a slightly rolled up, not too wide, brim. My newer one has a sunscreen coating. After 5 years of 'no, no, no' my OH finally bought one this year from M&S, also with a sunscreen coating, to protect his follicly-challenged head and after a few days of thinking he looked ridiculous now wears it without a second thought if it is really sunny. It was only because I was pointing out to him how many men were wearing hats similar to his that I noticed how many women of all age groups were wearing hats.
Try loads on, different shapes and colours until you find one that doesn't feel too bad. We only feel silly because we are not used to wearing them, and often don't look as silly as we think we do.