So many MoB outfits are in unflattering, often embroidered, pastel polyester and are, I think, so unlike what most of us like to wear in fabric, style and colour. I was horrified when I was looking, some years ago.
I think it is a question of adapting - if you usually wear trousers, then do so - with a lovely top or jacket, and in a colour that doesn't make you look like a birthday cake!
There is a thread on all this lurking somewhere, might be worth a look?