there is always one who causes problems and seems to rub everyone up the wrong way
That is not the same as being the devil's advocate, Societies need them and they are often perfectly nice people, just awkward enough to stop a committee getting complacent and stuck in their ways.
The problem causer is someone, who however effective they are, is just, plain, unworkable with.
Often it is the person who is treasurer. The treasurer's job is to run the finances efficiently and advise on monetary matters whether income and expenditure, but too often they seem to think they have a veto on everything a society does that involve's money and become unreasonable monsters, using their power over the money to try and shape the group to their own ends.
I was the chairman of a society where we had such a treasurer and I overruled him on some expenditure and I and the third signatory signed the cheque. The Treasurer resigned and then wrote to the bank saying that the society was financially in danger due to our reckless spending and we were not fit and proper persons to run it. then left the area leaving no forwarding address. It took months to sort out our relationship with the bank and replace him with someone else.
A few years later, we had a new treasurer that nobody knew that well. He turned out to be a bully who used his size to intimidate others. He decided that only he understood new Charities legislation and this legislation increased his powers. Committee meetings descend to shouting matches. It was awful. In the end the chair suspended all meetings until the AGM, when we planned to get him voted out.
Thankfully, he resigned before that happened. Since then life has been peaceful.