In westrn society, the majority of us are programmed by religions (even if we eschew religion) to believe sexual fidelity is important.
In many cultures, infidelity is irrelevant and rarely practised.
If there are children involved, if the other partner will be hurt, if there is any chance of a sexually transmitted disease, then certainly it is wise to be faithful and thoughtful.
But these are options and there are many situations where people can be unfaithful (and not just having their socks darned by another woman!) and it causes no harm at all.
So on principle my answer would be 'no' but in reality, in western society, you reap the whirlwind if you say 'yes'.