Always! I can usually smell/taste them in mugs and I don't want to ingest detergents as they contain chemicals that may be harmful.
About 20 years a go a friend, who is a GP, said that research into early male baldness indicated it was linked to oestrogenic compounds in drinking water - largely from the contraceptive pill and detergents.
Here's a section from an article which you may find interesting:
The findings from the four year study, led by the universities of Exeter and Brunel, has important implications for understanding the impacts of these chemicals on ecosystem health and possibly on humans.
Endocrine disrupting chemicals (EDCs) disrupt the ways that hormones work in the bodies of vertebrates (animals with backbones), including humans.
They can be found in everything from female contraceptive drugs and hormone replacement therapy pills, to washing up liquid, with the most well studied EDCs being those that mimic oestrogen (female hormone).
EDCs have been seeping into rivers through the sewage system for decades and have an observed effect on fish, altering male biology to make them more female -- hence the 'gender bending' reputation of these chemicals.
The full article can be found here: www.sciencedaily.com/releases/2010/10/101026111613.htm