I have been through similar and understand how you feel but as others have said, you must remember that there is absolutely nothing you have done/not done/ said/ not said that makes his behaviour your fault, he is a pervert, because he has pervert friends he thinks its normal, but its not.
you deserve so much more, a man who loves you, loves your body and the time you spend together, don't settle for anything less. Remember, its never going to get any better than it is now, and could quite possibly get a whole lot worse, research shows that as the brain gets used to one level of stimuli, it seeks out more and more extreme stimuli to get the same buzz, be it alcohol drinking drugs gambling or sexual. Kick him out, report him, those girls need people like you to help protect them from monsters like him, report him, and give the evidence of those others involved, that was a rape they were watching for entertainment, to the police. Yes your children might be angry with you at first, but think how angry they would be if they found out and thought you were complicit by your silence. Best of luck, be strong. There are thousands of us on this website who are completely behind you, backing you up. x