I'm glad things have simmered down, vq. Sometimes it is better to write things down ( I suppose it used to be on paper that could be destroyed but now it is on a forum like this which can't).
I will say, though, that I have got confused in the past when you have referred to 'our children' and must say that I would feel miffed if that is how you refer to them when speaking to their mother. That could make her feel rather jealous and threatened and indeed worried that they might love you more than her. Probably irrational but if they come back saying what a brilliant time they had with you and DH that could make her feel insecure, however unjustified that is.
However, it is wonderful that they have 4 adults who love them so much and would do anything for them and that is far better than the alternative, that you don't get on with them and resent the time your DH spends with them..
Children are very good, though, at detecting underlying currents however nice and smiley you all are together!
Do I recall that you have DD of your own - how do they feel about their step-siblings and your very close relationship? I hope they are fond of them too.