A friend was having similar problems, Tegan. It started as a Friends with Benefits and lovely trips out and holidays - all great fun, then he started staying, then turning up whenever he liked even if she wasn't there and letting himself in. Things became very difficult for other reasons and the relationship is now over, but she misses having a fun SO when everyone seems to be in couples.
However, he was never resentful of her DGC and was very good with them.
It sounds as if he wants to be in charge and is jealous of your relationship with your DC. some heart-to-heart talking is required, I think, don't let it simmer.
As for not being invited to the pub, can't you surprise them by saying 'well, actually I'd like to come too for once!' Painting can wait for another day. Perhaps they just don't invite you if the answer has always been 'no thanks'.