About the "disloyalty" issue. I don't think seeing YDS and family is being "disloyal" to your other DSs. Kudos to them for standing by your after DH's loss, but chances are, they got solace from that, too. Also, it may have been hard for YDS to be around you, at that time, if his brothers were frequently with you, given their estrangement. It would have been very understandable if you tried to get them to reconcile, back then, also. But IF you did, this in itself, may have upset YDS and driven him away for a while. I don't think you should be expected to "punish" him for that now by keeping him away.
Besides, the GC are innocent. But if you don't see YDS, for whatever reason, you're not likely to see them.
Do you still see your other DSs reasonably often? Then the little bit you see YDS and family shouldn't be an issue for them and probably isn't. Whatever choice you make concerning YDS, please try to let go of this so-called "loyalty" issue.