Sounds as if this couple were impatient to be gps long before their dd ever became pregnant. Or maybe they were very close to sil when he was growing up and feel as if he is their son. Either way, it's very generous of them to be so good to your gc. But, imo, they were also filling some need of their own.
"Problem is they refer to the children as their grandchildren, even in front of me and my husband and other set of grandparents"
Do they do this in front of dd and sil, too? And are they ok with it? If so, then maybe it's better left alone. If not, then, imo, they should have nipped it in the bud, however gently. Kind of hard for anyone to tell this couple now, "No, you're not the gps, even though you've been allowed to think so for so long."
If you do say something, I would take MawBroon's suggestion.