I think most of us are saying similar things, it isn't about what -you want in this relationship. However hard it is to accept, they also have their own lives, with a baby of course they are busy but the answer is not to expect them to park the baby on you, they will want to do things together as a little family. Having a strop and threatening to not invite them is going to hurt you much much more. You seem to have your own preconceived idea of "grandparent type things, walks in the park, peekaboo". How about treating your DS and DIL like valued members of the family and adults in their own right, not just the procreators of "your" grandchild?
Ask them to lunch or tea, but don't expect it to be next weekend they do have their own lives, and stop fixating on your "rights" as grandparents. If this sounds unsympathetic I suppose it is because I really think you are on a collision course and still can't see what you are doing wrong.