You have seen more of him than I did with any of my grandchildren, but I have built a good relationship with them all. You don't need to be in and out of each others pockets to relate to him.
If you make a big deal of wanting more and more, you will put their backs up and make it more difficult for everyone to be relaxed and natural when you do visit. Enjoy him when you see him and keep his photo in your purse to look at and show off.
I don't see why you can't visit, or why they can't visit you? It doesn't sound as though there has been a row.
Have you actually invited them for lunch one Sunday? Told them of an interesting event that they might like to go to with you as a family? Asked if you could pop in to see you when you are in the neighbourhood on a certain day?
Have you thought that perhaps your son likes to look after his child, and drive him around? Many fathers don't do as much hands on with their chilkdren, so he is being a good father. It is their child, not yours, to look after.
It is not necessary for you to be as bonded as his parents are. You are a special grandmother, of course, but it is inevitable that you are not dead centre of his family circle. Accept that you had your own children and brought them up to have their own family in their turn, and don't sulk because they have a new toy that you want to play with.
They won't be more inclined to share it with you if you stamp your feet and go into a huff. Well, would you?