First great grandchild

Golfer my Grandma left each of her four granddaughters a gold sovereign.
I got a gold mount for mine and wear it on a chain - the last occasion on my Golden Wedding day.

I like the idea of a gold sovereign with the year of your GGC's birth...
Or open a savings account (it would have to be in your name) and stash away a little bit each month for when the child is 18/21 etc. Update your Will, just in case...to make sure that the money goes where you intend it to if you don't make it to their birthday.
A nice piece of jewellery for when the child grows up (I had one half of a pair of pearl earrings reset onto a pendant fitting and put it onto a gold chain for when the child is older.)
Or invest in a really good watch for when they get older...
I know that modern young people don't wear wristwatches on a day-to-day basis, but it might be a lovely thing for a boy (or a girl) to wear on a graduation day or wedding day....

Things which are specifically "for the baby" will be grown out of so quickly, and toys etc likewise. I would go for the longer-term thinking.

I think the photography idea is perfect!

A personlised piggy bank is a good gift, sets them on the road to saving.
Or a handmade gift knitted blanket,crochet, quilt anything of that ilk.

Once you know your GGC's name, how about some personalised tableware, maybe something like in this picture?

Oofy

@Doodledog
A friend in Uni got engaged (and subsequently married) to a lovely chap, who came from a very posh background and had been given a “pipe” of port (no idea how much that is without googling it) laid down for him as a christening gift. Friend proceeded to drink her way through it as port and lemon.

?

When we were first married I used half a bottle of my husband's fancy port in the Christmas gravy. It tasted great, but he wasn't impressed . . .