Am I being over sensitive?

Ask your daughter about his comment over the phone. NOT to intervene or stand up for you, but just to make sure it really is ok for you to go over.

I would have come back at him with something like "It's ok I won't be moving in until you've painted the spare room" It sounds like it was meant light hearted.

If I were you, I’d let it go. It sounds like a joke from a dry sense of humour to me.
If you do bring it up, be prepared for your DD to side with her husband. I’d also avoid the passive aggressive retorts being advised and if you are genuinely upset make it clear in the moment “ha ha very funny” in a dry tone of your own.
Stop overthinking this - it was one comment and if it’s upset you so much that you’re now avoiding yourself DD and GS, maybe think about getting some professional help to deal with your feelings and fears associated with them.

I wouldn't take umbrage if I were you. If YOU know that you never take advantage of your status as his wife's mother to turn up uninvited or overstay your welcome, then assume that he knows that and was just being jokey at seeing you again.

If he says it next time, say something like "Yes, I'm still here. Are you going to invite me to move in again?" and smile. Then "Too much work here with a new baby for me to move in - you don't need to worry!"

If he IS joking, then you have answered in the same way. If he ISN'T, then you have shown that you can make light of it, and no, you have no intention of over-doing your visits.

Your daughter invited you. Go!

I hope she's not as sensitive as you or she'll be weeping into her coffee at this moment.

I asked my mum and she didn't want to come ?

You didn't mean it like that. But it could be taken that way. Why not give him the benefit of the doubt too.

I would stop making mountains out of molehills, she has invited you over , so go & enjoy it .
If such comments become more frequent or serious, then that would be the time to have an honest conversation about any potential problem, best wishes

I would ignore it at the moment, whilst filing it away for future reference.
I have observed that many relationships have come under stress during lockdown - a lovely neighbour made some very strange remarks to me (not personally upsetting, but critical and unpleasant about other people, very unlike him).
I also think that coming out of lockdown has increased stress as people cope with more worrying change in routine.

I think:
Your daughter's marriage may be under stress, she may want to see you and chat about it, but didn't want to spoil your first visit.
Your s-i-l may indeed have been 'joking' but is rather out of sorts at the moment, and dealing with his new way of working.
Whatever is going on - and it may be nothing significant, your daughter wants you to visit, so be guided by her.
If she is under stress, she doesn't need to cope with you taking offence, she needs you to be calm and helpful.

I would take the cue from her, and if there are other remarks, it may be time to say something then.