Next time you go round, make sure you wear a mask and distance yourself as much as possible.
I am sorry that your DD sees social services as a threat - whilst their prime concern is the safety of the children (as I am sure yours is too) they can often be very helpful in this sort of circumstance, providing advice, support nd practical help....although the virus is hampering what they can do at present.
It is vitally important that your DD does not allow this man into her home. Try www.nationaldahelpline.org.uk/ - this is the national domestic abuse helpline. And Women's Aid is also good: www.womensaid.org.uk/information-support/
You can ring these yourself to get advice as to how you might best help your DD. You do not have to reveal your name.
It must be hard for you to assess exactly how much risk your DD and DGC are under; this is why you need external help. It does not all have to fall to you. And you do not have to place yourself at risk. You went round - this establishes that you care - so future communication could be via the phone.
🦞 Locked down no longer but still firm friends 🦞