Heart goes out to you Welshmist 
With his history of not having been around for his kids as wee ones, he's clueless, and perhaps on top of that, forgetful/irresponsible, or as others have said, might have dementia issues.
Either way, it is too stressful for you. If he WANTS to continue to have your GC there, I'd make a rules list and put on the wall. Say it's a reminder for yourself as well.
Safety is so important... never underestimate the curiosity of a child.
A worst case scenario wass that my niece's 7 year old son was allowed by his dad (they were divorced) to sit on the tailgate of a pickup truck that was being reversed out of a drive. He fell off, sustained a head injury and died. This was in the US and my niece was away for her 30th BD weekend. The guilt has not left her after 16 years. She will not acknowledge her BD any longer, and never had any other children. The father did not blame the neighbour, but himself. The neighbour had a nervous breakdown over it.
Again...safety is so important... never underestimate the curiosity of a child. You cannot bring them back.