I was looking at some online photos of children playing years ago and their environment was full of dangers. Bomb sites, rusty old machines, broken pipes, rickety stairs. Not an adult in sight. Or country kids, playing in the fields, making dens in the woods....
More recently, my heart was in my mouth when my young grandchildren had a tree house built for them, but they were taught to climb properly and now at 15 and 13, are excellent climbers on real rocks in the Peak District.
I agree that 4 and 2 are young ages, but with the parents there, the best thing is to say nothing. If you are in charge, then full precautions are the order of the day.