If there are a lot of children to include, it can bump up the cost considerably, and most people who aren't loaded will surely understand that.
My dd had a lot of children at her wedding, but then we were extremely lucky in having a lovely big venue to use for free, with loads of outdoor space for them to run around in. However some couples did choose to leave their children at home anyway.
If we'd had to pay to hire such a venue, as well as for the food, booze, and everything else, I'm quite sure the guest list would have been considerably shorter, with only close-family children included.
As for babies, I think it's a shame to exclude them - it may often mean that the mother of a breast fed baby simply can't come. However parents should understand that a crying baby should swiftly be removed from the ceremony or the speeches.
Unfortunately some parents can't or won't appreciate this, and I dare say it's previous experience of a ceremony being marred by wailing, that makes some couples say a blanket no babies.