You can't assume that Andrew was caught up in everything that each person was doing. I would imagine that Epstein was aware of what went on, but each of his "clients" probably thought that Epstein was his own special friend and was just fixing it for him to carry out his own fantasies.
They may have wondered about the other people they met at Epstein's parties, and may even have interacted with some of them, but each of them would have been very protective of their own secrets, and so they would not have enquired too closely about anyone else's. Epstein kept all the correspondence to keep tabs on everyone, but it wasn't passed round like a Christmas round robin.
I suspect Andrew thought only of his own desires and preferences. He may even have been genuinely unaware of the age of the women and girls he was introduced to - sixteen or seventeen year olds can look like young women in their twenties, and he wasn't expecting to hold mature philosophical conversations with them. There is no mention in what has been released of the Epstein papers that he had any interest in really young teenagers or preteens.
I am not saying this to excuse him if he knew that the girls were underage, trafficked and unwilling, but do remember that legally he is to be considered innocent until proved guilty, and that you cannot say that he was involved in evil with "Not just women or teens and pre teens but toddlers and even babies" unless you have evidence that it is true. It could leave you in danger of a charge of libel.