Agree with other posters. Your house, your life, your family, your rules. Obviously the never ever giving mil a house key is now clarified. Mil has already demonstrated that she will abuse that privilige. Make it clear that you will not open the door unless she has checked with you first that it is convenient to visit. You do not have to explain to her why it may not be convenient. Allow babysitting at your house only when, and if, it suits you.
As you say the time for cosy conversations with mil has now passed but it is very important that you do have these conversations with your husband and that he understands the importance of supporting you totally. Your mil will sense any disagreement between you and use it to drive a wedge between you and your husband. Hopefully setting and consistently enforcing limits on your mil's behaviour now may save you years of conflict and pain. Your mil is a bully and bullies need to know when they have overstepped the line. Good luck. ?
Army horses loose on London streets