From my perspective, communication is key. The Grandparents should of at a minimum included one of you, better both in the conversation. Personally, especially with a first trip away from mom and dad, I would have approached you and your DH first to discuss plans and to figure the best way for my grandchild(ren) to be the most comfortable with the trip and to figure the best activities.
That being said, though they perhaps did something uncomfortable for your daughter and for you, I think you should address it non-confrontationally, bring it up with them in friendly way, maybe as a way to help them succeed with their GD. This would open up a discussion, letting them know that you aren't opposed and opens an opportunity for you to set boundaries. I suggest not to stringent of boundaries, but things that would make you and your children feel happy and safe. I think you might find them pleased and more open with you this way.
These are just my thoughts and advise, use what you wish, or not at all. But please be happy and find joy with your children and your in-laws. You all can be an extended family team.