I have two suggestions:
1. I have discovered’ brown noise’, a derivative of ‘white noise’, to help with sleep at any time, not just when you’re being disturbed. It would drown out their talking, as it’s not disturbing no matter how loud it is. It works a dream with my husband’s snoring!
2. It may not work with ‘normal’ people, but we have students around us, and mostly they’re fine, but talking late in gardens is a popular thing. I just ask them to be conscious, that as noise travels at night, every word can be heard very clearly. This has never failed, and they soo go inside!
I think you do need to have some tolerance, but I know how disturbing it can be, even for one night. They may also be breaching tenancy rules. Have a word if it continues, and tell them you can hear them. Good luck.