Unless the other couple have similar interests to yourselves it will always be a bit of a problem.
I think that you need to be honest with them and just explain that for this time, you would prefer that just you and your husband go on this holiday, but perhaps you might think about something else later in the year.
The men in situations like this always seem to be the ones that just go with the flow and it's the women that nearly come to blows!
Maybe there is a reason that the other couple go on holiday with them !
If you do go away as a foursome, you will need to just say what you have said here - that there are times when you would like to go sightseeing/ shopping on your own and as she is happy to sunbathe, that you don't really enjoy, perhaps in future you would split up for a couple of mornings / afternoons and then meet up for coffee or your evening meal . Either that or just say no when the subject of a holiday is raised!
Is a new relationship possible without sex?


.
. We did go away in September -on our own - but that was a delayed holiday to celebrate our special wedding anniversary. Maybe we should make it a thing to always go away (alone) on our anniversary. Maybe I could invent some other anniversaries e.g. day we met etc. 