I think Mr and Mrs C displayed poor judgement with their actions but the vitriol on here and the media, in general, is purely because he led the Leave campaign and is disliked by many. I believe if he had been an arch Remainer then excuses would have been found for him because he must be a decent all round bloke. As for the drive out to B Castle, it makes sense after a car being idle for weeks to do a test drive to check the battery wasn't flat before tackling the long journey home.
If he is forced to resign then all public figures, who travelled to second family homes at the point of or during lockdown and who are paid for by the tax payer, should go too. That includes the Queen, Charles, William, Boris et al, which leaves us with Harry as the Monarch and Queen Markle!