As someone who lives in a Listed house (only a fraction of the size of the one this family owns) and who has been through all the procedures necessary for adding anything to the building, I am absolutely amazed that this family thought they could ever get consent for a building this size and so devoid of architectural merit, so close to the house.
One of the questions that needs to be answered when applying for Listed Building planning consent is whether the new building will adversely affect the setting of the main building and anyone looking at this new building that has all the charm of a council community centre, can see that it is oversized, under designed and so close to the house it damages the setting.
The reason they built it first and went then for retrospective planning consent is because they knew if they had submitted an application for a building like this they would never have got consent for it. I cannot see the council being able to do anything other than insist on its demolition.