Except that the poorer countries would not have the resources to properly use and maintain all these new facilities and they would join the increasing number of decaying white elephant projects that litter these countries, a burden on their resources.
Better still scale the Olympics right down, spread the different disciplines across many countries. One country run the equestrian events, another the swimming, but all happening at the same time.
Each country need invest money only in one discipline, so more of the poorer countries would have an opportunity to stage one discipline in their country because it would be lest costly.
At the same time there would be less need for the huge expensive infrastructure projects that benefit the countries concerned far less than promised. How many of the flats in the athletes village in London are social housing?
They could cutback on those hugely expensive opening and closing events and the costs of the huge teams of freeloaders on the Olympic committees and in national governments that use the Olympics as an excuse for freebie holidays paid for by the Olympics organisations and the home governments.