I think the problem is there is no accountability with POAs.
The solution is to make it obligatory for POA's to be registered at the court when they are activated and for it to be obligatory for those exercising a POA to file audited annual accounts with the court.
I have had to deal with both POAs prepared while the person concerned was able to set it up themselves and those where it has been done through the courts, and those prepared in advance are so much easier and cheaper to deal with.
However I always kept meticulous accounts of how I spent the money, with receipts wherever possible and, as I was not a legatee, in either case passed these accounts onto the solicitor dealing with the will when the person whose POA I held died.
I have made a POA, naming my DCs as attorneys, they are also the legatees in my will, bar a few small legacies to godchildren. So, even if they did raid my estate, they would only be taking what they were due to get anyway.