Get him to buy a small flat to rent out and give him some financial security.
Then sort your Will out and give him the rights to remain in your home for the remainder of his life should you pre-decease him, but leaving the property to your children.
This would remain the same even if you downsized.
You should seek legal advice about this because you would need to decide if he should provide rent to your children and sort out things like maintenance on the property and who is responsible for it.
The other scenario (however unpleasant) to consider is if you should need to go into a care home and how this would be funded. If you are forced to sell your home for this he would have no right to remain unless you declared this legally.
For those following Rory Stewart's personal processing of politics
Went to Boots today to buy some basic things.