You do not know what the future holds. You are still relatively young and not yet retired, so you may actually need the money yourself, given the way that state pensions are going, these days.
But you are old enough to think about what you want out of your retirement. My MIL sold up in Richmond and moved to Cambridge where she had all of the same facilities for far less, and then to Ely because she had made friends there. Each time she released a little equity to enhance her pension - being single, her income was less than a couple - which happens to far more women than men. Do get a pension quotation to check if you have not already done so - could you manage on just your income?
This may influence any decision.
Then look at your lifestyle, are you heavily invested in your community? is your family close by? or would you like to move to a smaller town, or perhaps by the sea?
Then look at the housing market, remembering the cost of moving, to see if downsizing to your chosen/current location is worth the effort.
Would the amount you can release be worth the change for the children anyway? (I live in a small town, with many Londoners, in walking distance of everything, with plenty to do, 1hr from Kings X, my 4 bed modern bungalow is worth only about £300K). Research is vital.