Abbey, for you there is a middle road, which is the one I took.
I was very lucky, I had a major mental break down at 55, and after that found work very stressful for a variety of reasons. At 57 I was made redundant, which forced my hand. My boss at the time, the financial director, help me decide the best way forward (I respected his judgement).
I decided to take all the monthly pension on offer from that day (no lump sums). I continued to work on contract for a few months, but the decided I'd had enough and we (Mrs P and I) decided that we could survive somehow.
And here is where the middle ground comes in, I found a part time job with Age Concern as a carer in a day center, this only paid minimum rate, bit was stress free and very worthwhile, hence enjoyable. Being part time 3/4 days a week it gave me time to enjoy other things. Whilst this still left us worse off financially at that time, it gave us enough to live on.
Going from a senior position in IT with a multi national company to being a carer was such a big change that it never seemed like work. During my 6 years with AC, I laughed and smiled more that I had in the previous 34. Life is too short to hate your job.