It is said that the vast majority of people massively underestimate how much money they will need in retirement, along with how long they are likely to live. Most of us these days are expected to live well into our eighties and beyond in a lot of cases.
You say you have a family history of poor health but does that include you and your husband?
If you are reasonably fit I would say it would be a very bad idea to give up work prematurely.
The State pension was never intended to be the sole income for retirees, just a top up, so if that was all you had to live on, you would be very limited in what sort of retirement you had.
This is exactly the time of our lives to treat ourselves to nice holidays, meals out, and to generally be able to spend money on things which make us happy.