AIBU not to want to retire yet be exhausted and to find it hard to fill/enjoy my free time?

Thank you for your replies! Will investigate medical possibilities.

Perhaps you should try to find out why you are exhausted? It could be iron deficiency, thyroid or whatever, or mild depression. Don't let it rule, deal with it! I felt rather like you when I stopped working, but after a few years of not doing much joined a line dancing class - I love it and am rarely too exhausted to go. I also joined a local U3A. Perhaps you are going through a similar thing to a 56 year old lady (Minnie58) on another thread - trying to adjust to being your age. It is so different these days to what our mothers faced, generally their lives were mapped out with few choices, but today we can do anything at 64 (or any age) money and health permitting - learn to fly, dance, cruise, study, work, not work, swim, surf, - or just vegetate. How to choose?

That freewheeling feeling that comes with retirement can take a while to get used to. Suddenly not living life to a time table, doing things when you want to and when it is convenient rather than when you are accustomed to do it.

I used to feel almost light-headed when I started to do the weeks shopping during the day and not in a rush, after decades of doing out when I got home. 20 years in to retirement, I have actually stopped and had a cup of coffee and a sit recently after doing the shopping and quite enjoyed it, but it needed a badly bruised back which meant I couldn't stand for long to discover its pleasures.

Try and find some thing you enjoy that you can do at home and pick up or put down as you feel like it. It sounds to me as if you need to retire or at least have a break. This constant need for activity and exhaustion suggest a check with your doctor might be an idea

^What is this life if full of care
we have no time to stand and stare

No time to stand beneath the boughs,
And stare as long as sheep and cows:

No time to see, when woods we pass,
Where squirrels hide their nuts in grass:

No time to see, in broad daylight,
Streams full of stars, like skies at night:

A poor life this if, full of care,
We have no time to stand and stare.^

It sounds to me as if you are trying too hard and really need to get off your hamster wheel.
If you are working part time, do you absolutely have to fill every other minute with activity for which you do not sound all that enthusiastic?
Days don't have to be "filled" just lived and you might find it useful to look at something like Mindfulness, or Yoga to help you to relax. You sound like a tightly wound up rubber band , no wonder you are exhausted!