If you really do not want to drive, sit down and work out what it actually costs to run the car your DH has annually.
Take everything into account, petrol /diesel, oil changes, servicing, insurance than work out how many times you could take a taxi for that amount.
Tell your DH frankly that you never have liked driving nor felt good at it, and you don't want to start again, and find out what the other options are.
Would you honestly not be better off moving so you don't need to drive if the day comes when DH shouldn't?
Yes, you can grit your teeth and drive now, but what about the day when you shouldn't be behind the wheel either?
Enjoying this stage of our lives can be a lot easier if you are within walking /cycling distant of the things and places you need daily and not too far from the hospital, dentist, optician, chiropodist, etc. etc.