Wasps !

Wasps and hornets try and build their huge nests in the attic space every year, which means we sometimes get them inside the house as well.Have got used to them, although it’s an expense to call the exterminator ( terminator!?)in.
Some of the family have an awful reaction to stings, so in the garden we always put out a glass of lemonade, well away from us, and they come and go to that, and leave us alone.

If you google ‘wasp repellent’ you’ll find a few ideas. There are plants you can grow in your garden that will help too. I’m going to try the waspinator - we had success with an inflated brown paper bag but it’s not very waterproof!

I'm another one who got attacked by a swarm when I was a child. I wasn't disturbing the nest but I was in the line of fire so to speak. Its left me with a real phobia and, as in your case, its spoils summer days for me.
I have tried a wapinator with some success. I think I might try hypnosis and see if that helps.

I honestly thought wasps didn't swarm!
A pharmacist will advise on repellent (a qualified pharmacist, not a shop assistant in a white coat), I've seen things advertised that are supposed to keep them away from gardens, but have no idea how effective they are.
I hope you find a way to get over your fear.

I share your fear of Wasps.I had a terrible experience with a swarm as a child.I wasn't doing anything to provide them and still I got swarmed on and stung endless times with wasps trapped in my clothing,it was horrific for me.As Ive got older I have got better.I try not to overthink about them.I do use a good Bug spray,some claim to help repel them.Now I am a Grandparent I will protect my Grandchildren from them and not let them know my fear.My adult children have a fear of them mostly caused by me.I do hate them and on average get stung every few years,some stings hurt and others don't, so I really try my best not to worry too much.I don't know what your experience was,but until someone has suffered an "attack" from Wasps they will never know how terrifying it is.I would suggest you look up a repellent that may help.I have a battery operated zapper thay looks like a tennis racket that I keep nearby if I'm outside.Anything that gives you some confidence will help but please don't let them spoil the lovely sunny days and being outside....because then they've won

Get a waspinator which is an artificial wasps nest that you hang up. Apparently wasps are territorial and won’t come near if they think there is a swarm already in residence.

Hi GraneeChrissy - what a shame you don't feel able to enjoy your garden at the moment.
I'm sure you've already thought of it, but what about putting something 'tastier' than you nearby to lure them away ? A smear of jam on a plate or something?