These are the reasons you don't have to wear one:
"Face coverings exemptions
The requirement to wear a face covering on TfL's public transport stations, platforms and services does not apply to:
Children under the age of 11
Employees of, or persons providing agreed services to, TfL
Police constables (including British Transport Police officers) acting in the course of their duty
Members or employees of the emergency services responding to an emergency
You also do not need to wear a face covering if you have a good reason not to; such as:
If you have a physical or mental illness or impairment, or a disability that means you cannot put on, wear or remove a face covering
If putting on, wearing or removing a face covering would cause you severe distress
If you are travelling with, or providing assistance to, someone who relies on lip reading to communicate
If you are travelling to avoid injury or escape the risk of harm, and you do not have a face covering with you
If you need to remove it during your journey to avoid harm or injury or the risk of harm or injury to yourself or others
If you need to eat, drink, or take medication you can remove your face covering
If you are asked to remove your face covering by a police officer or other official, for example to check your railcard"
From the TfL website
They are a bit vague, who is going to prove whether your distress at having to wear one is severe?
For those with a genuine mask phobia, sadly there are some, or if you feel you fit one of the criteria above, you can download an exemption card tfl.gov.uk/cdn/static/cms/documents/face-covering-exemption-card.pdf
I will wear one in shops, but stable doors and bolting horses spring to mind.