Lillies are poisonous ; cats die because they brush against the plant and then groom themselves thus ingesting the poison . Most bulbs are poisonous if eaten , some greedy dogs try that and oleanders and lily of the valley are also poisonous .
There are many plants and weeds in your garden that are poisonous, they don’t usually cause a problem unless a pet or a child decides to eat them. All you can do is check up on any new plants and decide if there a risk.
Our cat nearly died from lily poisoning. I was given a lovely big bunch of lilies a few years ago. Didn't know they were poisonous to cats (even the fumes). Were in the same room as the cat overnight. When we came down in the morning she was frothing at the mouth and gasping for breath. Was admitted to vet hospital for 2 days on a drip and medication and fortunately survived. Don't know about gladioli.