I lived in a Victorian terraced house for 50 years but decided to down size in my early seventies to a modern house. One reason being the difficulty of gardening without suffering backache. I had always loved gardening and it provided solace when my husband died suddenly.
Although you are younger than me, perhaps keeping a garden will prove to be more difficult as you grow older.
What I am trying to say is, don't make the garden your only reason to buy a house.
Is it me or am I getting mixed messages