As a cyclist, l get often stuck at the gates with trains crossing, and am still constantly amazed at nearly every carriage being empty. There is the odd person here and there, and time-wise, it's usually late morning to mid afternoon when l cross the tracks.
With that in mind, perhaps it's not so terrifying being on a train, as l first thought.
I am in West Sussex, so if l needed to, l would not be too scared to travel thus. For the biggest danger seems to be surfaces, door handles, hand rails etc.
Basically, it's your call, but at the times mentioned above, it would seem to be eminently do-able.