I think the answer entirely depends on your general financial circumstances.
If the amount needed to visit is really going to leave you eating bread and butter to the exclusion of all else for a month, then no, I would not go, but explain to my son why we could not come.
On the other hand, family matters, so I would try to put a tenth or a twelfth of the amount needed aside every month, so we could afford to visit.
We pay every November to have snow cleared off our pavement, if and when there is any snow, as we cannot do it ourselves, or don't want to. I put a tenth of the annual cost aside every month -that way I find it easier to afford what is after all a task that is easier to pay for than do myself, and as it is illegal here not to clear your pavement, I can't just not do it.