Interest rates are historically low. Yes, the country does have to pay some interest, but not a lot. The increase is being caused, because the government is continuing to borrow, for other reasons. The economy is stagnating, productivity is low and the country isn't raising enough taxes. I think Hammond knows that, but Cameron's manifesto pledge meant that he couldn't raise taxes, which is why he had to do a u-turn on raising NICs. If Labour had known what it was doing, it would have applauded that rise, because it was actually progressive.
The Conservatives are probably going to win a landslide and this government will start with a clean slate. I don't know what they'll do, but I expect to see tax rises. The poorest have already had most of their pips squeezed, so those rises are going to have to come from the upper end of the wealth and income scale, including some who probably don't see themselves as rich. I would be amazed if the pensioner triple lock doesn't go. That's why the back benchers don't like it. The country also has to pay for Brexit somehow.