If they don't represent all their constituents, whether they voted for them or even voted at all, then they are not doing their job. It may be that some of their constituents have different priorities to others, and the MP has to work hard to reconcile all their needs, but that does not mean that they are to be ignored as inhabitants of that area or their letters to their MP on local issues just thrown unread into the bin.
Even with a selfish motive it makes sense - an MP who works for all his constituents, of all parties or none, and has a good reputation for doing do, will stand more chance of gaining voters when he stands at the next election than someone who only looks after his own.