The absence of intimacy does not mean you are "old, fat and unlovable" - it could be due to a lot of different things, some not to do with you at all.
I think the question you need to ask yourself is: "Would I mind about the absence of intimacy if I did not feel I was old and fat?" In other words is it the lack of intimacy itself that you mind about but the fact that you link it with your poor self-image at the moment. If you felt like a real dish, might you be happy with the absence?
It may be that he has a problem that he has not told you about.
A bit of a heart to heart is needed if you feel able to initiate that. Or a chat with a counsellor to help you unpick your feelings. This absence of self-worth is a problem of itself, regardless of anything else.