Just my opinion - alarm bells are ringing for me. I would keep away from him. The very fact that he isn't interested in you as a person (other than sex wise) and talks about himself all the time, says he would not be a supportive and kind and trustworthy partner, or even a friend.
If you were not in a rather lonely place now, you probably wouldn't be interested in him.
Keep on making efforts to expand your social life locally. How about some volunteering, something that interests you. Do you have a local Meet Up group? If not, you could start one, for your age group. There are some our way for over 50s. www.meetup.com/