Tanith you are not being selfish, in guarding your territory and needing time by yourself. We are all different, some need company a lot others, and I am one, need time on their own.
In working years DH used to travel a lot, rarely for long periods, usually just a few days at a time, but it met my need for a bit of space; a couple of times a month I would come home from work to an empty house and isolation. DH is still working intermittently, so still is a way occasionally and I am used to having him around most of the time but I still treasure those odd hours on my own. Fortunately DH has begun to understand this, which makes it much easier.