You are not selfish to want a dog, but perhaps you will need to accustom the dog to sometimes spending time in a room alone, so that your son and dog won't be in the same room when he visits,
Can your son's allergy be controlled with anti-histamines while he is at yours?
Otherwise I am afraid the dog will need to be in kennels or looked after by a friend when you son visits, and after th dog is out of the house, you will need to clean very thoroughly to get rid of dog hairs.
From the start I suggest you accustom the dog not to lie on chairs or sofas but on its own rug or bed. It may be necessary to use loose covers on all upholstered furniture. It is nearly impossible to get rid of animal hairs, although a plastic sponge pot scourer wrung out in water does a very good job.
I don't think you are being selfish here, as your son only comes a couple of times a year. To me, he sounds selfish expecting you to not have a dog for forty-odd weeks of the year, so he can want to visit for maximum two weeks.