It is not a question of fairness, surely, it is a question of your relationship with your son. Has he always treated you as a menial to run his life for his convenience? In that case the problem is somewhat deeper than this manifestation of his expectations.
When you speak of Christmas, do you mean the season or the day? If the day I can understand that Christmas for him and his family may mean a turkey, in which case why not get together on Boxing Day or Christmas Eve or even New Years Day instead. That way you each stay in your own homes on Christmas date and eat as you wish, and when he visits on another day the family can reasonably be expected to eat your preferred food.
Personally, I will always trim my food offering to meet peoples restrictions - vegetarian, vegan, allergies - but I would not expect them to demand that they be fed food I would not normally eat - battery reared chicken or pork, rather than free-range and organic alternatives, for example.