If my mother had said that my son was all she had for the rest of her life, my immediate response would have been "Gee, thanks, Mum!" Did she only have you in the hope of you providing her with a grandchild? I doubt it very much. I think you will have to tread carefully, and gradually reduce the time she has with him. And please, don't let her talk you out of sending him to nursery! It does them a lot of good in the long term, and I'd be willing to bet that the family members who apparently agree with her haven't even been asked for their opinion. And if they have....why? None of their business, this is between your mother and you! I wish you all the very best 
My fat balls bring all the birds to the yard
I'm not a pheasant plucker....
