When our GC visit, either with or without their parents, being given treats or activities that they may not have at home is what makes the visit special for them. If our AC are present I sometimes ask if it's OK, they rarely refuse.
Quite frankly I don't think my AC would ever tell me off for doing it, they know I'd give back as good as I got! In our house it's very much a case of 'my house my rules', and if I want to give them sweets or cakes or let them climb trees or splash in puddles then I will.
After spending all day with a small child, twenty minutes of TV so that you can sit quietly and relax is not a crime, it's a necessity!
Remind his parents that child care is exhausting at your age, and a few minutes quiet time in front of the TV was pleasant for both of you, and any tantrum or bedtime problems are probably a response to their rigid rules at home. Point out that they're always free to find alternative child care and pay for the privilege, but while he's in your care you reserve the right to make the rules to suit your situation. I realise that might mean they place him with a childminder, but she too will have her own rules for children in her care.
Don't be bullied by your children!