I sympathise, some children are harder to bring up than others. Being tired compounds any negative feelings the child (or their parent) may be experiencing. Getting the youngest of 3 children, who are close in age, to bed at an age appropriate time can be a challenge. It's worth putting the time and effort into this. Having one of his parents to himself to go through his bed time routine, especially the story in bed, usually helps.
I agree with those who say clear boundaries, consistently and fairly maintained, along with good routines and "catching him out being good" are essential. Threatening to send children away is a definite no no. Is there a family centre attached to the nursery school he attends? If so, maybe they run some kind of parenting support group. The staff are usually parents themselves, and empathise with the struggles most of us have at some point or other in bringing up our children.
I expect it'll all settle down Funnygran, family life eh?