When I was at work I used to describe all the smart suits, dresses and accessories I wore, specifically for work as my 'School uniform'. Coming home in the evening I took my school uniform off and got into clothes chosen for comfort rather than style, worn jeans, old sweaters, sometimes, my dressing gown. Casual, but not smart
At weekends and all week now I wore/wear the kind of clothes that were too casual for work, jeans, trousers, sweaters, casual skirts, however, all would have been carefully chosen and co-ordinated to be comfortable but smart but chosen with the same care I used to use for my 'school uniform'. That to me is smart casual.