Yes, notice when she is behaving as you want her to and give lots of praise and cuddles, with the star chart on show. My daughter has a line for each set of 10 stars with Treat written next to the space for the 10th star. Then, if she does something you can reward (and you can create such opportunities) let her get the star and stick it on whilst you praise her. It didn't take long for my grandsons to cotton on about the rewards, and they would come to mum saying 'I did my teeth without you asking, so can I have a star?' Works like a dream!
Also, expect success and make it clear to her that you know she's going to do well as you engineer success opportunities. It'll soon pass. Apparently, I tipped my little sister out of her (my!) pram when I was 4, but all I can remember is how we loved playing together.