What will happen when all the toys are broken I wonder. Taking toys away would really only be appropriate if he had been bashing the baby with the toy or deliberately making noise perhaps. Those are natural. Consequences that the child will be able to understand, well eventually.
I used to pop baby sister on my back where she would sleep sometimes and we would almost be just the 2 of us again, we'd do ordinary things together and also special things that I'd planned, I had a ladybird book of things to do like I remember making and painting a postbox that was a fun posting toy, but what it is is not relevant its spending time with the older one.
It is also very disrespectful of the child the toys are his, and while he has the responsibility to be thoughtful of others with them, so you can put them away for example.
Breaking toys is not a sustainable method of dealing with the situation and even worse storing up problems for the future