Three weeks does seem a little too long, especially as you have a back-slab which is only meant for initial support while there is swelling around the fracture. My understanding is that once the swelling has abated, the correct support needs to be put in place and this could happen from about the 5 day mark onwards - certainly within the first 2 weeks. If you have a walk-in clinic / A&E near you, it might be worth popping in and asking for a check so that a more appropriate support can be given if needed. In my view you have nothing to lose - I completely understand your concern that 'minor' as it is, you want to ensure that it heals properly / without the risk of longer term impediment.
Last year I broke my ankle and was not properly attended to in A&E - no cardiovascular checks done, numbness not investigated, sent on my way with a very sub-standard cast, no pain relief and no instructions for aftercare. I did lodge a formal complaint, but not before I had had to return to a different A&E on day 5 and they were appalled at the state of the cast and the lack of initial care. They said the original cast was in danger of doing further harm!
So - if you are concerned, I would get a second check.