You are not being rude, telling callers like these that you are not interested in whatever they are selling, or their religious or political views.
You have a sign on your door, which they chose to ignore, so don't worry any more about it. I have found that I often have to be ruder than I would like to be understood at all by cold callers, either at the door or on the phone.
There is one, and only one, very good way of getting rid of Jehovah's Witnesses, or Mormons. Do as I, open the door, smile and tell them you are a practising Catholic, then close the door.
If any of you happen to be observant Jews , that will work as well, at least it did for a friend of mine.