I can see both sides of this issue.
If you have watched your DH die of colon cancer, it is understandable that you are afraid of the same thing happening to you. The prospect of a colonoscopy can be scary too.
Let's start by saying that it is quite likely that you haven't got cancer anyway.
Even if you did turn out to have cancer, it doesn't mean that your illness would follow the same progression as your husband's. You might be at a much earlier stage. Yes, you might need a colostomy but it's not the end of the world. I have had an ileostomy since my early 20s,due to ulcerative colitis, but I know a lot of people who have colostomies. (We can all get together on Facebook!) These are all people who have had cancer and have had successful operations. They are living normal lives with every chance of a normal life expectancy. They are happy that they have survived, for their own sake and for their families. They wouldn't be here now if they had refused tests and treatment.
As for the colonoscopy, it's nothing to worry about. I had several, back in the days before my operation, and there was nothing painful or frightening about them. (They weren't very dignified but that applies to a lot of medical procedures.) In fact, most of my family have had them! DD (irritable bowel), ex-SiL (ditto), current SiL (diverticulitis), DH (polyps). My DH was a bit freaked out by the idea but the medical staff couldn't have been kinder, and afterwards he said that he didn't know why he had worried. His polyps were benign but could have become cancerous if not removed. That's another good reason for getting a test done.
Please think very carefully before you refuse the test. If you don't get it done, then you are going to worry endlessly. If you have it done, hopefully it will reassure you and, at the very least, it will enable you to make an informed decision. And try not to imagine that your DH's case and yours are the same. That's always a mistake.