It sounds as though you are not going to be able to put this behind you as things stand and you therefore have to do something.
I definitely think a counsellor can help. I went to one about a serious work-related issue and it really helped me to decide what I had to do, which was to leave that job.
It doesn't sound as though you want to give up on your marriage, however.
I don't think you should feel guilty for looking after elderly parents - things like that crop up in most marriages and, ideally, both husband and wife should be helping and certainly not feel the need to look elsewhere for attention during such a difficult time.
If your marriage is otherwise happy and you want to stay together, I think you have 3 options depending on what your husband's response is:
1) Find a way to have a full, honest discussion with him about it, explaining you can't put it behind you till you understand it fully. If possible, don't be accusatory. I think this may be very difficult from what you've said
2)Persuade him that you should both go to a counsellor - not because you don't believe he didn't have an affair, but because you're having trouble understanding what went on, what caused it and why you're struggling to put it behind you. Say you know repeatedly bringing it up has been upsetting for him and you want both of you to make the best of your marriage, put this behind you and have many happy years ahead of you. Describe what's been good about your marriage and why you love him etc .
3)If he won't agree to see a counsellor with you, go yourself to see if you can find a strategy for dealing with what's happened and get advice about whether you're better to accept what he and the other woman have said about there being no affair.
I'm not saying there was or wasn't an affair - just that not taking some definitive action will probably lead to the breakdown of your marriage from what you say about it "destroying you". If you do discover that he did have an affair, you would then need to decide whether to forgive this and work on your marriage to stay together.
I do know that it is possible to have a very close friendship with someone of the opposite sex without it being a sexual thing. I was very close to someone I worked with - partly as a result of having to support each other and sort out problems on a daily basis due to incompetent senior management. Another colleague wouldn't believe we weren't involved in a sexual relationship .
Whatever you decide to try - all the very best for the future. 