How do I stop these feelings

He told you that he ended the affair 'because it was going nowhere', to my mind that perhaps indicated that if he had felt it might have had a future he would have persevered with it. Its not a nice thing to have in the back of your mind.
I had to accept that my first H had destroyed the trust we had between us. He had an affair, then regretted it and asked to return....I accepted this but in my mind he was going to have to work very hard to prove his commitment. By this I do not mean I wanted him to show contrition or be permanently apologising....I just felt he needed to 'woo' me again and make me feel I was 'the one'.
It did not happen, I slowly fell out of love and realised I could never trust him. Reader, I divorced him.
Unfaithfulness is not really the reason for breakdowns, its the loss of trust and the sense of betrayal. Unless he can build that back up then you are always going to be doubting that and he will probably start to accuse you of not trusting him....and who knows where that will end.

The whispered "when are we leaving" rings a great clanging bell with me! It is very annoying isn't it?!

Maybe, to deal with this specific situation, you could ask him why he keeps ringing; and tell him that you do not wish him to and will not be answering the calls. No-one knows why he is making these calls, and your suspicions may or may not be unfounded; but they are spoiling your visit and making you sad, so just tell him to stop.

It is a big betrayal of trust and your feelings will not just slip away - you cannot simply stop them, however hard you might try. But maybe you could tell yourself that you are not going to let this (understandable) absence of trust stop you doing what you want, and develop your own life aside from him.

It does sound as though the trust issue is very much still alive for you, and I sympathise with you in this sad situation. I know that betrayal leads to a loss of self-esteem, but you are a person in your own right with a life to live, and need to grab hold of this.

Good luck with all this.

As mombs said can you not invite him with you to visit your DD if he constantly refuses I would make it my business to come home earlier than expected and see what turns up when you get home.
I would certainly not forgive a second time .

He does come with me sometimes but it’s not very relaxing, although he gets of really well with them he isalways itching to get back home and to his beloved computer! Then I get wispered ‘when are we leaving’. I just want to scream.

Could your DH not visit your DD with you? Would that make you more relaxed? Otherwise tell your DH how you feel and make it very clear if anything happened again you could not forgive a second time. Let him know how insecure you feel. Hope you can both sort this out.