Lots of 'bad stuff in the past for me too' and it is hard to forget it. Like you, I came from a family where anger wasn't expressed (my mother could give us the silent treatment for days though, which is just as bad!). When things are bottled up so long, I know it can all come out at once which just puts the person at the receiving end on the defensive and gets you nowhere. I beat myself up for years for not having the courage to leave but, a few years ago, around my 60th birthday when I had a party for women friends, I realised I had a lot of friends (only two I would trust with secrets though) and this made me feel better about myself and my choices (or non-choices some thought). My husband had already been working from home for some years (as was I) before retirement so there wasn't the big fear that you have of being in each other's company too much and, as he has always been involved in various groups, I began to make a separate life for myself too, doing things I like (that he doesn't) with friends. This may not be what other couples do in retirement but it suits us, and that's what matters and our relationship is at least on an even keel and we do see family together.
I also started writing a gratitude diary about three years ago, at first, every morning (I use an online app), jotting down three positive things. I don't do it every day now but it really does focus the mind to look at all the good things in one's life rather than dwell on the not so good.
I wish you well - just do whatever feels right for you, and enjoy your retirement.