As long as you spend some time together, and have a good relationship, there’s nothing wrong with ‘doing your own thing’, though it’s good to find some programmes that you can watch together.
Neither of us used to watch a lot of tv, until DH had a stroke. He then became an addict of old tv series. He especially liked many of the old comedies. Only Fools and Horses was a favourite, but he was also addicted to Mash and Sharpe..
I enjoyed peace and quiet until he got up, usually around 10am. Then the tv was switched on, and remained so until his bedtime.
It was his only pleasure, so I simply tuned it out. Now that he’s gone I’m back to not watching a lot. I record some programmes that I like - quizzes, antiques, history and gardening mainly, and watch them when I’m in the mood.
It took me some time to get used to the silence, after I was left on my own, but I enjoy it now.