One of my sons has always, like me, been an avid reader, while the other has never been the least bit interested in reading fiction, although he did love to be read to as a toddler. His reading outside of school has been limited to non-fiction since he was about ten.
Of the 4 dgc, the eldest (12) has loved books from a very early age and he is an avid reader, but the other three show very little interest in reading for pleasure, especially the 18 month old, whose interest in a book still rarely lasts more than a minute, as he would rather be constructing things.
It would never crossed my mind to be judgemental of people who don't read - not unless they were in some way belittling those of us who do like to read, anyway.