These are the ONS figures from 2017. Apart from Poland, Ireland, Romania and Italy, EU countries don't even make it into the table of top 10 nationalities of immigrants.
The UK can't stop immigrants from Ireland. The UK could and still could stop immigration from India, Pakistan and Bangladesh and any other non-EU country. Leaving the EU won't make any difference, nor will it make any difference to so-called health tourists and illegal immigration from non-EU countries.
It's likely that more people will be classified as illegal immigrants. Some of these will be people who have lived here legally for decades, but can't find/don't have the paperwork to be granted settled status.