It doesn't say it collects the info from the site you were on before this one, it says "from the site you came from". It is anonymous, too, so they only know that someone has been looking at pictures of Merlin and hbis magic wand, not that it was you.
When I was webmaster (webmistress? sounds like a female spider about to eat her mate) of a site, I received anonymous statistics about visitors. It said when there was a visitor, time and date, ISP provider, browser, whether they wer new or repeats, and which pages they looked at for how long AND "referring site" that is, any link on another site that they had followed to get to mine. If they had typed in the last site or had it in their favourites/bookmarks that column was blank.