Why you can't trust statistics!

i have just discovered that I no longer have my copy of 'Facts from Figures' by M J Moroney. Published 50 years ago it is a marvellous analysis of the many ways statistics are misused. In our household we still us the word 'Moroneygraph' to describe any graph that exagerates the increase/decrease in the variable by leaving half the values out, like population figures where the scale starts at 50 million and makes any increase look disproportionately larger than it actually is.

Statistics are defined as numerical facts so, by definition, there are factual. This should be true if it relates to a scientific experiment in a laboratory or, say, counting the number of items sold in a shop. The difficulty comes when numbers are described as statistics when they are estimates, guesses or lies. When even real statistics can produce incorrect deductions, fake statistics are almost certain to produce rubbish answers. This is particularly true when rubbish answers are what the users are seeking.

Ben Goldacre's book should be required reading in all schools. The way it is written is perfectly within the comprehension of a teenager with average reading skills.

I have always been a cynic and over the years I have come to distrust almost everything said by politicians, whatever their party. As soon as I hear statistics being quoted, I wonder what axe they have to grind.

We have had a thread before about Ben Goldacre's book, 'Bad Science' - it has an excellent account of how 'good' science conducts experiments and what information we need to assess the results of research.

I don't think so yogagran. It depends on what kind of statistics you are talking about. In science your work stands or falls by your statistics and other clever people scrutinise them. Those on shaky ground may cite 'statistics' but if you scrutinise them, they certainly do not "prove" anything.
Proof only comes in science when many well conducted, statistically sound studies all come to the same conclusions.
The kind of "statistics" - used by politicians and journalists are a completely different thing. The current 'statistic' being used by Cameron and Pickles about 120,000 problem families for instance is highly questionable. There is a wonderful Radio 4 programme about statistics - excellent and entertaining - and their item on this particular topic was very enlightening. As usual politicians have picked up a bit of research, liked the headline, twisted it to meet their objectives, journalists done a bit more twisting and distorting and away we go. In the original research for instance "mothers with mental health problems" was one of the issues. Now what does Eric Pickles propose to do about that I wonder.
Economics of course not what I would call a science and uses 'statistics' in its own idiosyncratic way.

Well, statistics can be mis-used to prove whatever you want, particularly if they are derived from an unreliable source - like an unrepresentative survey. How may are those reported every day? Even ones you might accept can give a wrong impression. We always thought our economy was doing terribly in the 1970's but a diligent economist found that there was an unintended downward bias. The first figure for economic activity was invariably compiled very cautiousy and always revised up later, sometimes many times. So, economic growth, as reported, always understated the figure. I suspect politicians have changed that now! I still like vital statistics and I don't really care if they are an inch or so out.

Statistics can be used to prove whatever you want

I agree, Flickety, and of course two million men died in WW1.
The best estimate of life expectancy seems to be what it is after you have reached a certain age, so you are taking out of the figures all the young men who die in road accidents.

I love statistics - I just think that when somebody is using them to back up some claim, we need to know exactly what their terms mean, how the research was funded, how big the sample was, etc. etc.
I nearly gave up using HRT after a hysterectomy because I read that it doubled the risk of breast cancer. Then I dug a a bit more deeply, and although I can't remember the figures it was something like an increase from one woman in ten thousand to two women in ten thousand.

..... and the closer mean, mode and median are the more likely the distribution is to being symmetrical around the mean, meaning that half the values are above the mean and half below.

The problem is many distributions are skewed with more values at one end than the other. The classic example that regularly drives me to fury with its misuse is life expectancy. When talking about the problems caused to pension fund, government etc media commentator always note that average life expectancy in 1900 was only in the upper 40s and now it is over 80 so that over the 20th century we have almost doubled our life span. Yet most of us know from our families, visits to cemeteries etc that most people who died 50 years ago or more seemed to live considerably longer than that.

What the idiots quoting these figures fail take on board is that in 1900 around 15% of children died before they were 5, that figure now is thankfully around 1/2%. For every 30 children who died under the age of 5 in 1900 only 1 dies now

If 15% of children die before they are 5 this brings life expectancy for the whole population right down. In 1900 the majority of the population who lived beyond childhood made it to their 70s and many into their 80s so life expectancy has not increased by anything like the amount commentators and government would like us to believe. The pension crisis is not caused by how long we live but how few children we have. In 1900 on the majority of families had three or more children, now it is less than 2 - and that statistic is reliable.

But we should not forget that without statistics (and there is much more to statistics than the mean, median and mode) there would be no experimental science, because it is only through statistics that the results can be shown to be meaningful or meaningless

Agreed. The Mean can be very deceptive.

If a newspaper article says the average income of a group of people is £x, that tells you immediately that most people have an income smaller than that. As Greatnan says, it only takes a few large outliers to skew the whole thing.