January 2002

Lies and Statistics

The 19th century British Statesman, Benjamin Disraeli said, “There are three kinds of lies: lies, damned lies and statistics.” Such is the mistrust many people have of statistics.

One problem is that many speakers make the mistake of assuming that all figures are statistics. This simply is not the case; some are simply numbers. Statistics show relationships, largeness or smallness, increases or decreases, or summarize large collections of facts and data. When you choose statistics, there are some questions to ask.

1. Are the statistics recent? Figures concerning the cost of living in 1982 have limited usefulness for today’s family planning its budget. When using statistics be on guard if no date is given or if the statistics are outdated.
2. Do the statistics indicate what they purport to? A single test score may not be the true measure of a student’s ability. The amount of money a company grosses may not indicate how much the company actually earned.
3. Do the statistics cover a long enough time or enough samples to be reliable?
4. If the statistics are drawn from a sample, does the sample adequately represent the group to which you are generalizing?
5. When statistics report differences, are the differences significant? Minor differences can often be attributed to chance. If you were to collect the statistics again would the results be the same?
6. When comparing things, are the units of measure compared the same? If more students fail one course than another, you cannot necessarily conclude that one course is more difficult than another.  Perhaps the grading scale was different.
7. Do the statistics come from a reliable source?  Statistics are no better than their source. The source should be one that is both expert and trustworthy.  Check the source of all statistics to make certain that both criteria are satisfied.

One final thing: Check your statistics carefully for accuracy to make certain that your statistics are not lies or damned lies.