This site is under construction. Please check back every few weeks for updates


COMMON MISTEAKS MISTAKES IN USING STATISTICS: Spotting and Avoiding Them

Introduction        Types of Mistakes        Suggestions        Resources        Table of Contents         About

Why Is Random Sampling Important?

The myth: "A random sample will be representative of the population".

In fact, this statement is false -- a random sample might, by chance, turn out to be anything but representative. For example, it is possible (though unlikely) that if you toss a fair die ten times, all the tosses will come up six. If you find a book or web page that gives this reason, apply some healthy skepticism to other things it claims.

A slightly better explanation that is partly true but partly urban legend : "Random sampling eliminates bias by giving all individuals an equal chance to be chosen."1
It is true that sampling randomly will eliminate systematic bias. Moreover, this statement is often the best plausible explanation that is acceptable to someone with little mathematical background. However, this statement could easily be misinterpreted as the myth above. Moreover, there is an additional, very important, reason why random sampling is important, at least in frequentist statistical procedures, which are those most often taught (especially in introductory classes) and used.

The real reason: The mathematical theorems which justify most frequentist statistical procedures apply only to random samples. 
For more details, see the Overview of Frequentist Hypothesis Testing (a similar situation  holds for confidence intervals).
1. Moore and McCabe (2006), Introduction to the Practice of Statistics, Third edition, p. 219. The quote and discussion here should not be taken as criticism of the book -- it is one of the best introductory textbooks for a wide audience.