COMMON MISTEAKS MISTAKES IN
USING STATISTICS:
Spotting and Avoiding Them
Introduction
Medical researcher John P. A. Ioannidis has asserted, "It
can be
proven that most claimed research findings are false."[1].
There are some criticisms of his "proof"[2],
but even most of his critics agree that there is a high incidence of
false conclusions in research papers. Indeed, we frequently hear
in the news results of a research
study that appears to contradict the results of a study published just
a
few years ago. Although there is occasional deliberate falsification,
most of the problem comes from lack of understanding of statistical
techniques, their proper use, and their limitations.
The intent of this website is to discuss some of the common
mistakes made in using statistics, and offer suggestions on how
to avoid making them.
Types of mistakes
Many mistakes in using statistics fall into one of the following
categories:
Suggestions for reducing the incidence of mistakes in using
statistics
Notes:
1. Ioannidis
JPA (2005) Why Most Published
Research Findings Are False.
PLoS Med 2(8): e124. doi:10.1371/journal.pmed.0020124, available at http://www.plosmedicine.org/article/info:doi/10.1371/journal.pmed.0020124
See also the following popular press articles:
- David H. Freedman, Lies, Damned Lies, and Medical Science, The Atlantic, November 2010, http://www.theatlantic.com/magazine/archive/2010/11/lies-damned-lies-and-medical-science/8269/1/
- Jonah Lehrer, The Truth Wears Off: Is there something wrong with
the scientific method?, The New Yorker, December 13, 2010, http://www.newyorker.com/reporting/2010/12/13/101213fa_fact_lehrer
- Ioannidis, John P. A., An Epidemic of False Claims, Scientific American, June, 2011,
http://www.scientificamerican.com/article.cfm?id=an-epidemic-of-false-claims
and the semipopular article by Andrew Gelman
and David Weakliem, Of Beauty, Sex, and Power, The American Scientist, 97(4),
July-August 2009, www.stat.columbia.edu/~gelman/research/published/power4r.pdf
2. e.g.,
Steven Goodman and Sander Greenland. "Why Most Published Research
Findings Are False: Problems in the Analysis". PLoS
Medicine 4 (4):
e168. doi:10.1371/journal.pmed.0040168; Pauker SG (2005) The
Clinical
Interpretation of Research. PLoS
Med 2(11): e395.
doi:10.1371/journal.pmed.0020395; Wren JD (2005) Truth,
Probability,
and Frameworks. PLoS
Med 2(11): e361. doi:10.1371/journal.pmed.0020361;
The PLoS Medicine Editors (2005) Minimizing Mistakes and Embracing
Uncertainty. PLoS
Med 2(8): e272. doi:10.1371/journal.pmed.0020272
Last
updated April 10, 2012.