Common Mistakes in Using Statistics - Spotting Them and Avoiding
2015 Summer Statistics Institute Course, University of Texas at
May 26 - 29, 2015
- Files are in pdf format.
- Most students will want to download the slides and either print them to
take notes on in class, or follow along in class on their
- The appendices contain
additional material and references. They are available for your
reference later according to your own needs.
- Copies of course materials will not
be handed out in class.
- Computers for individual use will not be available in the classroom
for this particular course.
- If you need a different print size or would prefer a .doc file to
take notes on, please email me so
I can email you .doc files to adjust
to your needs. (Please note: Past experience is that .doc files often
changes in formatting or symbols when posted on the web; this might
also happen with email attachments on some platforms.)
Editors, and IRB Members
Please note: Some of these
links use Java applets, which your computer might block (depending on
the verion of Java you have and your security settings. For more
information, see http://wise.cgu.edu/)
Virtual Lab in Statistics Sampling Distribution Simulation
Not to be Misled by the Jobs Report
Includes two simulations showing how sampling
variability can tempt people to see patterns that aren't there.
Confidence Interval Simulation
H Freeman's Confidence Interval Simulation
Confidence Interval Simulation
Try settings: Means, Normal, t, with defaults for
the rest of the settings.
Click "sample" several times, watching how the CI
Set "intevals" to 20 to see 20 CI's at once. Notice
the Running Total.
Rice Virtual Lab in Statistics Confidence Interval Simulation
Virtual Lab in Statistics Robustness Simulation
University's Wise Project's Statistical Power Applet
Simulation of Multiple Testing
This simulates the results of 100
independent hypothesis tests, each at 0.05 significance level. Click
the "test/clear" button to see the results of one set of 100
tests (that is, for one sample of data). Click the button two more
times (first to clear and then to do another simulation) to see the
results of another set of 100 tests (i.e., for another sample of data).
Notice as you continue to do this that i) which tests give type I
errors (i.e., are statistically significant at the 0.05 level) varies
from sample to sample, and ii) which samples give type I errors for a
given test varies from test to test. (To see the latter point, it may
help to focus just on the first column.)
Jelly Beans (A Folly of Multiple
Testing and Data Snooping)
More Jerry Dallal Simulations: More Jelly Beans
Cancer Coffee and ...
Your Data: Jacob Harris on Six Ways to Make Mistakes with Data
A case study illustrating six common mistakes
(including "sloppy proxies" in
funds training in behavioral intervention to slow progression of cancer
by improving the immune system Both the blog post by
James Coyne and many of the comments provide examples of several
Consequences of Dichotomizing Continuous Predictor Variables
p-value video (For your amusement; made by UT grad
Content similar to the content of the course notes,
but includes embedded links and more information. (However, needs some
A companion to the preceding
website Common Mistakes in Using Statistics. It contains updates to
that site and occasional comments on other things related to statistics
that come to my attention. It may be of interest to the following
categories of people:
Teachers of statistics (especially those, such as
myself, who come from backgrounds other than statistics)
Undergraduates and early graduate students in
Users of statistics (especially people who read
research using statistics)
See especially the series of eight "Beyond the Buzz" posts (June 24 -
August 26, 2024) discussing two of the articles in the May, 2014
special issue of the journal Social
Psychology devoted to registered reports. These posts show how
registered replications can exemplify poor practices and thus do not
alone solve the problem of possibly misleading findings.
Last updated May 19, 2015