Common Mistakes in Using Statistics - Spotting Them and Avoiding Them

2015 Summer Statistics Institute Course, University of Texas at Austin

May 26 - 29, 2015 

Course Notes 

Please Note:

    Day                                             Slides (2 per sheet)                                Appendix

1 (Tues May 26)                                     Slides Day 1                                    Appendix Day 1

2 (Wed May 27)     (Be sure to download all three files for Day 2)             (No appendix for Day 2)

                                                   SlidesDay2Part1(pp. 1-31)             
                                                     SlidesDay2Part2(p. 32)

                                                     SlidesDay2Part3(pp. 33-58)

3 (Th May 28)                                  Slides Day 3                                        Appendix Day 3   

4 (Fri May 29)                                     Slides Day 4                                    Appendix Day 4                                      

Additional Appendices

    Suggestions for Readers of Research            Suggestions for Researchers

    Suggestions for Teachers                            Suggestions for Reviewers, Editors, and IRB Members

External Links

Please note: Some of these links use Java applets, which your computer might block (depending on the verion of Java you have and your security settings. For more information, see
Empirical Probability Example

Wise Sampling Distribution Simulation

Rice Virtual Lab in Statistics Sampling Distribution Simulation

How Not to be Misled by the Jobs Report
    Includes two simulations showing how sampling variability can tempt people to see patterns that aren't there.

Bioconsulting Confidence Interval Simulation

W. H Freeman's Confidence Interval Simulation

Rossman-Chance Confidence Interval Simulation
    Try settings: Means, Normal, t, with defaults for the rest of the settings.
    Click "sample" several times, watching how the CI changes. 
    Set "intevals" to 20 to see 20 CI's at once. Notice the Running Total.

The Rice Virtual Lab in Statistics Confidence Interval Simulation

Rice Virtual Lab in Statistics Robustness Simulation

Claremont University's Wise Project's Statistical Power Applet

Jerry Dallal's Simulation of Multiple Testing
This simulates the results of 100 independent hypothesis tests, each at 0.05 significance level. Click the "test/clear" button  to see the results of one set of 100 tests (that is, for one sample of data). Click the button two more times (first to clear and then to do another simulation) to see the results of another set of 100 tests (i.e., for another sample of data). Notice as you continue to do this that i) which tests give type I errors (i.e., are statistically significant at the 0.05 level) varies from sample to sample, and ii) which samples give type I errors for a given test varies from test to test. (To see the latter point, it may help to focus just on the first column.)

Jelly Beans (A Folly of Multiple Testing and Data Snooping)

More Jerry Dallal Simulations: More Jelly Beans    Cellphones and Cancer    Coffee and ...

Spurious Correlations

Distrust Your Data: Jacob Harris on Six Ways to Make Mistakes with Data
    A case study illustrating six common mistakes (including "sloppy proxies" in analyzing data.)

NIH funds training in behavioral intervention to slow progression of cancer by improving the immune system  Both the blog post by  James Coyne and many of the comments provide examples of several questionable practices.

Negative Consequences of Dichotomizing Continuous Predictor Variables  (applet demo)

p-value video (For your amusement; made by UT grad students)

Website on Common Misteaks Mistakes in Using Statistics

    Content similar to the content of the course notes, but includes embedded links and more information. (However, needs some updates!)

Blog: Musings on Using and Misusing Statistics

A companion to the preceding website Common Mistakes in Using Statistics. It contains updates to that site and occasional comments on other things related to statistics that come to my attention. It may  be of interest to the following categories of people:

    Teachers of statistics (especially those, such as myself, who come from backgrounds other than statistics)
    Undergraduates and early graduate students in statistics
    Users of statistics (especially people who read research using statistics)

See especially the series of eight "Beyond the Buzz" posts (June 24 - August 26, 2024) discussing two of the articles in the May, 2014 special issue of the journal Social Psychology devoted to registered reports. These posts show how registered replications can exemplify poor practices and thus do not alone solve the problem of  possibly misleading findings.

Last updated May 19, 2015