Descriptive Statistics by Groups (Breakdown) Introductory Overview - Post-hoc Comparisons of Means

The breakdowns facility in the Basic Statistics and Tables module also offers all common post-hoc tests for means comparisons. Usually, after obtaining a statistically significant F-test from the ANOVA, you want to know which of the means contributed to the effect (i.e., which groups are particularly different from each other). You could perform a series of simple t-tests to compare all possible pairs of means. However, such a procedure would capitalize on chance. This means the reported probability levels would actually overestimate the statistical significance of mean differences. Without going into too much detail, suppose you took 20 samples of 10 random numbers each, and computed 20 means. Then, take the group (sample) with the highest mean and compare it with that of the lowest mean. The t-test for independent samples tests whether or not those two means are significantly different from each other, provided they were the only two samples taken. Post-hoc comparison techniques, on the other hand, specifically take into account the fact that more than two samples were taken.