Statistics by Groups Results - Post-Hoc tab

Breakdown: Descriptive Statistics by Groups

Select the Post-hoc tab in the Statistics by Groups Results dialog box to access options to compute post-hoc comparisons for selected variables in the analysis.

Refer to Descriptive Statistics by Groups (Breakdown) Introductory Overview - Post-hoc Comparisons of Means for a discussion of the basic logic behind these tests. Discussions of post-hoc procedures are also provided in Winer (1962), Hays (1988), or Milliken and Johnson (1984).

In short, usually, after obtaining a statistically significant F-test from the ANOVA, you want to know which means contributed to the effect; that is, which groups are particularly different from each other. You could of course perform a series of simple t-tests to compare all possible pairs of means. However, such a procedure would capitalize on chance. This means that the reported probability levels would actually overestimate the statistical significance of mean differences. Without going into much detail, suppose you took 20 samples of 10 random numbers each, and computed 20 means. Then, take the group (sample) with the highest mean and compare it with that of the lowest mean. The t-test for independent samples tests whether those two means are significantly different from each other, provided that they were the only two samples taken. Post-hoc comparison techniques, however, specifically take into account the fact that more than two samples were taken. A large selection of post-hoc comparison procedures is also available in the General Linear Model (GLM) module, including Dunnett's test, as well as facilities for using customized error terms.

Variables. Click this button to display the standard single variable selection dialog box listing the variables used in the analysis. Select the variables for which you want to perform post-hoc comparisons.

LSD test or planned comparisons. Click this button to create a matrix of p-value in a spreadsheet. These p-values indicate the post-hoc significance levels for the respective pairs of means. The LSD test is equivalent to the t-test for independent samples, based on the N in the groups involved in the comparison. It offers the least amount of protection against the increased alpha error rate due to multiple post-hoc comparisons.

Scheffé test. Click this button to produce a spreadsheet with the post-hoc p-values for the Scheffé test. The Scheffé test is usually more conservative than the Newman-Keuls or Duncan test (see Winer, 1962).

Newman-Keuls test & critical ranges. Click this button to produce a spreadsheet with the post-hoc p-values for the Newman-Keuls test. Note that Statistica does not merely report cut-off values for p, but computes the actual probabilities based on the distribution of the studentized range statistics. A second spreadsheet displays the critical ranges between ordered means, given the respective alpha level (by default p < .05). The Newman-Keuls test is based on the studentized range statistic. Computationally, Statistica first sorts the means into ascending order. For each pair of means Statistica then assesses the probability under the null hypothesis (no differences between means in the population) of obtaining differences between means of this (or greater) magnitude, given the respective number of samples. Thus, it actually tests the significance of ranges, given the respective number of samples.

Duncan's multiple range test & critical ranges. Click this button to produce a spreadsheet with the post-hoc p-values for the Duncan test. A second spreadsheet displays the critical ranges between ordered means, given the respective alpha level (by default p < .05). This test is based on the same logic as the Newman-Keuls procedure, however, it uses a less conservative test criterion (see, for example, Milliken & Johnson, 1984).

Alpha level for critical ranges. Enter the desired Alpha level for critical ranges into the edit field or use the microscrolls. The default alpha level is .05. This option applies only to the Newman-Keuls test & critical ranges button and the Duncan's multiple range test & critical ranges button.

Tukey honest significant difference (HSD). Click this button to produce a spreadsheet with the post-hoc p-values for the Tukey HSD test. This test falls between the Newman-Keuls and Scheffé procedures with regard to conservatism.

Tukey HSD for unequal N (Spjotvoll & Stoline). Click this button to produce a spreadsheet with the post-hoc p-values for the Tukey HSD test. This test is a generalization of Tukey's test to the case of unequal sample sizes (see Spjotvoll & Stoline, 1973, p. 975).

p-value for highlighting. Enter the desired critical p-value for highlighting into the edit field or use the microscrolls. In all results spreadsheets, significant differences are highlighted. The default p-value for highlighting is .05.  For more details on p-value, see Elementary Concepts.