Analyzing Designs with Random Effects Using GLM vs. Variance Estimation and Precision

Different approaches are available and commonly applied to the analysis of ANOVA designs with random effects, such as split-plot designs where the plot factor (ID) is usually considered a random effect. The purpose of this topic is to highlight the differences between and limitation of different approaches.

GLM and ANOVA-based denominator synthesis. This method (Satterthwaite, 1946) is historically the one most commonly applied to the analysis of random effects; it is implemented in the Variance Components and General Linear Models (GLM) modules (and this approach is described in greater detail in the context of those modules). In short, the approach is to compute, based on the ANOVA sums of squares for each effect and the characteristics of the design, approximate appropriate error terms and degrees of freedom for testing the fixed effects in the design (see, for example, the topic GLM Hypothesis Testing - Tests in Mixed Model Designs). The resultant ANOVA results table will then use the "appropriate" error terms to compute F tests (ratios), so that the respective expected mean squares in the numerator of the F ratio contains some variance due to random effects plus variance due to the respective fixed effect, while the denominator contains only the respective variance due to random effects; hence, the F ratio test is a test of the respective fixed effect.

However, in this approach, the analysis usually "ends" at this point, i.e., with the final adjusted ANOVA table and significance tests. The variance/covariance matrix of the ANOVA parameters are not adjusted to reflect the random effects design, and hence, any confidence intervals for least squares (predicted) means, parameter estimates, planned comparisons, and so on would not be appropriately adjusted (and they should typically not be relied on).

Variance Estimation and Precision and the mixed effects model. STATISTICA Variance Estimation and Precision implements the true mixed model approach. The computational details of this approach are explained in the Variance Estimation and Precision Introductory Overview - ANOVA and REML Method Implementation in Variance Estimation and Precision topic. In short, the program will estimate the fixed effects design and random effects design separately, but incorporate the variance of random effects in the tests of the parameters of the linear model, and related statistics. As a result, using the Variance Estimation and Precision ANOVA method, the ANOVA results table (tests of fixed effects) is usually the same in Variance Estimation and Precision as it is in GLM; however, the least square means, their standard errors, comparisons between least square means (and the standard errors of their differences), as well as the standard errors of the parameter estimates of the linear model for the fixed effects will be different, i.e., Variance Estimation and Precision will compute those statistics consistent with the mixed linear model, while GLM will not.

Therefore, when analyzing, for example, split plot designs (e.g., D-optimal split plot as they can be constructed in Experimental Design), Variance Estimation and Precision is the preferred method if you want to evaluate planned comparisons between least square means, confidence intervals for parameter estimates in the linear model for fixed effects, or if you want to use restricted maximum likelihood methods (REML; see Variance Estimation and Precision Introductory Overview - ANOVA and REML Method Implementation in Variance Estimation and Precision) to estimate variance components. However, the overall ANOVA results table will in practically all cases be the same, and appropriate, using either Variance Estimation and Precision or GLM to analyze such designs.