Estimation of
Variance Components  Testing the Significance of Variance Components
When maximum likelihood estimation
techniques are used, standard linear model significance testing techniques
may not be applicable. ANOVA techniques such as decomposing sums of squares
and testing the significance of effects by taking ratios of mean squares
are appropriate for linear methods of estimation, but generally are not
appropriate for quadratic methods of estimation. When ANOVA methods are
used for estimation, standard significance testing techniques can be employed,
with the exception that any confounding among random
effects must be taken into account.
To test the significance of effects in mixed or random models, error
terms must be constructed that contain all the same sources of random
variation except for the variation of the respective effect of interest.
This is done using Satterthwaite's method of denominator synthesis (Satterthwaite,
1946), which finds the linear combinations of sources of random variation
that serve as appropriate error terms for testing the significance of
the respective effect of interest. The spreadsheet below shows the coefficients
used to construct these linear combinations for testing the Variety
and Plot effects.
Denominator Synthesis: Coefficients (MS Type:
1) (wheat.sta) 

The
synthesized MS Errors are linear
combinations of the resp. MS effects 
Effect 
(F/R) 
VARIETY 
PLOT 
Error 
{1}VARIETY 
Random 

1.000000 

{2}PLOT 
Random 


1.000000 
The coefficients show that the Mean
square for Variety should
be tested against the Mean square
for Plot, and that the Mean square for
Plot should be tested against the Mean
square for Error. Referring
back to the Expected mean squares spreadsheet, it is clear that the denominator
synthesis has identified appropriate error terms for testing the Variety and Plot
effects. Although this is a simple example, in more complex analyses with
various degrees of confounding among the random
effects, the denominator synthesis can identify appropriate error
terms for testing the random effects
that would not be readily apparent.
To perform the tests of significance of the random
effects, ratios of appropriate Mean squares are formed to compute
F statistics and pvalues
for each effect. Note that in complex analyses the degrees of freedom
for random effects can be fractional
rather than integer values, indicating that fractions of sources of variation
were used in synthesizing appropriate error terms for testing the random effects. The spreadsheet displaying
the results of the ANOVA for the Variety
and Plot random effects is shown
below. Note that for this simple design the results are identical to the
results presented earlier in the spreadsheet for the ANOVA treating Plot as a random
effect nested within Variety.
ANOVA Results for Synthesized Errors: DAMAGE
(wheat.sta) 

df error computed using Satterthwaite method 
Effect 
Effect
(F/R) 
df
Effect 
MS
Effect 
df
Error 
MS
Error 
F 
p 
{1}VARIETY 
Fixed 
3 
.270053 
9 
.056435 
4.785196 
.029275 
{2}PLOT 
Random 
9 
.056435 
 
 
 
 
As shown in the spreadsheet, the Variety
effect is found to be significant at p
< .05, but as would be expected, the Plot
effect cannot be tested for significance because plots served as the basic
unit of analysis. If data on samples of plants taken within plots were
available, a test of the significance of the Plot
effect could be constructed.
Appropriate tests of significance for MIVQUE(0) variance component estimates
generally cannot be constructed, except in special cases (see Searle,
Casella, & McCulloch, 1992). Asymptotic (large sample) tests of significance
of REML and ML variance component
estimates, however, can be constructed for the parameter estimates from
the final iteration of the solution.
The spreadsheet below shows the asymptotic (large sample) tests of significance
for the REML estimates for the Wheat.sta
data.
Restricted Maximum Likelihood Estimates (wheat.sta) 

Variable:
DAMAGE
2*Log(Likelihood)=4.50162399 
Effect 
Variance
Comp. 
Asympt.
Std.Err. 
Asympt.
z 
Asympt.
p 
{1}VARIETY 
.073155 
.078019 
.937656 
.348421 
Error 
.057003 
.027132 
2.100914 
.035648 
The spreadsheet below shows the asymptotic (large sample) tests of significance
for the ML estimates for the Wheat.sta
data.
Maximum Likelihood Estimates (wheat.sta) 

Variable:
DAMAGE
2*Log(Likelihood)=4.96761616 
Effect 
Variance
Comp. 
Asympt.
Std.Err. 
Asympt.
z 
Asympt.
p 
{1}VARIETY 
.048552 
.050747 
.956748 
.338694 
Error 
.057492 
.027598 
2.083213 
.037232 
It should be emphasized that the asymptotic tests of significance for
REML and ML variance
component estimates are based on large sample sizes, which certainly
is not the case for the Wheat.sta
data. For this data set, the tests of significance from both analyses
agree in suggesting that the Variety
variance component does not differ significantly from zero.