Example 12: Test Theory Models for Sets of Congeneric Tests
A variety of interesting test theory models can be tested and estimated
using structural equations modeling. These models are all special cases
of the common factor model, and are discussed in Jöreskog (1974) on pages
49-56.
The classical test theory model can be expressed as a common factor
model. Suppose a group of tests are congeneric, i.e., have the same true
scores underlying them. Then if the observed score is equal to the true
score plus error, and error is uncorrelated with true scores, we have
the path model shown in the diagram of Model
A below for two sets of two congeneric tests.
Suppose you want to assess whether two different tests measure the same
trait. In classical test theory, we would say that the tests measure the
same trait if their true scores are perfectly correlated, i.e., if the
correlation coefficient between the tests, corrected for attenuation,
is unity. Jöreskog (1978) refers to such tests as tau-equivalent.
Model A:

Lord (1957) proposed a test for tau-equivalence. His test required the
assumption that the elements of each pair of tests have equal variance
and equal reliability, i.e., that the tests are parallel within the pairs.
The model corresponding to the hypothesis tested by Lord's method, that
the tests are both parallel and tau-equivalent, is illustrated in the
path diagram for Model B below.
It is possible to test the assumptions underlying Lord's (1957) test directly,
i.e., we can test the hypothesis that the elements of each pair of tests
are parallel (i.e., have equal variance and equal reliability) without
testing the hypothesis that the disattenuated correlation coefficient
is unity.
Model B:

This hypothesis is shown in the path diagram for Model
C below. Models B and
C form a nested sequence, since
B is a special case of C. If we compute the Chi-square statistics
for Models B and C,
and take their difference, we obtain a Chi-square difference statistic
with one degree of freedom for testing the hypothesis that the disattenuated
correlation is 1, given that the tests within pairs are parallel.
Model C:

Lord's test is valid provided its assumptions are met. However, Model D shown below allows us to test
whether the disattenuated correlation coefficient is unity without requiring
the restrictions of Lord's test.
Model D hypothesizes that the
tests are congeneric (but not necessarily parallel) and tau-equivalent.
Model D is a special case of
model A, i.e., the two models
form a nested sequence. Consequently, subtracting the Chi-square statistics
for model A from that obtained
for model D will produce a Chi-square
statistic with one degree of freedom. This provides an alternative test
that the disattenuated correlation is 1.
Model D:

Jöreskog tested Models A, B, C,
and D on some data given in Lord's
(1957) paper. The results of the testing are summarized both in Jöreskog
(1974) and in Jöreskog (1978).
The covariance matrix for this example is in the file Lord.sta.

The PATH1 statements for the
4 models are contained in the files Lorda.cmd,
Lordb.cmd, Lordc.cmd,
and Lordd.cmd.
Both Chi-square difference tests yield the same conclusion: the hypothesis
that the disattenuated correlation coefficient is 1 can be resoundingly
rejected. For example, the Chi-square difference statistic (with one degree
of freedom) for Model A versus
Model D is 35.51. On the other
hand, the maximum likelihood estimate for the disattenuated correlation
is quite high (.899). Moreover, the standard error for this value is only
.019.