Example 12: Test Theory Models for Sets of Congeneric Tests

A variety of interesting test theory models can be tested and estimated using structural equations modeling. These models are all special cases of the common factor model, and are discussed in Jöreskog (1974) on pages 49-56.

The classical test theory model can be expressed as a common factor model. Suppose a group of tests are congeneric, i.e., have the same true scores underlying them. Then if the observed score is equal to the true score plus error, and error is uncorrelated with true scores, we have the path model shown in the diagram of Model A below for two sets of two congeneric tests.

Suppose you want to assess whether two different tests measure the same trait. In classical test theory, we would say that the tests measure the same trait if their true scores are perfectly correlated, i.e., if the correlation coefficient between the tests, corrected for attenuation, is unity. Jöreskog (1978) refers to such tests as tau-equivalent.  

Model A:

Lord (1957) proposed a test for tau-equivalence. His test required the assumption that the elements of each pair of tests have equal variance and equal reliability, i.e., that the tests are parallel within the pairs. The model corresponding to the hypothesis tested by Lord's method, that the tests are both parallel and tau-equivalent, is illustrated in the path diagram for Model B below. It is possible to test the assumptions underlying Lord's (1957) test directly, i.e., we can test the hypothesis that the elements of each pair of tests are parallel (i.e., have equal variance and equal reliability) without testing the hypothesis that the disattenuated correlation coefficient is unity.  

Model B:

This hypothesis is shown in the path diagram for Model C below. Models B and C form a nested sequence, since B is a special case of C. If we compute the Chi-square statistics for Models B and C, and take their difference, we obtain a Chi-square difference statistic with one degree of freedom for testing the hypothesis that the disattenuated correlation is 1, given that the tests within pairs are parallel.

Model C:

Lord's test is valid provided its assumptions are met. However, Model D shown below allows us to test whether the disattenuated correlation coefficient is unity without requiring the restrictions of Lord's test.  

Model D hypothesizes that the tests are congeneric (but not necessarily parallel) and tau-equivalent.  

Model D is a special case of model A, i.e., the two models form a nested sequence. Consequently, subtracting the Chi-square statistics for model A from that obtained for model D will produce a Chi-square statistic with one degree of freedom. This provides an alternative test that the disattenuated correlation is 1.

Model D:

Jöreskog tested Models A, B, C, and D on some data given in Lord's (1957) paper. The results of the testing are summarized both in Jöreskog (1974) and in Jöreskog (1978).  

The covariance matrix for this example is in the file Lord.sta.

The PATH1 statements for the 4 models are contained in the files Lorda.cmd, Lordb.cmd, Lordc.cmd, and Lordd.cmd.  

Both Chi-square difference tests yield the same conclusion: the hypothesis that the disattenuated correlation coefficient is 1 can be resoundingly rejected. For example, the Chi-square difference statistic (with one degree of freedom) for Model A versus Model D is 35.51. On the other hand, the maximum likelihood estimate for the disattenuated correlation is quite high (.899). Moreover, the standard error for this value is only .019.