This paper presents two methods for estimating the
reliability of a computerized adaptive test (CAT) without
using item response theory. The required data consist
of CAT and paper-pencil (PP) scores from identical
or equivalent samples, and scores for all examinees on
one or more covariates. Multiple R's and communalities
are used to compute the ratio of CAT and PP reliabilities.
When combined with the PP reliability calculated
by a conventional procedure, these ratios yield
estimates of CAT reliability. Index terms: computerized
adaptive testing, item response theory, predictive
validity, reliability, tailored testing.