Expected Agreement Coefficient for Norm-Referenced Tests With Classical Test Theory

Abstract

There are two types of agreement coefficients for psychological test scores: norm-referenced and criterion-referenced agreement coefficients. These coefficients were derived within the framework of generalizability theory that is known for its theoretical and practical complexities. Under the framework of classical test theory, the paper derived the norm-referenced agreement coefficient. This derivation was based on the assumption of randomly equivalent test forms. The resulted expected agreement coefficient was different from its counterpart in generalizability theory. However, the estimators of this norm-referenced agreement coefficient were equal under the two frameworks to coefficient alpha reliability.