An empirical comparison of statistical construct validation approaches

The use of measurement instruments to examine causal relationships among co nstructs constituting theoretical frameworks is important to advancing engi neering management research. This paper examines two broad implementation a pproaches to statistical refinement and validation of measurement instrumen ts. The two approaches differ in their refinement procedures in their use o f principal component factor analysis (Approach A) and conventional confirm atory factor analysis (Approach B). It is difficult to evaluate the net imp act of these fundamental differences between the two approaches on the resu lting statistical construct validity merely using theoretical arguments. To assess their power of construct refinement and validation, we undertook a comparison of the outcomes of the two approaches using two measurement inst ruments (the TQM instrument and the Supervisor instrument). In addition, we tested the potential benefits of blending the two approaches into a third "Hybrid Approach." Results indicate that Approach B and the Hybrid Approach provide refined scales with higher unidimensionality, reliability, converg ent validity, and discriminant validity. However, Approach A and the Hybrid Approach can identify and split constructs with underlying patterns indica ting existence of multiple dimensions and yield better operationalization o f the nomological framework. In conclusion, the Hybrid Approach combines th e strengths of Approach A and Approach B. It performs well not only in term s of the statistical validity of constructs, but also incorporates the feat ure to recognize patterns suggested by exploratory methods. We recommend it s use for refining and validating measurement instruments in relatively une xplored research domains as well as in matured research domains. The result s have strong applicability for statistical construct validation of instrum ents in engineering management and other fields using measurement instrumen ts.