Measurement error in research on human resources and firm performance: Howmuch error is there and how does it influence effect size estimates?

Studies of the relationship between human resource (HR) practices and firm performance typically use a single respondent to assess firm level HR pract ices or HR effectiveness. However, previous research in other substantive a reas suggests that rater differences are a potentially important source of measurement error. We demonstrate analytically the potential consequences o f both random and systematic measurement error in research on HR and firm p erformance. However, our main focus is on random error and we show how gene ralizability theory can be applied to obtain better estimates of reliabilit y by simultaneously recognizing multiple sources (e.g., items, raters) of r andom measurement error. These more inclusive reliability estimates, in tur n, offer the possibility of more precisely quantifying substantive relation ships in the KR and firm performance literature. In our sample, reliabiliti es (as estimated by generalizability coefficients) for single-rater assessm ents of HR variables were generally below .50. This degree of measurement e rror, if present in substantive studies on HR and firm performance, could l ead to considerable bias, given that an unstandardized regression coefficie nt is corrected for measurement error in the independent variable by dividi ng by its reliability coefficient (not its square root). We also found only limited convergent validity between HR and line managers ratings of a seco nd type of HR measure, HR effectiveness. In general, our findings suggest t hat future researchers need to devote greater attention to measurement erro r and construct validity issues. Our study provides an example of how gener alizability theory can be useful in this pursuit.