Improving the validity of letters of recommendation: An investigation of three standardized reference forms

Although letters of recommendation are (LORs) widely used, little research has examined how accurately they predict job performance. The few existing studies have yielded mixed results, and meta-analytic estimates of validity range from .14 to .27 (Hunter & Hunter, 1984; Reilly & Chao, 1982). This i nvestigation was designed to improve predictive validity by developing a st andardized reference form and evaluating 3 different rating formats: Multi- Item scales, Relative Percentile Method (RPM) scales, and Global Trait Rank ings. A total of 520 individuals applied to the Canadian military, and 544 LORs were obtained. Complete predictor and criterion data were available fo r 57 participants. Regression analyses indicated that the validity of the R PM rating format (R-(adj)(2) =. 18; R-(adj) = .42) was substantially higher than previous estimates of LOR validity. The 2 remaining methods produced nonsignificant results. Limitations of the study, suggestions for future re search, and implications for the field are discussed.