PROCEDURES FOR DESIGNING COURSE-EVALUATION INSTRUMENTS - MASKED PERSONALITY FORMAT VERSUS TRANSPARENT ACHIEVEMENT FORMAT

One important consideration in designing instruments to measure studen ts' attitudes about a course or an instructor is the influence of the format of the instrument on students' responses. This study contrasted the effects of randomly distributing the items throughout the questio nnaire (personality format) versus grouping the items together from th e same dimension (achievement format) on students' end-of-term course evaluations. On the last day of class, 376 undergraduate students were randomly assigned to complete anonymously the Academic Motivation Pro file (AMP) constructed using either a personality or achievement forma t. A 2 (Format) x 10 (Class Section) MANOVA of the 12 scales of the co urse evaluation instrument demonstrated a statistically significant Fo rmat effect (Wilks's lambda = .85, p < .001), a statistically signific ant Section effect (Wilks's lambda = .37, p < .001), and no significan t Format x Section interaction (Wilks's lambda = .70, p > .05). Examin ation of the factor structures using confirmatory factor analysis indi cated that the measurement model underlying the AMP fit the data reaso nably well for both the achievement and personality formats; however, the achievement format provided a better fit. Based on the statistical results and the fact that the achievement format may be more straight forward, produce less suspicion from the student, and be more time eff icient, it should be considered when designing course evaluation instr uments.