A new methodology for aggregating tables: Summarizing journal quality data

Often successive studies are conducted to rank items across time or across different raters. For instance, different consumers may be asked to rank pr oducts, or banks may be ranked at different points of time based on their l ending practices. In such cases, a simple model would suggest arriving at a summary set of rankings by averaging across the sets. However; when the ra nk of an item is missing in some of the sets, a simple mean cart produce a biased result. This is because the absence of an an item in a particular se t may be important information that is being ignored. MacMillan conducted a series of studies to identify a forum for publishing business policy research, the most recent of which appeared in this journal . The design of the study involved two stages of selection: first, the jour nals were to be voted into a set, and second, they were to be ranked by a p anel of scholars. This procedure resulted in some of the journals not appea ring in every study. Thus, taking a simple average across the studies would result in misleading information, because the absence of journal in a part icular study is information that is important (because the panel chose not to vote it into consideration) but lost in averaging. One of the objectives of the MacMillan studies was to provide scholars with information about the importance of the journal as a publication outlet. T hus, it is important to be able to arrive at a summary set of rankings to p rovide cumulative information. To achieve this, a left-censored model was d eveloped. Assuming that the ranks follow a normal distribution, with unknow n mean and the same variance, maximum likelihood estimates of the means wer e calculated.. The summary set of ranks were then calculated using these es timates. The summary ranks were compared with those arrived at fi om an ind ependent estimate of journal quality and found to have validity. Our model helps enhance the value of the MacMillan studies by providing cum ulative information on the journals and facilitating their comparison with other studies. We see the applicability of the model in other areas too. Kn own by the generic term of league tables, such sets of data are generated i n marketing research, educational studies, etc. Thus, the model is an impor tant contribution to the tool kit of empirical research. (C) 1998 Elsevier Science Inc.