Journal quality assessment: An integrated subjective and objective approach

Many universities, research institutions, and government funding agencies a re continuously attempting to grade or rank journals for their academic val ue. Such grading is needed both for funding and resource allocation purpose s and for personnel decisions. In grading journals, objective information, such as the impact factors (the most commonly used journal citation informa tion provided by Journal Citation Reports) and/or subjective information, s uch as experts' judgments about the journals, are used. Grading of journals is typically done by a committee. Most of the existing approaches only con sider either the subjective or the objective information. While such approa ches work in many cases, they may not work in many others. Complaints about politics, inconsistencies, and unfairness are common in many situations. T his paper presents a methodology that integrates both types of information, providing a comprehensive method for assessing the quality of journals. Th e method employs a fuzzy set approach that also deals with the imprecise an d missing information frequently inherited in the evaluation process. The p roposed evaluation model is implemented in a Web-based system. A pilot eval uation study indicated that participants clearly preferred the system and i ts major features.