Web-based analyses of e-journal impact: Approaches, problems, and issues

This study(1) assesses the ways in which citation searching of scholarly pr int journals is and is not analogous to backlink searching of scholarly e-j ournal articles on the WWW, and identifies problems and issues related to c onducting and interpreting such searches. Backlink searches are defined her e as searches for Web pages that link to a given URL. Backlink searches wer e conducted on a sample of 39 scholarly electronic journals. Search results were processed to determine the number of backlinking pages, total backlin ks, and external backlinks made to the e-journals and to their articles. Th e results were compared to findings from a citation study performed on the same e-journals in 1996. A content analysis of a sample of the files backli nked to e-journal articles was also undertaken. The authors identify a numb er of reliability issues associated with the use of "raw" search engine dat a to evaluate the impact of electronic journals and articles. No correlatio n was found between backlink measures and ISI citation measures of e-journa l impact, suggesting that the two measures may be assessing something quite different. Major differences were found between the types of entities that cite, and those that backlink, e-journal articles, with scholarly works co mprising a very small percentage of backlinking files. These findings call into question the legitimacy of using backlink searches to evaluate the sch olarly impact of e-journals and e-journal articles (and by extension, e-jou rnal authors).