Randy Souther, Gleeson librarian and editor of the open access journal Bearing Witness: Joyce Carol Oates Studies, discusses Impact Factor and alternative journal metrics.

When you need to rank the “quality,” “prestige,” or “impact” of a peer-reviewed journal, one name always comes to mind: The Impact Factor.

The Journal Impact Factor (JIF) has been around for decades—longer than any other journal metric, and has been widely used by faculty, researchers, students, librarians, and administrators to help decide where to publish, what to purchase, who to hire or promote, even whom to award grants. It’s a powerful metric, but few really understand how it works.

Impact factors are calculated each year in a product called Journal Citation Reports (JCR). The Impact Factor for a particular journal is the average number of citations received in the JCR year by articles published in the previous two years. If a journal’s Impact Factor was 18.5 in the 2015 JCR, that means that its articles published in 2013 and 2014 were cited, on average, 18.5 times each in 2015. In the absence of other quantitative means to measure journals, the Impact Factor was really quite revolutionary.

Does every peer-reviewed journal get an Impact Factor? No. In fact, the majority of peer-reviewed journals are not even tracked in JCR. How many does JCR cover? In 2016, it covers 11,365 peer-reviewed journals, but surprisingly does not include journals from humanities fields.

Question: based on their Impact Factors, which of these peer-reviewed journals is the most prestigious?

JAMA: The Journal of the American Medical Association
Impact Factor: 37.684

Academy of Management Journal
Impact Factor: 6.233

Acta Mathematica
Impact Factor: 3.719

Philosophical Review
Impact Factor: none

The answer is “All of them!” Each is among the most prestigious peer-reviewed journals in its respective field.

The problem is that Journal Impact Factors cannot be compared across fields, or even across sub-disciplines of the same field, because every field or sub-discipline has a unique citation behavior. Remember: the Impact Factor is measuring the frequency of citations in peer-reviewed journals. In some fields, particularly in the sciences, the scholarship is heavily journal article-based, publication is frequent, and reference lists are very long—a recipe for high Impact Factors. In other fields, particularly in social sciences and humanities, scholarship may be more book-based, with less frequent publication, and shorter reference lists—which may contribute to lower Impact Factors. The Impact Factor takes none of this into account, and unfortunately, neither do some people who are responsible for hiring, promoting, or granting—three activities where consideration of Impact Factors is wholly inappropriate.

It is crucial to understand that the Impact Factor is a metric that applies to journals only. It has nothing to say about individual articles. And it has nothing to say about individual authors. Remember JAMA, with its impressive Impact Factor of 37?—meaning an average of 37 cites per article, etc. In reality, 20-40% of the articles in JAMA are never cited, but a handful of highly-cited articles skews the average.

More problematic is that the Impact Factor is very susceptible to manipulation, and unscrupulous journal editors, or scrupulous journal editors, manipulate it regularly. The Impact Factor calculation allows citations from non-peer-reviewed articles to add to the citation count, but these same articles are ignored in calculating the average. An editor hence, could write a brief non-peer-reviewed editorial for each issue of their journal and cite their recent articles in it, creating an artificial Impact Factor increase.

Fortunately, there are alternatives to the Impact Factor which attempt to rectify its most serious issues. I’ll highlight three: IPP, SNIP, and SJR.

IPP or Impact Per Publication

This is the closest equivalent to the Impact Factor, and uses essentially the same calculation but with the following differences.

• Only peer-reviewed articles can contribute citations to the calculation, which makes manipulation much more difficult. No more editorial inflation.
• It covers more than 21,000 peer-reviewed journals compared to Impact Factor’s 11,000.
• It covers humanities fields!
• It uses a 3-year window instead of 2, which may be a better compromise between fast-moving and slower-moving fields.
• The raw data is freely available to examine, which is not the case with Impact Factor.

SNIP or Source Normalized Impact per Paper

This is a more complex metric that takes the IPP calculations and normalizes the results to account for the different citation behaviors of different fields and sub-disciplines. The same advantages of IPP apply, plus:

• You can now reasonably compare SNIP scores between journals in different sub-disciplines or even entirely different fields.

SJR or SCImago Journal Rank

Another complex metric, SJR attempts to measure actual prestige. Citations are not treated equally. A citation from a high-prestige journal is worth more than a citation from a low-prestige journal. The same advantages of IPP and SNIP apply.

Let’s go back now and look at our four example journals:

JAMA: The Journal of the American Medical Association
Impact Factor: 37.684
IPP: 21.835
SNIP: 9.152
SJR: 6.44

Academy of Management Journal
Impact Factor: 6.233
IPP: 6.991
SNIP: 3.938
SJR: 10.317

Acta Mathematica
Impact Factor: 3.719
IPP: 3.191
SNIP: 4.026
SJR: 8.021

Philosophical Review
Impact Factor: none
IPP: 1.325
SNIP: 3.097
SJR: 3.062

Now Philosophical Review is accorded due respect, and suddenly Academy of Management Journal, in some measures, seems to be as good as if not “better” than JAMA.

The Impact Factor was designed with the sciences in mind, and it tends to work best with many of those fields. If the Impact Factor helps you to find quality journals, then by all means, use it. But I would recommend taking a cue from the humanities: If there are thirteen ways of looking at a blackbird, surely there’s more than one way to measure a journal.

Resources: 

Look up a Journal’s IPP, SNIP, and SJR in the library’s Scopus database. These metrics are also freely available on the Journal Metrics website.

Frequently Asked Questions about Journal Metrics like Impact Factor, IPP, SNIP and SJR

Raw Data for IPP, SNIP, and SJR

The library does not subscribe to Journal Citation Reports (source of the Impact Factor). The nearest access is at the publicly-accessible UCSF library, which is a pleasant walk south of USF across Golden Gate Park. Impact Factors are frequently displayed on journal websites as well.