Unknown Professor: Manipulating Journal Impact Factors finds a nice article about impact factors and cites:

In today’s Wall Street Journal (online subscription required) Sharon Begley provides a rare look into the world of academic journal rankings. She describes some of the ways that scientific journals manipulate their “impact factors”.


Note: For a little more information on impact factors, here’s a recent Chronicle of Higher Education piece on the topic, and here’s the Wikipedia entry on the topic.

The article shows that once you have a quantitative scheme to measure things, people will game them. It must matter on the margin, but perhaps the equilibrium effects are small—all this gaming may is easy to see through. And you can pick where to send your papers. You do want to send them to places with higher impact factors. Which means that there is a bit of a viscous circle here. On the other hand, it is not really costly to add extraneous cites.

How permanent are the effects? There must be some path dependence. But I have seen journals wax and wane over time.

All that said, I would think that people in the field see through much of this. It is when you need to talk to outsiders—the dean, university-wide promotion committees, or outside funding agencies—that this is going to matter. How much, I am not sure. It does, however, show that quantitative data is not obviously more objective than more qualitative data.

Here is an example from outside academics. There seems to be a new ipod killer every month or so—quantitatively a better mp3 player, more memory, higher bandwidth or whatever. They never catch on. Why? Probably because the ipod killers’ designs are worse. How would you completely quantify that other than saying people like the Ipod experience better? (Surely everything in not quantifiable by ease of use metrics, but also the coolness factor.)

Back to academics. Good ideas cause people to think differently. It’s not clear that depends on where the paper appears, as long as it is a credible place. And on the margin, I bet the impact factors matter less than the WSJ article implies.

UPDATE: I wonder what the ‘impact factor’ of Asian Journal of Mathematics is: Poincare conjecture solved. And what will it be if the proof is correct? (I know nothing about the journal or its impact factor. I am not a mathematician.)


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s