sociologist, rank thyself!

So, apparently some people have repeated the exercise of providing rankings of the research productivity of sociology departments based on publications in the American Sociological Review, American Journal of Sociology, and Social Forces (via Chris). The authors appear to be from Notre Dame, which by the authors' methodology (e.g., the selected time window and counting affiliations by where authors are now as opposed to where they were when doing the article), comes out #5, far above its usual reputational ranking and well above where it was the last time productivity rankings were done, although then it was by a group at (my beloved) Iowa rather than at Notre Dame. I think Notre Dame has a number of great people, has made some extremely good recent hires, and is underrated by reputational rankings, but one of the curious things about these rankings is that they have a way of appearing just at the time and using just the criteria that manage to favor the department doing the rankings. When the Iowa group did the earlier rankings, they published a way of calculating them by which Iowa came out #1.

I'm all for departments promoting themselves, and I can understand where departments that feel like they have productive people and are collectively underappreciated would want to get the word out. But I don't much like the process of dressing it up like one is engaged in a dispassionate enterprise that just happens to produce results favoring one's home department.

Anyway, there are many obvious criticisms of using this as a general metric of departmental prestige or even department article-producing productivity, which the authors acknowledge (even if they plow ahead nonetheless). I want to offer an additional criticism the authors don't acknowledge: I want to see a defense of the concept that there is presently a "Big 3" of sociology journals. I think there is a "Big 2": ASR and AJS. Nobody in sociology confuses the prestige of a Social Forces with the prestige of an ASR or AJS. But the bigger problem with including Social Forces is that it's not obvious to me that Social Forces is the "next best home" for articles that don't make it into ASR or AJS. I think that much of that sociology right now is conducted in subareas for which the top journal in that subarea the most prestigious outlet after ASR and AJS, and then Social Forces comes somewhere after that. If that's true, then it really makes no sense to include it in rankings like this, as then there is all kinds of bias induced by whether work is in a subarea for which SF is the top outlet after ASR/AJS or not (e.g., ever noticed how many papers on religion appear in SF?).

I don't really mean to diss Social Forces--especially since, um, they could be getting a submission from me in the next few months--it's just that using them in these rankings raises two irresistable ironies. One is that the way the Notre Dame groups motivates their article is by the idea that reputational rankings of departments are fuzzy and history-laden so we should look to some hard-numbers criterion. Fine, then, I want to see a hard numbers criteria applied to establishing the prestige of the journals included, as then otherwise the authors are just using the same kind of fuzzy history-laden reputational reasoning for journals that they see as problematic in departments. The other is that, even though in the best case scenario, Social Forces is the clear third of the "Big 3", yet it publishes more articles than either AJS or ASR, so it actually ends up counting the most in these rankings.

For the record, I'm not much into prestige rankings for either departments or journals. Indeed, it's for that reason that I think when rankings are offered, they warrant some scrutiny.