Link: Do journalists decide what gets cited?

York’s Advances In The History of Psychology blog had an interesting entry today about how pop media coverage influences the spread of scientific research.  It mentioned a fascinating natural experiment published in the NEJM in the early nineties.

UCSD sociologist David Phillips and his colleagues wondered how much academics relied on the popular press to gauge the importance of developments in their field of study. To find out, they culled through articles from the New York Times and academic references in the Science Citation Index.  They found that journal articles mentioned in the Times were more than 70% more likely to be cited than those ignored by the Old Grey Lady.

But what if the Times was just really good at picking winners?  Maybe they just wrote articles about inherently important research that’d be cited no matter what.  Here’s the neat natural experiment:

“We also performed the comparison during a three-month period when the Times was on strike but continued to prepare an “edition of record” that was not distributed; doing so enabled us to address the possibility that coverage in the Times was simply a marker of the most important articles, which would therefore be cited more frequently, even without coverage in the popular press.”

What the authors found was that the coverage effect on citations disappeared during the period that the Times was on strike and didn’t circulate.

I described this finding to my partner Indra, and she had an interesting take on it.  Her thought was that it might be funding agencies rather than scientists themselves that are responsible for this press-coverage-citation effect.  Perhaps  the folks who hold the research pursestrings (administrators, funding agencies, foundations, etc.) are more likely to fund programmes of research that pursue ideas they’ve seen covered in the press.  Research agendas with funding are more likely to generate publications, pushing up the citations for studies that are mentioned in the Times.

In any case, this is an interesting (and scary!) finding.  Although I think we’d prefer the spread of research ideas to occur on their merits rather than the whim of reporters, it seems the reality is that popular press coverage has an important effect on how scientific research gets cited.