“To his indescribable dismay, a colleague here learned this week that students can easily obtain copies of textbook test banks [online]. I’ve never been a test bank user. But if you (or your colleagues) are, this is a potential game-changer. Thought you might want to know.”

— UCF’s Marshall Schminke this week on the AOM’s OB listserv.

“Markets in everything”, as Tyler Cowen might say.

Test banks, for the unacquainted, are large lists of potential test questions that publishers often bundle with desk copies of popular textbooks.  It has come to the point where textbook vendors often sell desk copies that come with exercises, slide decks, teaching notes, and evaluation materials.  A course in a can!  Marshall’s note, of course, underscores that instructors are not the only consumers in this market.  Caveat testor.

Lasse Lien:

“During World War 2 the British Royal Air Force (and Navy) pioneered the use of empirical and statistical analysis to improve performance.  The RAF collected large amounts of data on exactly where returning aircraft had received damage [from German air-defence fire]. The intuitive recommendation would be to reinforce the aircrafts were the data indicated they took the most damage.  However, realizing that they only had data from surviving aircraft, the [Operations Research] group under leadership of Patrick Blackett recommended that they reinforce the aircraft in the sections where no damage was recorded in the data.”

Link: “Can We Infer Social Preferences from the Lab? Evidence from the Trust Game”

Nicole Baran, Paola Sapienza and Luigi Zingales got a bunch of MBA students to play the Trust Game, an economic game where partners exchange money.  From the students’ behaviour in the laboratory, they assigned them scores on their tendency to reciprocate.

They then compared these scores to MBA students’ pledges in a graduating-class donation campaign.  (They argue that this kind of behaviour amounts to a real-world form of reciprocity, paying back educational opportunities with alumni giving.)  Sure enough, those who reciprocated tit-for-tat with their partner in the Trust Game pledged and gave considerably larger gifts (31% more) to their graduating class campaigns.

Since I’ve run a few studies that use this Trust Game paradigm, it’s nice to see that there’s mounting evidence about the real-world correlates of the behaviour we measure.  (Another neat paper also showing nice links between economic game behaviour and real-world outcomes is this 2007 Journal of Politics article, showing that altruism in the Dictator Game predicts political participation, and this one I mentioned earlier on public-good games and Ugandan co-op farmers).

Do academics collectively scratch each others’ backs, citing researchers who are liberal in citing others?  Gregory Webster from the University of Florida dug through the reference lists from 50,000 articles published in Science, and found evidence that they do.  Articles that cited lots of other articles tended to be better-cited than articles with slender reference lists.  A sign of well-functioning paradigmatic science, or evidence of a shadowy academic mutual-backscratching club?  The jury’s still out (though Webster does a nice job of disassembling the most intuitive interpretation, i.e., that it was being driven by review articles.  Turns out it’s not.  Huh.)

What I found particularly interesting was the change in this relationship over time.  In Ye Olden Days, when dinosaurs roamed the earth and statistics were calculated with an abacus, your article in Science could cite everyone and their dog, and it wouldn’t change how many times you would get cited.

But by the end of the twentieth century, there is a robust association between your citing-ness and everyone else’s willingness to cite you.  Expansive reference lists citing everyone in the field is, for some reason, associated with being well-cited yourself.  The chart above details how the association between citing-ness and cited-ness grew over the course of the last century.

Webster doesn’t yet have data for the 2000-2010 period, but I’ll make two speculative guesses about what is likely to change:

1)  The correlation between citing others and being cited will increase in the 2000s, and

2)  The correlation will be stronger in smaller specialist journals than in mondo-gargantuan journals like Science (the reverse is currently true, according to Webster’s analysis).

Here’s why:

Scholars are lazy.  Or, to use a more flattering term, we’re cognitive misers.  We use search tools like Web of Science and Google Scholar to conduct our literature reviews and figure out who’s done what.  And when you use either of those tools, any paper you turn up includes links to other papers that have cited it (in Google, there’s a ‘cited by’ link right under the result; in Web of Science, the list of papers shows right on the sidebar.)

In the Age of Search, liberal use of citations suddenly adds you to the WOS sidebars and Google Scholar cited-by pages, which makes your article more likely to be found, read, and cited.  So, if you want to get your work read and cited and avoid the dusty scrapheap of uncited science, cite early and cite often.  That’s why I think the reference-citation link will grow stronger in the 2000s and beyond.

The reason I think this association will be stronger for smaller journals is based on the same logic.  Your masterful article on the spawning habits of the Atlantic Cod is unlikely to be overlooked if it’s in Science or Nature.  In fact, most people conducting a literature search will be carefully check in each of their field’s top journals for work on their topic of choice.  But if your towering analysis of cod-spawning was published in some third-tier, specialist journal — say, the Canadian Technical Journal of Fisheries Management — it’s most likely to be discovered by showing up in Google’s cited-by list, or popping up on a Web of Science sidebar.  And the chances of appearing in those lists, as I described earlier, hinges on being a generous citer of other people’s work.

In other words, my prediction is that the Citation Problem Of The Future… will basically be academic linkspam.

On Twitter, GMU historian Dan Cohen (@dancohen) writes:

Twice today I have read that scholars only want to read comments on their work by other scholars. Why?

Let’s presume that this is true, and that there is a tendency in the social sciences to prefer comments from fellow academics to interaction with non-academics.  Here are four (pretty obvious) reasons that spring to mind for me:

1)  Professional socialization:  The hiring, promotion, and tenure processes reward academics for being held in high esteem by other academics.  Occasionally there are rewards for sharing one’s expertise more broadly (teaching, consulting, press interviews), but there are few incentives I can think of that reward seeking feedback on one’s work from outside the academy. 

2)  Specialization:  For a number of reasons, narrow specialization is productive for scholars.  However, non-academics’ interests aren’t tidily confined to a tiny corner of a specialized sub-discipline.  This makes answering their questions harder, in many ways.

3)  A fetish for methods:  For all sorts of reasons, social sciences have become increasingly oriented around sophisticated analytical methods.  Methodological rigour and analytical precision have become particularly important.  The questions academics are used to and are comfortable fielding often have a methodological character to them.  Non-academics don’t share this methodological fetishism, and thus ask very different, unfamiliar, uncomfortable questions (often having to do with, gulp, practice.)  Though these types of questions might be very much valid and valuable, they are unlikely to be “useful” in the twisted sense of helping the researcher to narrow their focus, improve their methodological rigour, and gain them the esteem of reviewers, editors, and RP&T committees.

4)  The research-practice divide: A catch-22 if there ever was one.  Academics aren’t good at (or rewarded for) sharing their research outside academia, so the practice of policy, management, etc. tends to bear very little relation to the findings from research.  Researchers, looking at the fads, mimicry, intuition and other non-science that shapes practice, think of practice and practitioners with disdain.  Lather, rinse, repeat.

So, there’s the challenge:  Four reasons that many academics might shrink away from having to actually get the feedback of non-academics on their work.  And, four reasons that academics relegate themselves to irrelevance by not talking with non-academics about their work.

Photo of noted public academic The Professor (of McDonaldland fame), via Flickr.

Link: Behavioral games and real life outcomes

Economic games in the lab are cute.  But do they really relate to real world problems?

Delia Baldassarri has a post up about her experience with experimental public-goods games and the agricultural economics of Ugandan farmer co-ops.  A great read.

Key alarms and detectors that could have provided an early warning of the Deepwater Horizon explosion were disabled because false alarms were interrupting rig leaders’ beauty sleep, the LA Times is reporting today.

Have a read of the testimony.  It’s gutting.  One of my committee members does a lot of work on safety, and I recall him arguing that one of the most dangerous words in the English language is ‘accident’.  Accident implies random chance.  Accident implies the unforeseen, the unexpected, and the unpreventable.

The Deepwater Horizon blowout wasn’t an ‘accident’ in those ways.  It was a direct causal consequence of a host of deliberate decisions:  On the rig itself, choosing to disable alarms and deciding to stop testing key emergency systems.  And at the levels of the organization and the state, choosing to subcontract in ways that encouraged risk and allowing liability caps that created moral hazard.

Reading the testimony immediately made me think of Organization At The Limit (which I reviewed here), a volume edited by Starbuck and Farjoun about the Columbia space shuttle disaster.  There, the shuttle kept shedding foam – one of the causes of the eventual disaster.  At first, it raised alarms and engineers took notice.  But after the same problem surfaced and resurfaced on subsequent tests, NASA started thinking of it as ‘in family’, their lingo for well-understood and expected issues.  Pressured by time and operating in an organizational culture focused on delivery rather than safety, they stopped attending to the problem, treating each ‘near miss’ disaster like it was a success.Fircture contributed to unsafe decisions and practices at a micro level.

The risk here is that the blame will fall on the folks who make the bad micro-level decisions, as if a massive structural problem could have been avoided with one or two bad apples gone at the lowest levels.  Firing a few irresponsible flunkies is to fixing deeply embedded structural problems as disabling an alarm is to preventing a blowout.   It just doesn’t cut it.

Photo via There, I Fixed It, diligent chroniclers of exceedingly unsafe workarounds.

Experiments and social surveys and game theory are all so terribly passé.  The new way to study trust is by… building a giant trustworthiness robot with big creepy lidless eyes?

SCIENCE!

“Somewhat ironically, harsh public rhetoric can help smooth the way to successful settlements during secret negotiations.”

— Julie Browne and Eric Dickson, ‘We don’t talk to terrorists’:  On the rhetoric and practice of secret negotiations, Journal of Conflict Resolution 54(3), 379-407.

Refusal to negotiate is a common posture in statecraft. Hamas refuses to negotiate with Israel.  Turkey refuses to negotiate with the PKK.  It’s even deployed in lower-stakes conflicts:  In a number of instances, firms (Nestle in India, for instance, or McDonalds in Russia) have refused to negotiate with labour unions that they considered to be illegitimate.

But states, firms (and even individuals) do end up negotiating against counterparts they view as illegitimate, evil, or otherwise beneath diplomacy.  Sometimes they negotiate in secret at the very same moment they are promising never to negotiate.

Browne and Dickson present some interesting game-theoretic modelling showing how the rhetoric of refusal to negotiate can, counterintuitively, help parties reach a negotiated settlement.  If one party blusters and make commitments to powerful constituents never to negotiate, they have high ‘audience costs’ for engaging in negotiation.  They can’t afford to be spurned, won’t come back if a deal can’t be made, and won’t tolerate impasse.  This can be a powerful incentive for otherwise reluctant parties to meet them at the table.

Forget Freedom 55.  This is Zheng Ji, who at age 110 is still an active researcher — and may be the world’s oldest academic.  He started his most recent program of research (on aging) in his 70s.