Again, my worries about publication

Let me start by saying that I not doing much research anymore. I aim for the odd paper or two in between teaching and managing. I don't have illusions that my work was ever that good anyway and these days I aim for thorough and interesting, but it is not cutting edge (really good work requires a different production process from the one I am using). Thus, even though I am not worried for myself, I am still worried about the nature of publication and whether most of us are ready for what it has become.

Last year I wrote about how the goal posts of quality work have moved in Economics. These days even local journals want strategies to deal with unobserved heterogeneity: panel data models or RCTs. There are some other options, but the bell has tolled for quick and dirty papers. In fact, just OK papers will take more time and resources than ever.

The assumption behind this is that if you do good work you will get a fair chance. I have always believed that, but these days I am not so sure. More seems to be required. This week on the Marginal Revolution blog, Tyler Cowan asked whether it is still possible to have a smell test for a good paper. He writes:

As recently as the 1990s, you could pick up an academic paper in economics and by examining the techniques, the citations, how clearly the model was explained, and so on, you could arrive pretty quickly at a decent sense of how good a paper it was. Today there are still many evidently bad papers, but also many more papers where “the bodies” are buried much more deeply.

He predicts that a small number of frauds may do well, producing novel results from privately built data. Because of this possibility, everyone has become more strict and the average researcher may find it more difficult to get their actual novel results published. His second prediction is that having close and reputable associates will thus become important. His question is:

What would you think of a new paper from Belarus, or how about Changchun , which appeared to overturn all previous results?

This bodes ill for South African outsiders, even if we are doing good work.

It also links to a bit of satire that Francis Woolley posted a while ago: The rejection letter I am too politically correct to send. I particularly like this bit:

Unfortunately your paper has nothing that a mainstream North American economist would call theory, and the survey you analyse is one that you collected yourself small, not representative of the population as a whole, and subject to response bias. Basically all you do in the paper is describe a bunch of correlations. Issues of endogeneity are not even addressed.

If you were based in a US university and using American data, I might send your paper out to referees. As it is, I’m not going to bother. I rely on my friends and colleagues to write referee reports for me. None of my friends know anything - or care greatly - about your country. After all, what relevance does your country’s experience have to us? I’m not going to use up my scarce good will on your paper. After all, it’s not like I know you personally, or am ever going to run into you at a conference or anything. You’re not in a position to ever return any favours.

You might say that these are just two blog posts and that good work will always triumph. Maybe.

I agree that you will need to have good work to get into those good journals: Throw in the mathematics, search for those large and novel data sets, learn the latest techniques, crunch the numbers, see if you can identify those causal relationships. It will take time, effort, lots of revisions and incremental improvements before you submit. 

But you also need to work at overcoming the disadvantages of distance, our developing country context and a dearth of old friends at Harvard or the LSE. You should be heard speaking about your good work at the right seminars and conferences. Don't go to nice places for conferences, go to where the journals' editors and reviewers are. Get your work out on RePEc, blog and tweet about it. Build a network with the right people. Locally and internationally.

At least, we are all in it together. For the lighter side of all this:

• Read this Sample cover letter for journal manuscript submissions.
• Watch the meme: Hitler hears from the third reviewer.

    

As recently as the 1990s, you could pick up an academic paper in economics and by examining the techniques, the citations, how clearly the model was explained, and so on, you could arrive pretty quickly at a decent sense of how good a paper it was.
Today there are still many evidently bad papers, but also many more papers where “the bodies” are buried much more deeply.
- See more at: http://marginalrevolution.com/marginalrevolution/2013/02/the-smell-test-for-an-academic-paper.html#sthash.YQPvkbKV.dpuf