Archive | December 2014

Peer review

It must be that time of year.

Not for REF – let’s leave that well alone as there’s enough flag waving to power all of the UK wind farms.

No, it’s peer review time. I’m not alone it seems in having a backlog of reviews to complete. Twitter – magical portal to awesome researchers – revealed a few people also in this situation (@michelleoyen @soozaphone)…now we can wallow in our shared experience of having to give up our ‘free time’ for free. I’m also not going to get into that.

…actually, I briefly* will, despite not planning to write about this. This is specifically about grant reviewing. If you’re running a funding call, should you include legitimate costs for the time of the people you’re expecting to contribute to the success of the call administration? If you have a pool of grant funding you’d like to share with investigators; let’s take a small example, say £600,000. You’ll need academics to give up their home time to review these. It’s unlikely an academic will have time in their ‘working’ day to review a grant.

So, of your £600,000 you expect to fund 20 proposals with a success rate of (lets be generous – 10%). That’s 200 proposals, but you’ve performed an initial sift down to 50. With 3 reviewers each, that’s 150 reviewers. Paying for the reviewers’ time at a sensible rate could mean around £100 per reviewer. In this example that means a total of £15,000 from the budget for reviewing costs. Which is 2.5% of the total funding available for the call.

Don’t get me wrong, I’m not a money-grabbing academic…everything I do is for free apart from my university wage…and there are a LOT of things outside of my official role and within it that will never fit within my contracted hours. I’m not unique in this. I think it would be rather unique if an academic’s home time wasn’t regularly dominated by administration. And I honestly don’t expect the reviewing process to change either. But as the demand to do administration for free increases, and our time away from work becomes even more filled with administration, how are we to balance this, and more importantly ensure our time is valued?

Right, back on track. Peer review season. The reason I started writing was to briefly gather opinion on whether to sign a publication peer review or not. This morning I reviewed a manuscript and decided to sign my review. I want to be as open as possible, but ultimately don’t want it to screw me over in future. Thinking about it, the decision was based on a few factors. Was my review dominated by negative feedback and rejection: No. I included suggestions of how to make the manuscript better for a couple of different audiences. I was also only reviewing a specific part as an expert in a technique. Again, this made my decision easier. Are the authors likely to be b*stards and douche me over in future? Probably not, given the above. So, it turns out my decision to sign my review wasn’t very courageous at all. It’s just a small step to possibly making the peer review process more open. I’d like to think that a consequence of signing my review could be that if I’m not 100% correct or the authors have good reasons for some of the points I raised, then I can learn from their subsequent response, and maybe talk to them at conferences and events in future, and we can all develop and our science can develop.

Rationally, signing my review wasn’t a very risky decision in this case, but will I do the same in different circumstances? I honestly don’t know, but it will probably depend on variations of the points I raised above.

Today’s review was actually quite a pleasant process, being asked to review specific sections based on my expertise and not being asked to comment on other journal-focussed factors I consider irrelevant to me as a peer reviewer. While completing this review today I came across a previous review I did a while ago and the comments to the editor. In the interests of openness, my response to that editor/journal is pasted below:

Regarding the reviewing process, I would also like to point out that the question “If published, is the paper likely to be cited? 1=Yes; 3=No (please expand below on its importance and where it might be cited)” is in its current form an irrelevant question for a reviewer. In my role as reviewer, I check to see if the science is sound, the literature relevant, the conclusions appropriate, and to put forward improvements…not whether this paper will increase the impact factor of the journal. If this isn’t the intention, then perhaps the question could be reworded.

Here are a couple of links to good advice and guidance for peer review:

http://blog.peerj.com/post/73296165864/how-to-become-good-at-peer-review

http://theresearchwhisperer.wordpress.com/2011/10/04/professionally-judgemental/

https://archosaurmusings.wordpress.com/2009/01/03/how-to-review-a-paper/

*   wasn’t brief at all. Apologies. A good peer review of this blog is desperately required.