Monday 17 September 2012

This is what evidence is made of

Posted by Jean Adams

I recently re-joined the systematic review club. I did a systematic review once. It was fine. I learnt how to do it, I did it, I published it. It was a good learning experience. Certainly good enough to learn that I didn’t need to do another one in a hurry. Or at least I didn’t need to do the nitty-gritty reviewing myself. But things happen and before you know it you’re second reviewer on a systematic review that you just can’t pass on to anyone else.
I love a good (and sometimes not so good) radio drama
There are some jobs that were designed to be Friday afternoon jobs. Jobs that clearly need to be done, but that you don’t need to think too hard about. Jobs that you can do whilst catching up on BBC Radio 4 drama serials on the iPlayer. Reformatting the tables in your latest rejected manuscript to meet the exact, esoteric, requirements for the next journal in your list. Adding references from Endnote into Word.

I love a little pile of Friday afternoon jobs. As they don’t require much brain input, I find them easy to churn through and they make me feel unusually productive. Productive, unthinking, with added radio stories. Just what I need to end the week.

In contrast, other jobs are very clearly Tuesday morning jobs. Jobs that need sustained, un-interrupted thought. Jobs where even Radio 3 is intrusive. Drafting the justification section of grant applications. Deciding what exactly is the key message in your latest paper. Working out the analysis plan for the 3MB of data you’ve just received.

I don’t mind Tuesday morning jobs. If I have the time, the space, the right environment and am making progress, I really like the satisfaction of biting off big chunks of Tuesday morning jobs. In fact, high quality Tuesday mornings jobs are what keep me in the job.

I know some people don’t mind systematic reviewing. I know some people even positively enjoy systematic reviewing. These are wonderful people. We need systematic reviews and we need systematic reviewers. I am pleased to count systematic reviewers among my friends. But, really, I am not a systematic reviewer. I’m always happy to come up with the idea and justification for a systematic review on a quiet Tuesday morning. But the real-life screening and data extraction, bread and butter of systematic reviewing are not my bag at all.

The problem, I have decided, with systematic reviewing, is that it is neither a Friday afternoon job, nor a Tuesday morning job. You need to concentrate to decide if the paper you’re reading meets all of the inclusion criteria you’ve set. You can’t possibly listen to radio stories whilst you’re systematic reviewing. But you don’t really have to come up with any great new ideas. The ideas happened way back on a Tuesday morning in November when you drafted the protocol.

I procrastinate outrageously when I am systematic reviewing. I check Twitter. I make a cup of tea. I decide I’m procrastinating too much and that I must not do anything but review until I have reviewed 10 more papers. I wonder what’s happening in the tennis and convince myself that I’ll review much better if I just check the scores and get it out of my system. I think of blog posts I could write.

But, as I am slogging my way through and slowly passing papers from the ‘to screen’ to the ‘screened’ pile, I try and remember that it is systematic reviews that we hope might guide decisions; that this pain is what evidence is made of.

No comments:

Post a Comment