On evidence

Posted by Simon Howard

In my first week at medical school, one of the professors warned that most of what we were to be taught was factually wrong. It was an arresting statement, but it may have been true: Studies have shown that textbooks and experts frequently lag behind evidence, sometimes recommending “treatments” that are actually known to be harmful.

Do Primary Care Trusts do the same? PCTs, like the one I work in, currently commission the majority of NHS services provided to patients in their catchment areas (though not for much longer). Sometimes, academics get frustrated with PCTs for seemingly doing things that either have little evidence, or appear to contradict it altogether. Given that evidence is the bedrock of public health, and given the potential for decisions to affect whole populations, this might seem worrying.

In defence of PCTs, a lot of evidence based work does happen. Most major pieces of work include a review of academic literature at an early stage, and follow the findings. The annual Joint Strategic Needs Assessment and regular detailed Health Needs Assessments also take into account published literature and local and national data in a fairly systematic way.

But there are lots of barriers to following the evidence. Books and books could be written on this topic, from the applicability of evidence in the real-world to deciding if research is really relevant to a particular population. But I’m no expert, and I’m not going to try and describe anything technical, complicated, or even remotely clever. These are just a few examples of practical barriers to following the letter of the academic evidence in public health.

One huge barrier is – as with most things in life – money. In a world of ever-tightening budgets, an academic’s seemingly reasonable intervention can be unaffordable. As an extreme example, research by the FAA and CAA suggests that three or four lives would be saved in an average aircraft fire if all passengers were provided with smoke hoods. However, the vanishing rarity of in-flight fires, the enormous cost of supplying and maintaining smoke hoods, and the cost of the fuel required to propel them around the world, all make this proposal financially unjustifiable.

Not all examples are quite so clear-cut. Sometimes, instead of choosing not to do something, PCTs try to cherry-pick the best bits of interventions in a way that is almost certainly infuriating to the academics who pioneered them, and possibly less effective in practice. But, sometimes, doing something is better than doing nothing.

Often, there can be a big lag between publication of evidence and its implementation. One reason is the complex contractual nature of commissioning: it’s often difficult to make small changes to services that have already been commissioned. The constant pressure to reduce costs incentivises longer contracts which spread the financial risk, but which also increase the evidence-practice lag. I’m sure it’s deeply frustrating to be an academic shouting “there’s a better way to do this” while services continue unchanged.

There’s also a political element to public health. Decisions to cut services that are no longer supported by evidence are particularly tricky. In England and Northern Ireland, the evidence that cervical screening in women under the age of 25 causes more harm than good has led to a withdrawal of the service in this age group. The clear evidence, combined with clear recommendations from the World Health Organisation and National Screening Committee hasn’t stopped this becoming a topic for political debate and petition, and hasn’t (yet) changed policy in Wales or Scotland. It seems likely that this political element will play a bigger part in decision making as public health moves to the overtly politicised world of local authorities.

To me personally, the most frustrating barrier to following the evidence is an inability to access it. It continues to baffle me that the NHS doesn’t have anything like the level of straightforward desktop access to literature that university colleagues have. In the 21st century, it seems crazy that I sometimes have to ask the BMA to take a paper journal off a physical shelf, scan it in, and email it to me as the only practical cost-effective way to access a paper that’s of general interest, rather than something specific to any individual project.

I think a latent awareness of what’s going on in academia is important in public health. It might not matter so much when someone’s doing a big literature review prior to introducing a new service, but it can help with horizon-scanning, and with those little every day decisions that aren’t worthy of a trawl though the literature, and with planning for the future. This is something we can all play a part in: public health professionals probably need to broaden their awareness of the academic things going on around them, and academics probably need to shout louder about the latest developments in their fields. As an associate member, I’m probably biased, but I think FUSE is great at helping both groups.