Bad statistics in health and medical research

The NSW Branch of the Statistical Society of Australia hosts the Lancaster Lecture every year. On Wednesday 24 March 2021, the lecture was presented by Professor Adrian Barnett of QUT. Over 70 people either attended in person or Zoomed in from around the world.

Adrian’s talk was engaging, shocking and entertaining all at once. Some of the sound grabs that really spoke to me were these ones.

Bad statistics is abetting bad science, with all the criminal overtones of the choice of verb. Some of the examples of bad static’s that Adrian had included smoothing syndrome, and the famous example of researcher degrees of freedom (also known as the garden of foreign paths.) I’m getting more interested in this phenomenon as I advise more research students, and have even read the relevant Jorge Luis Borges short story, but that’s a post for another time. Adrian’s own example of researcher degrees of freedom was the “one data set, many analyses” of the relationship between race and red cards in soccer.

There are some fields of research where there are more systematic reviews than there are pieces of original research. As someone put it on Twitter, “People! What are we doing?”

One of the hooks Adrian used to get us interested in his talk was the promise of the three worst statistical methods sections ever. Now to be fair these did come from the register of clinical trials not published research as such. I don’t think I should give them away right now, suffice to say they each consisted of fewer words than I have fingers on my right hand!

One of Adrian’s final points was about research projects require of medical students, another topic close to my heart as a statistical consultant. I liked Adrian’s concept of a focus on competence, not excellence, a being more likely to lead to successful graduates, and more likely to reduce research waste.

Leave a comment