Fudging Research

“Fudging” is a nice word; “falsifying” is more accurate.

Here is a news story about a psychology researcher who has been caught
out manipulating his data.

http://news.sciencemag.org/scienceinsider/2012/06/rotterdam-marketing-psychol…

There is one very interesting aspect to this case: the researcher
regarded the dodgy manipulations he used as completely standard
practice.

Recently there has been a small barrarge of research documenting the
high prevalence of statistics misuse in psychology and neuroscience. Briefly, I
thought it might be useful to draw some threads together about this in
one place. I’m not aiming to berate one corner of academia, but just to line up
some context.

Here, brain imaging studies report more positive findings than their
numbers can support:
http://www.badscience.net/2011/08/brain-imaging-studies-report-more-positive-…

Here, neuroscience researchers make an elementary statistical error
with very high frequency:
http://www.badscience.net/2011/10/what-if-academics-were-as-dumb-as-quacks-wi…

Here, evidence of publication bias, where positive findings are easier
to publish (lots on this, in all fields, in my Bad Pharma book this
September):
http://www.badscience.net/2011/04/i-foresee-that-nobody-will-do-anything-abou…

Here, a 2011 paper showing that flexibility in data collection,
analysis, and reporting systematically increases the rate of false
positives:
http://people.psych.cornell.edu/~jec7/pcd%20pubs/simmonsetal11.pdf

Many of these problems occur in all fields of research, especially
medicine (again, I’ve a ton of stuff coming on dodgy medical trials in
September). But it’s good to see that they are being more
systematically assessed and investigated elsewhere, as this is
hopefully the start of a meaningful fix.

It’s my view that the information architecture of academia is flawed,
in several interesting ways, not least of which are that: negative
findings go missing in action; and simple statistical errors are
missed by journals, while the to and fro between peer reviewers and
authors is often dominated by petty disputes, over the contents of the
discussion section of a paper. And, by god, that’s a “buyer beware”
part of an academic paper anyway, at the best of times…