Ben Goldacre is a bit of a hero to me. Like a lot of people, I discovered Bad Science and skepticism at the same time and found something I wanted to be part of. But now Dr. Goldacre has stepped into my field – education – and, frankly, he’s made a total balls-up of it.
Now, there are lots of my peers in the social sciences who have flakey ideas about science. They’re full of post-modernist relativism, all “different ways of knowing” and “challenging positivist presumptions”. I am not among the science doubters. I like my evidence robust and my social science scientific. When Ben starts calling for randomised controlled trials in schools, I’m on his side, in principle. But on this subject, he has no idea what he’s talking about.
The idea of RCTs in education is an attractive one, but we hit our first problem immediately: how do we agree on what ‘good education’ looks like? Trials in medicine are less problematic, because it is generally uncontroversial what constitutes a healthy body. It is much more controversial what constitutes a well-educated child. I could not disagree more profoundly with the currect education secretary’s view of education, so I am almost bound to reject the conclusions of any trials he commissions.
Perhaps we will be looking to see which interventions result in better exam grades. This, though, would be a spectacular exercise in begging the question, because it would assume that the tests are valid and measure the right things.
We can get around this by only measuring things which are uncontroversial – attendance, perhaps. But then we will have a tendency to pick our research topics based on what’s easy to measure, not what’s necessarily most meaningful.
Bad Science explains superbly why trials must be double-blind to be meaningful. This can’t be done with education. The teacher has to know about the trial, and the children probably do too. If we’re serious about research ethics and informed consent, they definitely do. This is a problem because there’s a thing called the Hawthorne effect, which is that people behave differently when they’re in a trial.
All the problems boil down to dealing with minds, rather than bodies. Minds are complicated things, and they enjoy a great deal more variation than pancreases. As Rebecca Allen argues over at the IOE blog, social science research is far more context dependent than medicine. What works for my kid might not work for yours.
But in the hands of a force-10 moron like the current Education Secretary, RCTs could become weapons of mass destruction as interventions shown to have some usefulness in Surrey are rolled out nationwide without regard for social context and individual differences.
Individual differences matter a lot in education. If we find a teaching method that works for 70% of students, that would be a titanic success. Unless you’re a parent of one of the 30%. Good teachers know this; they employ a variety of strategies to help their pupils succeed.
Differentiation is hard, but essential. If Michael Gove thinks evidence can help us arrive at the Right Way to Teach, he is even more delusional than my colleagues already believe him to be.