"Bad Science" and bad science.

Posted by Sebastienne





Ben Goldacre - columnist, GP, empiricist hero of the secular left - published a column in the Guardian this Saturday: "How can you tell if a policy is working? Run a trial". In it, he lambasts UK politicians for their lack of interest in the effectiveness of the policies they propose. And he's right, as far as it goes; policies with intuitive appeal are much more likely to be accepted than policies which just happen to be effective. We might think we can increase the intuitive appeal of tried-and-tested policies by educating people (the MPs who will vote, and the public who will later vote on them) on the evidence; but I don't know if anyone has actually run a trial on this approach, so we'd better not rely on it.

There is a real problem here. It is not a trivial proposition, to convince people that their intuitions are not to be trusted - but it's an important goal that anti-oppression activists and pro-science empiricists share, although we rarely acknowledge it.

People who have studied experimental science, and people who have opened their minds to the experiences of oppressed people, are used to having their intuitions - their preconceptions - challenged by the evidence. Whether the evidence takes the form of a randomised controlled trial, or listening to the lived experience of members of oppressed groups, we learn things that we did not previously realise, and know how to alter our worldviews to accommodate the new information.

So it's easy for us to forget, as scientists or activists, that the majority of people in the world are not used to overcoming their false intuitions in this way. It's why people who are mocking Daily-Mail-readers often focus on phrases like "it's just common sense" - we know that "common sense" is suspect. But not everyone has the same life experiences, and being mocked for relying on common sense can seem esoteric, "ivory-tower", and evokes ideas like "the thought police". So we can set up websites like spEak You're bRanes, and we can mock these attitudes - and I'm not saying we shouldn't, it's important to create spaces of solidarity against the encroaching tide of bullshit - but we have to be aware that these people vote too. These people have a right to shape the country they live in just as much as we do; dismissing them is not going to make their views go away. Simply stamping our feet and insisting they accept our points, because we're right, and rational, and well-educated, is not actually going to change anybody's mind. In fact, it frequently comes across as quite classist.

Up until now, I've been speaking about experimental science and anti-oppression activism as if they are equals; but they are not. The former is practiced primarily by people who are more privileged in our society; the latter is practiced primarily by those who are not. Quite often, the claims of anti-oppression activists are disputed by those who identify with Ben Goldacre's "Do a trial!" philosophy, for not being 'rigorous' or 'empirical' enough.

It's one thing to call on a government to spend less of their money on randomly reforming benefits and more of it on discovering the best approach to take (note re: benefit reforms; this government's approach has been tried in Australia and the USA - both times it failed abysmally, increasing poverty, and the rich-poor divide). It's quite another thing to say to a group of oppressed people who are finally finding a voice, "we won't listen to you until you accumulate enough money and status (ie, privilege) to put together a study which meets our criteria".

Of course, even when the evidence is there, it's not always listened to. An interesting example of this is Stereotype Threat. This is a thoroughly-evidenced effect which demonstrates one of the many small ways that kyriarchal oppression functions. It can be demonstrated with this example:

Go into schools, and give each class a maths test. Also give each pupil a coversheet, asking for their name, gender, etc. In conditions where they do the test first, and fill in the coversheet later, you will find that the girls perform much better than they do in the condition where they fill in the coversheet (and call to mind the idea that "girls aren't good at maths") before they attempt the test.

This finding is reproduced for every oppressed group that have been studied - calling to mind the stereotypes which contribute to your oppression cause you to underachieve. And the world is full of reminders of the ways in which we are inadequate for being queer, female, disabled, of colour, trans*, fat...

And it's not just imposed on us from without - we internalise it, and from a very young age. Kenneth & Mamie Clark's 1930s doll experiments, replicated as recently as 2005, show that Black children have already been socialised by a young age to perceive Whiteness as superior to Blackness. Recent research in the US has shown that male protagonists in children's literature outnumber female protagonists by a ratio of 1.6:1, and "boys redefine female protagonists with whom they identify as secondary characters, and recast secondary male characters as central when retelling the same stories".

So why isn't the empiricist lobby up in arms about these findings? Why aren't they passionately fighting for a world free of this kind of bias with the same fervour that they are fighting against Bishops in the House of Lords, or lack of evidence in UK policy-making? Why hasn't Ben Goldacre blogged about Junkfood Science, or taken on the Obesity Myth, as iconoclastically as he has the "alternative medicine" industry? In fact, I follow Ben Goldacre's Twitter account, and have been saddened to see him post fatphobic comments.

Similarly, empiricist feminists of my acquaintance are torn over Tim Minchin's beat poem Storm - on the one hand, many of them love it for lines like "Do you know what they call alternative medicine that's been proved to work? Medicine!"; but at the same time it uses the gender of the antagonist to demean and diminish her. I don't know if it's the breasts themselves, or the tattooed "butterflies on [her] titties", that are apparently relevant to Minchin's dismissal of her views, but there is a strong implication that this type of interaction is highly gendered - rational man, irrational woman. In addition to this, the two other women in the poem fit into traditional gender roles by encouraging diplomacy over freedom of expression, while the men get all the substantive non-mocked speech-content.

It seems to me that these men's empiricism is limited, at best, as long as they remain unaware of the biases in their own thinking which come from their submersion in kyriarchal culture. I wonder how many people have tried to make this subtle point to them, and have been dismissed as if, like Storm, they are saying "science is just another form of religion"?

To do good science, you have to control for potential bias on the part of the experimenter. To do good social justice work, you have to control for the fact that we are all submerged in kyriarchal power structures which privilege some groups above others. These understandings need to co-exist. There is no such thing as an objective observer - and it's no accident that most of the people who are cast in the role of empiricist heroes by our society are economically privileged white men.