The Doctor & The Leech
Long ago a travelling physician diagnosed fevers as due to an over-supply of blood, and prescribed leeches as a cure to reduce the excess. ‘Blood-letting’, he said, ‘clears the mind, strengthens the memory, dispels torpor, reduces anxiety and lengthens life.’ He treated many poorly people in this way as he travelled from town to town. Whenever the patients recovered he would boast about the great remedy of the leech. But strangely enough, when they died of their fever, he was never seen at the funerals, for he had already left town.
‘Cryptic, remote, irrelevant and unusable’, writes Tom Bennett on the Times Educational Supplement website: ‘why is so much research in education purest snake oil?’
In March, Ben Goldacre published a treatise on building evidence into education, a long-term aim I share. Dr Goldacre has hundreds of thousands of followers on Twitter, his book Bad Science has sold over 500,000 copies, his TED talk has had over 1 million views, and now the Department for Education have asked him to share his ideas on how to make teaching an evidence-based profession. Headteacher Mark Keary of Bethnal Green Academy, the London school that hosted him this week, said that digitisation should put us on the cusp of a golden age of educational research.
The reaction from the educational blogosphere was cautious. Andrew Old questioned which problems randomised trials would solve, given that the problem is disagreement over educational aims. Tom Bennett thought it directionally right, but was sceptical of its practicality. David Didau said many thought it impractical. Dr Becky Allen welcomed it, but urged us to ask not just what works, but what works under which conditions?
One of the books I read on teaching before I trained was Neil Postman’s Teaching as a Subversive Activity. In it, he argued that the aim of education should be to develop in all pupils ‘a built-in shockproof bullshit detector’. This sort of education would help pupils recognise when they are being lied to and spot the hypocrisies and duplicities in the society they were to become a part of.
The peculiar thing is, the English education system as a whole is remarkably susceptible to faddish and unproven innovations. A new book by Tom Bennett, Teacher Proof, lists a whole raft of unevidenced ideas, from NLP, braingym and learning styles through multiple intelligences, learning2learn, gamification, the flipped classroom and 21st century skills. In Bad Science, Ben Goldacre utterly exposed the psychobabble used to justify Brain Gym. And in a 2012 study, Dekker et al found that teachers who are enthusiastic about the potential of using neuroscience in their practice, find it difficult to distinguish between scientific findings and ‘neuro myths’. Andrew Old exposes three of the most prevalent: brain gym, learning styles and one particular snake oil salesman. The foremost Multiple Intelligences theorist, Harvard Professor Howard Gardner has admitted to being ‘uneasy’ at the way his ideas have been adapted for use in the classroom. On a visit to Australia, he learnt that:
‘an entire state had adapted an education programme based in part on Multiple Intelligence theory…The more I learned about this programme, the less comfortable I was. Much of it was a mishmash of practices – left brain and right brain contrasts, sensory learning styles, neurolinguistic programming and multiple intelligences approaches, all mixed with dazzling promiscuity’.
Education is peculiarly susceptible to the equivalent of bloodletting leech doctors. How can teachers tell the difference between snake oil and useful evidence-based research?
There may yet be hope on the horizon. The question that Ben Goldacre and Tom Bennett are tackling is, how can we tell good educational research from bad? Similarly, the goal of Professor Daniel Willingham’s book ‘When Can You Trust The Experts?’, summarised here, in a field ‘awash with conflicting goals, research wars, and profiteers,’ is to help us evaluate evidence that proponents claim is scientific. He suggests a shortcut: strip it then flip it. To do this, you need to be very clear on three points: (1) precisely what change is being suggested, (2) precisely what outcome is promised as a consequence of that change, and (3) the probability that the promised outcome will actually happen if you undertake the change. Strip any educational advice down to the formula of ‘if you do X, then there’s Y percent chance of Z.’ Then flip the outcome (Z): for instance, 75% fat-free means 25% fat!
What Ben Goldacre is suggesting is a culture and system change where teachers use evidence-based research to improve education. He recommends that:
- research on what works best should be a routine part of life in education
- teachers should be empowered to participate in research
- myths about randomised trials in education should be addressed, removing barriers to research
- the results of research should be disseminated more efficiently
- resources on research should be available to teachers, enabling them to be critical and thoughtful consumers of evidence
- barriers between teachers and researchers should be removed
- teachers should be driving the research agenda, identifying questions that need to be answered.
He realises that ‘different research methods are useful for different questions’: randomised control trials are most useful for evaluating whether an intervention has worked, and to what extent; qualitative research is useful for why and how it worked, but less useful for measuring whether it has. ‘The trick is to ensure that the right method is used for the right question.’
But currently, ‘the basic structures for evidence-based teaching practice are lacking … creating an information architecture might take decades … we are talking about the creation of a whole ecosystem.’ Goldacre calls his idea ‘a first sketch –– I hope others will pull it apart and add to it’. It’s also a call to arms, because ‘good evidence on what works best is worth fighting for’.
Who in teaching is up for answering Ben Goldacre’s call to arms? After exposing Braingym, and demonstrating how much medicine has benefited from evidence-based research, he’s now put the onus on us teachers to create our own evidence-based profession.
Alea iactis est. Ludi incipiant!
Brilliant post Joe!
Great post. At work I run some courses which introduce Science teachers to Action Research and support them to carry it out. Am chatting with colleagues to see if there is some way to extend this, possibly through the National Stem Centre online community group facility.
Great post! I’m trying to bring evidence based teaching to the TEFL world…not sure how successful I’m being.
I’m a bit conflicted. Teacher research is alive and well on the Internet, and promotion of teacher research seems to be growing here, too.,Teachers ARE the researchers. More and more of the quality journals are recognizing this. Goldacre seems to believe that after the overhaul proposed, all education research can still be quantified and extrapolated, while the peer-reviewed journals I’ve been reading are loaded with qualitative classroom research findings using narrative inquiry and exploratory practice. Any teacher afforded the time and possibly funding from his/her institution can self-learn the appropriate approach for the study she/he wishes to undertake.
Perhaps the focus should be on helping the researchers produce more useful studies…by putting them in the classroom as teachers conducting their own research. Then they would no doubt come to understand what the rest of us have — that anything beyond a very short list of basic best practices comes down to the quality of interaction between particular students and their teacher. Sharing the stories of these experiences informs practitioners more usefully than most ‘research’.
Excellent! I have been a Science teacher for 29 years and have never been very interested in educational research until two years ago. An article in a Swedish national newspaper listed a lot of research based on evidence, which really appealed to me. I started to use quizzes for formative assessment and also (like Nobel Prize winner Carl Wieman) gave homework “in advance”. A bit like The Flipped Classroom, which I must say, has some evidence based elements, if you look hard enough 🙂
No wish to appear *extremist* in any way but there has to be something to the growing evidence backed idea that pupils can learn most effectively by themselves. “Education” as we know it should be regarded as the facilitation of that self learning. Perhaps one of the most important qualities a teacher can have is the ability to engender a real enthusiasm for learning that will never stop. The teacher can stimulate learning but should never be regarded as the source of it!
Great summary Joe and great to see you putting Postman’s work to use!
I wholeheartedly agree with your call to arms and part of me really likes Willingham’s simplification approach. This has a place – witness the effect that Visible Learning has had, offering wider discussions of educational research among teachers who might not otherwise be reading research. But the promise of a simple effect if you do x, y or z makes me a little bit nervous. If you look at some of the really effective interventions, they require a deep understanding of the principles of the technique applied to the specific context of the teacher:
To quote an article that contained some of the most intriguing work I’ve read recently:
“…because social-psychological interventions rely on subtle, nonobvious forces, they can lead to polarized reactions—either “uncritical acceptance and overgeneralization on one hand; [or] vilifying criticism on the other”… By understanding the mechanisms underlying the effects of social-psychological interventions, we hope that educational researchers can move past such reactions.” (Yeager and Walton, 2011)
I think the idea of the ‘spirit of AfL’ as against using some AfL techniques speaks to a similar problem.
First we need to get teachers reading and researching – but we need to think about the quality of that work and how we train them up in this in more detail than some of the discussion this week has suggested.
Did I really say Goldarce’s ideas were ‘utterly impractical’? I’d like to modify that to merely impractical. You’re right to point out that he says he welcomes attempts to pull his ideas apart and add to them – that’s all I think we should be doing.
The main problem is his thesis that medicine & education are essentially the same. He cites a few superficial similarities but this, surely, goes against everything he has to say about the role of evidence? We can agree that medicine is about making people healthier but there’s very little agreement on what education actually is. Is it to make people nicer or cleverer? Whatever your opinion on this, it certainly isn’t shared by everyone else in the sector.
This being the case, it’s ludicrous to transplant a system from the domain of medicine and expect it to work in the domain of education. We need to develop our own, domain specific methodology for evaluating the effects of teacher interventions. I’m sure some clever academic somewhere has already thought of this but I haven’t been able to find anything I’m happy with expect perhaps the methodology of Graham Nuthall (but his approach really would be ‘utterly impractical’ for individual teachers to undertake their own research.)
I’ll let you know if I come up with anything…
I agree; education is a particular kind of social system, not a physical one. There are aspects of it which are physical, or which are affected by physical properties, but there is much which is as yet indefinable and unquantifiable (although not necessarily unmeasurable in a broader or more impressionistic sense), because it is about human behaviour, creativity,and cognition. That is, it is about some of the most interesting, yet controversial, aspects of scientific and social scientific study.
The analogy with medicine is interesting but doesn’t hold up in the detail; better perhaps to look at the ways in which social sciences have tried -and failed – to square this circle in the past. Historicism, functionalism, structuralism and the rest are all interesting prisms through which to view people’s behaviour; but none of them, alone, provides a final, totally calibrated empirical answer.
In any social science it’s always worth collecting and codifying the research of the past, and undertaking standardized research where specific measurable outcomes are possible. But as with all social science, there must always be a lot of room in educational research for observational work and narrative studies.
I think Goldacre’s points are interesting – especially his emphasis on respect for the profession and the desirability of a single representative professional body – but all he has really done at this stage is to draw an analogy (with medicine) in order to create a hypothetical model. The issue is: to what extent is this model useful, and to what extent is it merely a stepping-stone to a more subtle use of educational research? And to what extent is his broader model of a more formal professionalization of teachers worth pursuing, whether or not RCTs are the way to go?
I also think it’s important for teachers to engage with this debate rather than rejecting Goldacre’s ideas out of hand as naive. They do not need to be afraid of a ‘scientific’ approach, if such an approach recognizes that teaching is in very large part a finely-tuned and complex kind of *social* skill.
Don’t forget “Bad Education” by Justin Dillion and others from King’s http://www.amazon.co.uk/Bad-Education-Debunking-Myths/dp/033524601X/ref=sr_1_3?ie=UTF8&qid=1363698714&sr=8-3 (they call Ben Goldacre their spiritual leader 😉
Agree with Tom Randolph’s comment – quantitative research has produced little that is more than a strong indication and the qualitative paradigm seems to be more suited to a field where there are so parameters involved. Which leaves educational research somewhat high and dry as the teacher as researcher model which this thinking is – very rightly, in my view – promoting runs up against a wall of refereed journals which take no account of and have no connection with teachers and teachers who have no access to the – often incomprehensible to them academic lingo – of refereed journals.
Reblogged this on DELTA Course Blog and commented:
An interesting blog post discussing Ben Goldacre’s recent artcle in the Guardian and his thoughts on research for education.
Pingback: What can science tell us about how pupils learn best? | Back to the Whiteboard
Pingback: What can science tell us about how pupils learn best? | Pragmatic Education
Pingback: What is the number one shift in UK education I hope to see in my lifetime? | Pragmatic Education
Pingback: The Evidence-Based Educational Research debate. Information in Schools
Pingback: Why isn’t our education system working? | Pragmatic Education
Pingback: What can science tell us about how pupils learn best? | Pragmatic Education
Pingback: researchED 2013
In most areas of medicine you give a pill and observe the effects. In education ‘giving a pill’ is a hugely complicated process involving context, environment and, most crucially, the skill, knowledge and attitude of the pill-giver (that has some effect in medicine, but it pales by comparison to what happens in and around a school) . And it’s not possible even to begin to measure the results without some agreement of what constitutes an effect. Not only that, the whole field is clouded by prejudice, politics and personal experience. Yes, Brain Gym, Learning Styles etc etc have been picked apart. But then there’s a tendency for people to add to the list of scorned procedures anything else they just don’t like the look of, or isn’t what they’ve been doing all their lives, or seems too right/left wing, liberal/authoritarian. I wonder if we could start at the beginning and agree on what constitutes a ‘well educated adult’ — that’d be a good debate — find examples and track how they got there. My guess is there’d be lots of very different routes, and all our fixed ideas about what makes a good school, for example, would turn to dust.
Pingback: Why are we still talking about learning styles? | researchED 2013
Pingback: Why are we still talking about learning styles? | Tabula Rasa
Pingback: instagram marketing tool
Pingback: A summary of ideas on this blog | Pragmatic Education
Pingback: Which cognitive traps do we fall into? | Pragmatic Education
Pingback: How can we tell snake oil from science? | Pragmatic Education | wheredidileavethat...
Pingback: Books, bloggers & metablogs: The Blogosphere in 2013 | Pragmatic Education
Pingback: Free Thinking: I agree with Katharine | Pragmatic Education
Pingback: A guide to this blog | Pragmatic Education
Pingback: What Sir Ken Got Wrong | Pragmatic Education | Magnitudes of dissonance
Pingback: Teachers lead the scientific revolution in education: 44+ seminal articles | Joe Kirby