Here’s the second part of what I said yesterday at the ResearchEd event near Birmingham.
From a hundred years of research, there are three deceptively simple insights that when applied well in the classroom, have very powerful effects. They are not quick wins, silver bullets or revolutionary innovations. Instead, they offer something more modest: a chance to focus our teaching and help pupils remember what they’re learning. And expert teachers have been doing them since time immemorial.
Here they are in three words: examples, practice and quizzes. Worked examples, extended practice and frequent quizzes are much-underestimated and under-valued, but there’s a huge volume of scientific research behind them.
Intuitively, using examples makes sense. Any time we learn an abstract concept, the mind yearns for concrete examples. For instance, when I was learning recently about the concept of reliability in assessment, I struggled to grasp it until I heard this analogy with a familiar device: weighing scales. The higher the reliability, the higher the likelihood that two weigh-ins of the same object will yield the same result on the weighing scale. Intuition tells us examples are needed. As Willingham says, ‘people find it hard to understand abstractions: they need concrete examples to illustrate what the abstractions mean.’
Counterintuitively, though, the worked example effect, replicated several times across several subjects since 1985, shows that learners required to solve problems perform worse on subsequent test problems than learners who study the equivalent worked examples. Studying and comparing lots of worked examples reduces cognitive overload. Working memory is freed entirely for the study of the problem and solution steps. In 1987, Zhu and Simon found in a series of long-term studies that a 3 year maths course was completed in 2 years due a focus on worked examples.
So here’s how I use worked examples in English. Take complex procedural knowledge that I want pupils to retain in their long-term memories, like analysis of a poem. Here’s a worked example of poetry analysis that I’d share with my pupils.
It is the insight from these experiments about how to make worked examples work best that most intrigue me. Here is the key insight:
Worked examples must make pupils think hard to identify critical insights by annotating them with what they are supposed to illustrate.
For instance, in English, and other subjects with a heavy writing load like humanities, this means getting students to compare worked examples of model paragraphs, to criticise and improve, to annotate and aspire to. If they haven’t seen an example of what they’re aiming for, how can they work towards achieving it? The best teachers write lots and lots and lots of example paragraphs, introductions, conclusions and essays.
Using this insight, I can improve my use of worked examples. I can provide two or three, and ask students to compare them, annotate them, and work out which analysis of the poem works better and why. Here’s an example of what an annotated worked example would look like.
Why don’t we do this more? It requires a lot of time – and no little expertise.
Intuitively, we recognise practice as vitally important. We can’t imagine anyone improving at tennis, golf or any sport, at piano, in an orchestra, or in any form of music, without extensive, deliberate practice.
The research shows how much better distributing extended practice over time, rather than massed, blocked cramming is for long-term memory retention. This has been tested and demonstrated in hundreds upon hundreds of replicable scientific and classroom experiments, across learning conditions (reading, listening, writing), student characteristics (age, ability & prior knowledge), materials (problems, texts and questions across subjects) and tasks (recall, problem-solving and comprehension). As Willingham says, the three key benefits of practice is that it ‘reinforces basic skills required for more advanced skills, it protects against forgetting, and improves transfer’.
It is the insight from these experiments about how to make practice work best that most intrigue me. Here is the key insight:
Compared to conventional problems, completion problems decrease extraneous cognitive load, facilitate the construction of schemas, and lead to better transfer performance.
Using this insight, I can design completion problems to hone in on what pupils find hardest in their practice. For instance, here is a completion problem I’d use with my pupils to guide their practice of analysing a poem before independent writing practice.
If extended independent writing is the ultimate aim, pupils must be given the opportunity to practice that with less guidance:
What’s exciting is how cognitive science dovetails with expert practitioners. Siegfried Engelmann held that pupils need ‘five times more practice than many teachers expect’. In Doug Lemov’s book Practice Perfect, based on decades of work with expert teachers, codifies how to improve practice. And my veteran colleague Barry Smith has always advocated pre-emptive planning that starts from what students find hardest.
Counterintuitively, longer lags and intervals between practice sessions improve retention. This is an insight more for curriculum design than for classroom instruction, which I’ll leave for another time. I just want to focus on what classroom teachers can do in their day-to-day teaching without changing the entire curriculum.
Intuitively, we know that being quizzed on something helps us remember it. That’s why actors memorising their lines don’t just highlight them or re-read them; they test themselves on them again and again and again until they’re automatic in long-term memory. We can remember poems off by heart if we quiz ourselves on them line-by-line.
Perhaps counterintuitively, the research shows that quizzing is better than studying for long-term memory retention. More than 100 years of research has yielded several hundred peer-reviewed, replicated experiments that testify to this. For instance, 4 blocks of study with practice tests outperformed 8 blocks of study without. Practice testing outperforms restudying. The advantage of practice testing with feedback over restudy is extremely robust. Distributed practice testing is better than distributed practice alone. So here’s an example of a quiz I give to my pupils to start a poetry lesson:
The insights about how best to use quizzes or practice tests are fascinating.
In terms of dosage, it turns out more is better. Shorter and more frequent tests (one a week) are more effective than longer and less frequent (once every six weeks).
On timing, it’s interesting to note that longer is better: sizable benefits are observed when repeated tests are spaced: longer lags produce greater benefits.
To improve on this, I prepare weekly homework quizzes. The more practice testing, the better the impact on long-term memory retention.
Lots of questions remain. Here are a few: how do we create enough worked examples and completion problems, given time constraints? How do we work out how to best space and interleave practice and practice tests? There were some excellent questions from the audience, particularly from David Weston about evaluating impact, and how all this applies to teacher training. David even adapted his own ResearchEd talk in the light of this, which I’m looking forward to seeing – and will hopefully write about.
I ended with an anecdote. My dad’s a prostate cancer surgeon – and he got prostate cancer last year. When I ask him what’s changed most over his career in medicine, he says without hesitation, scientific research has changed surgery. And it saved his life. The screening and operation that cured him were based on applying scientific research to the medical profession. Perhaps when we look back on our careers in education, we’ll say, scientific research has changed teaching.
The scientific revolution has brought us mastery over the world and our bodies: vehicles that permit flight and surgery that allow us to extend life itself. The promise of cognitive science – the science of the mind – is that it could bring us similar mastery over how we learn.
A hundred years of replicated scientific research can begin to tell us not only what works, but what works best, and why.
Many of the references I’ve collated above are from the peer-reviewed paper of five cognitive psychologists (Dunlosky, Rawson, Marsh, Nathan and Willingham) that synthesises over a century of scientific research evidence.
An excellent synthesis.
Here’s a few thoughts: You say of worked examples, “Why don’t we do this more? It requires a lot of time – and no little expertise.” I see your point, but isn’t this ‘just’ the practice of modelling? We can construct these worked examples most productively with pupils in order to share our expert thought processes – this will reduce cognitive load even further and is much more manageable and useful than preparing them in advance. See this post on modelling: http://www.learningspy.co.uk/english-gcse/teaching-cycle-stage-2-model/
And that got me thinking, your model for extended practise is remarkably similar in concept and execution to my thoughts on scaffolding. The key being that scaffolds should not make work easy, but allow pupils to complete difficult work. The trick is, I think, to use supportive scaffolding which can be easily dismantled to allow pupils to much seamlessly from guided practice to individual practise. More detail in this post: http://www.learningspy.co.uk/featured/teaching-cycle-stage-3-scaffold/
I quite like the reduction of the testing effect to frequent quizzes. Have you see Harry Fletcher Wood’s post on using a card sort activity to harness this effect? http://improvingteaching.co.uk/2013/11/20/what-i-learned-from-learning-the-periodic-table-and-other-thoughts-on-memory-and-retention-of-historical-knowledge-and-understanding/
As old as the hills! EDIP.
Explanation, demonstration, imitation and practice.
I particularly appreciate your point about worked examples. I’ve had considerable success using those when I taught writing. I hadn’t really thought about how they might also work in other subjects. I imagine that much of what you say is true. I am, however, uncomfortable with one aspect. Weekly quizzes, as you suggest, would be fine, but I’ve taught places where daily quizzes were the norm and that in my opinion is too much time testing and not enough teaching. Furthermore, the quality of these quizzes is important. The example you gave above is a terrific quiz, as it causes the students to interact and apply their knowledge, but many quizzes I see used for this purpose are little more than fact checks. Also, you advocate using the quiz as a learning tool–this is key, and yet probably the most neglected aspect of quizzing. All of this is to say, that I think teachers need more training on writing good quizzes, otherwise students waste a lot of time taking quizzes that are mere assessments and not teaching tools (as you suggest).
Reblogged this on The Echo Chamber.
You have a knack of neatly summarising and showing the application of research. With regard to the issue of generating the material, we need subject-specific, repositories. Probably these need to be a bit co-ordinated. Do you know if anyone is doing this? I’ll see if any scientists are on #asechat
Joe and anyone else, send me an email with a request for “5-5-5s” and I’ll send you a copy of the world’s greatest study skill. Talk about learning in the classroom.
Pingback: ORRsome blog posts from the week that was Week 13 | high heels and high notes
Pingback: A guide to this blog | Pragmatic Education
Pingback: We can tell VAK to “Vak Off”, but is Cognitive Science just another pedagogic placebo? | Education: the sacred and the profane
Pingback: Motivation and emotion | Pragmatic Education
Pingback: Please sir, can we do more tests? | Newman's blog
Pingback: The Signal & The Noise: The Blogosphere in 2014 | Pragmatic Education
Pingback: Resources to support our research Triads, curated by Richard D | UCL Academy
Pingback: Not reading, but doing | blog.pearsoned.co.uk/coursedesign
Pingback: 7+ tutor | 8+ tutor | 11+ tutor | 13+ tutor | London | 13+ scholarship tutor
Pingback: Articles | Joe Kirby's blog
Pingback: Treasure Trove #4: Cognitive Science Crash Course | Joe Kirby
Pingback: Treasure Trove #5: A Cognitive Science Crash Course | Joe Kirby