I’ve made the case before that our curriculum and assessment isn’t designed with memory in mind. Here’s what I spoke about at ResearchEd York: what we can do to improve how much our pupils remember of what they’ve learned.
There’s a mismatch between what science suggests and what schools do on this.
A century of scientific study converges on a key insight for our design of curriculum and assessment: an insight that can be put work immediately, widely, at no cost, and to great effect.
In the scientific literature there are hundreds (if not thousands!) of studies on this, some from as early as 1907, and the research in the last decade is particularly prolific:
In 2013, five cognitive scientists (Dunlosky, Rawson, Marsh, Nathan, Willingham 2013) collated hundreds such studies and showed that practice testing has a higher utility for retention and learning than other techniques:
Cognitive psychologists (Roediger, Putnam and Smith 2011) have summarised from this research the many benefits of low-stakes or no-stakes testing:
The scientists’ review of the vast literature shows that these benefits are enduring (Dunlosky, et al, 2013):
The questions to ask of this research as a teacher are: what’s the optimal format and frequency of low-stakes testing or retrieval practice? For simplicity, I’ll refer to this as quizzing, as Roediger, McDaniel and others do in their work.
Applied research suggests multiple-choice questions are as effective as short-answer questions. The latest research study is as recent as March 2014, so this is a fast-evolving field, and one to keep an eye on.
There’s also a question of whether regular frequency or longer spacing is more important:
It’s encouraging that the research evidence resonates with the work of successful school leaders like Paul Bambrick-Santoyo, who works in over 1,000 schools with 90% poverty rates and presided over dramatic academic achievement:
Now, over the last decade, eleven cognitive psychologists have taken over a hundred years of laboratory research on the testing effect and applied it to classrooms and subject curricula. Here’s what they recommend:
- Use frequent quizzing: testing interrupts forgetting
- Roll forward into each successive quiz questions on work from the previous term.
- Design quizzing to reach back to concepts and learning covered earlier in the term, so retrieval practice continues and learning is cumulative.
- Frequent low-stakes quizzes in class helps the instructor verify that students are in fact learning as well as they appear to be and reveal the areas where extra attention is needed. Cumulative quizzing is powerful for consolidating learning and concepts from one stage of a course into new material encountered later.
- Simply including one test retrieval practice in a class yields a large improvement in final exam scores, and gains continue to increase as the frequency of testing increases.
- Effortful retrieval makes for stronger learning and retention. The greater the effort to retrieve learning, *provided that you succeed*, the more learning is strengthened by retrieval.
- In virtually all areas of learning, you build better mastery when you use testing as a tool
- One of the best habits to instill in a learner is regular self-quizzing.
As far as I know, very few schools are applying these insights of cognitive psychology in their curriculum and assessment systems. Very few design systematically cumulative tests.
Here’s a very rough ‘before and after’ snapshot of what a Year 7 curriculum and assessment system might look like when changed to put frequent, cumulative retrieval practice at its heart:
In summary:
- Repeated retrieval improves long-term memory retention
- Practice testing outperforms often-used techniques like highlighting and summarising
- Quizzing has several benefits for memory and motivation
- The benefits of quizzing for memory retention are long-lasting
- The scientific research chimes with expert experience
- There’s a mismatch between what schools do and what the science suggests
- Cumulative revisiting can improve our curriculum design
- Recent research is exploring the optimal format and frequency of quizzes
I would like to do a much more extensive literature review on this, but for now here are twelve of the papers that I think are useful:
Improving Students’ Learning with Effective Learning Techniques: Promising Directions from Cognitive and Educational Psychology. Dunlosky, Rawson, Marsh, Nathan & Willingham 2011.
Repeated Retrieval Is the Key to Long-Term Retention. Karpicke & Roediger 2007
Retrieval practice is critical in long-term retention. Roediger & Butler 2010
Test-enhanced learning: Taking memory tests improves long-term retention. Roediger & Karpicke 2006
Test-Enhanced Learning in a Middle School Science Classroom: The Effects of Quiz Frequency and Placement. McDaniel, Agarwal, Huelser, McDermott & Roediger 2011
Test-Enhanced Learning in the Classroom: Long-Term Improvements From Quizzing. Roediger et al 2011
Generalising test-enhanced learning from the laboratory to the classroom. McDaniel et al 2011
Quizzing promotes transfer in science exams. McDaniel 2011
The power of testing memory: Basic research and implications for educational practice. Roediger & Karpicke 2006
Both multiple-choice and short-answer quizzes enhance later exam performance in middle and high school classes. McDermott 2014
Retrieval practice with short-answer, multiple-choice, and hybrid tests. Smith & Karpicke 2013
Repeated Testing Produces Superior Transfer of Learning Relative to Repeated Studying Butler, 2010
Reblogged this on The Echo Chamber.
This is a good reminder of what the research says about the movement from short-term to long-term memory, but you don’t draw the other half of the conclusion: the curriculum has to be more than a single march through content; it has to spiral through multiple ‘looks’ at the same ideas and complex tasks. Without moving to an iterative-idea-and-performance curriculum, the material recalled is only about the facts or discrete sub-skills – which is necessary but not sufficient. That is why the use of essential questions that recur with rich content is such a vital piece of design.
This is music to my ears! In particular for Science, the amount of content that needs to be at the forefront of their minds is, I believe, extraordinary when you consider how it’s taught. Looking forward to planning new sow for yr 9 and 7 with growing emphasis on this and repetition. Nice work Joe
This was such a thoughtful presentation. Its implications, as we move forward to curriculum redesign, have been buzzing round my head. I want to think through more of the applications for English and read again some of the stuff on short vs multiple choice answers.
Thanks, Joe.
Reblogged this on Roberts's Room.
An interesting post, and as grantwiggins says above “a good reminder”. These ideas are not new and indeed formed a part of my initial teacher training.
I wonder to what extent schools do not design learning in this way?
One can achieve frequent retrieval in a number of ways, and quizzing in my experience is as reasonable and effective as some others.
I am interested to know Joe, whether you are suggesting that quizzing is a better approach than others or just reminding people that quizzing is useful. I am not absoultely sure what you mean by quizzing but I have a rough idea i think.
I tend to agree with Grantwiggins that multiple looks are highly desirable and whether that is best achieved by quizzing or other means is I think an interesting discussion.
I think these ideas about frequent testing are important and it’s great that you are promoting this research. I have only read a couple of the papers you are referencing so if the answer to the question that follows comes from some of the others, that would be really helpful (and maybe I should look at the SA v MCQ papers). The research designs I have come across are based on recall of things that are simple to understand e.g. foreign vocabularly, lists of nouns, that sort of thing. Are you aware of research into the effect of quizzing on knowledge that is more difficult, or incompletely understood? I’m not clear whether there is such a thing as testing before students are ready, particularly in physics where students often come with misconceptions that are very hard to shift. Also, I guess it’s obvious, but it’s really important that incorrect answers to MCQ are strongly corrected otherwise there is some evidence that the wrong answers can be learned – that’s possibly an advantage to SAQs.
In my experience with maths misconceptions, constant re-enforcement is required and can be built into multiple choice questions when thought about. For example why is the statement x (a common misconception wrong?) or x is wrong, which is the correct answer. Etc etc
Reblogged this on Primary Blogging.
Reblogged this on HPE&things and commented:
Frequent quizzing improves learning…who knew!
Reblogged this on From experience to meaning… and commented:
So true!
I’m going to use this principle with redesigning as maths curriculum next year. Will trial it with 3 classes and i’ll let you know the results next year!
Pingback: Increasing bandwidth – Planning a revision session. | @mrocallaghan_edu
Pingback: A guide to this blog | Pragmatic Education
Pingback: Dereliction! A History, Geography, Art collaborative enquiry | #hackthecurriculum
Pingback: 25 practical blogposts for the English teacher | Reflecting English
Pingback: Using cognitive science to inform curriculum design | Belmont Teach
Pingback: Planning for Progress over Time – Mastery Learning | The View From the Maths Bunker
Pingback: Walking the Walk: Authentic Tests for GCSE classes | mrbunkeredu
Pingback: Make History Stick Part 1: principles | Clio et cetera
Pingback: The Signal & The Noise: The Blogosphere in 2014 | Pragmatic Education
Pingback: Assessment after levels: Don’t reinvent the wheel, just steal the hubcaps! | From the Sandpit....
Pingback: Combining Tradition and Innovation | Pragmatic Education
Pingback: Cumulative Testing | researchersinschools
Pingback: A 5 year revision plan | Pragmatic Education
Pingback: How should I revisit past content? | Bodil's blog
Pingback: The ‘artificiality’ of teaching | Evidence into Practice
Joe, love your posts, these are such a great resource as I’ve been planning Professional Development for teachers here in the Bronx. You might find a couple things I put together, based on this and other resources, around threshold concepts:
Handouts: https://docs.google.com/a/strongschools.nyc/document/d/1lXsUAGb3ksQMImK6dCs1waBUoqqBv3OH0DegGvrL9Gg/edit?usp=sharing
Presentation overview: https://docs.google.com/a/strongschools.nyc/presentation/d/1rIuFnGaT28ak4VPkSpX0ekOxnuB2NEHM6GLxFSXAZe8/edit?usp=sharing
Pingback: Stop disadvantaging the disadvantaged // Some practical tips for teaching & learning. | @mrocallaghan_edu
Pingback: Golden needles in a haystack: Assessment CPD trove #4 | Joe Kirby
Pingback: Treasure Trove #4: Cognitive Science Crash Course | Joe Kirby
Pingback: Treasure Trove #5: A Cognitive Science Crash Course | Joe Kirby
Pingback: KS3 Curriculum Reconsidered | Talking Tongues
Reblogged this on mcguffineducativo.es, translated into Spanish and commented: amazing article, as usual!