How to evaluate a knowledge unit

Ask questions that help improve the unit right away

 

stitch

A stitch in time saves nine

English proverb

 

In teaching, we seem to spend plenty of time planning and evaluating lessons. But we don’t spend much time evaluating units. I’m yet to see a blog post evaluate the impact of a scheme of work on pupils’ learning. By the time we’ve finished one unit, we’re rushing on to the next, and the previous unit probably won’t be taught til next year. Improving it straight after teaching it would be ideal, but who takes the time to do that? And so the humble unit evaluation gets marginalised.

A simple unit evaluation is just a set of questions, designed to improve the unit straight away.

  1. IMPACT: To what extent have students learnt what they’ve been taught?
  2. SEQUENCE: To what extent did the sequence aid or inhibit mastery?
  3. EXAMPLES: Which examples could be improved?
  4. MISCONCEPTIONS: Which were the most frequent misconceptions?
  5. TASKS: Which tasks had the most and least impact on learning, and which are missing?
  6. QUESTIONS: How can the (multiple choice) questions be improved?

These allow precise changes to be made quickly. It’s best to get multiple perspectives on these questions, so this blog is a dialogue with Katie, who co-planned and co-taught the unit on Oliver Twist with me, designing many of the resources. The aims of the unit were for all pupils to understand and remember the plot, characters, context, themes and Dickens’ social commentary.

 CharlesDickens

1. IMPACT

The kids I taught Oliver Twist to a year ago still remember the plot, characters, themes and much of the context – it’s secure in their long-term memories. This year, I taught two lower sets: 97% in one set achieved 80% mastery, but only 70% from my bottom set, who required considerable re-teaching. Here’s the correlation between their mastery of content knowledge and their essay performance:

 masteryessay

Katie: ‘By the end of the unit, my pupils could articulate their understanding of the text in both writing and speaking. They could make links between the context and the text, and remember lots of details about the plot, but were less secure in their understanding of Dickens’ intentions.’

 

17Oliver

Low ability Year 7 example conclusion

 

2. SEQUENCE

Selecting 15 episodes from a 53 chapter book is tricky, but I think we’ve found the right selection – apart from the sub-plot revelation at the end. Sequencing the concepts (e.g. injustice, prejudice etc.) will require more thinking.

Katie: ‘The beauty of teaching a novel is that the plot naturally creates a beautifully interwoven sequence. Characters, episodes and concepts are continually referred back to, securing understanding and building up a rich picture of the text. Vocabulary could have been sequenced more strategically, as this was what led to the majority of misconceptions.’ 

3. EXAMPLES

Preparing a model exemplar for each lesson is a good way to expose pupils to analytical thinking. We should now include pupils’ own exemplary paragraphs.

Katie: ‘Paragraph examples were fairly basic, meaning that pupils were able to master the structure quickly. Next time, I would like to introduce more complex paragraph structures later on in the unit, to ensure that all pupils are continually stretched.’

4. MISCONCEPTIONS
Most misconceptions were over complex vocabulary like ‘vigilante lynching‘, and the extent of how autobiographical Oliver was to Dickens: some thought both were in prison or orphans.

Katie: ‘Dickens’ language is notoriously difficult; pupils regularly misinterpreted the meanings of some important words, which hindered their understanding of the text. This prompted me to adjust the approach mid-unit; instead of explaining the meanings of words as they came up during class reading, I tried to preempt the words they might struggle with and teach them explicitly upfront. This is something we are still refining.’

 

UnitSequenceOT


5. TASKS

Weekly multiple-choice quizzes work well for testing the full breadth of the novel. Extended writing practice almost every lesson challenges pupils to think more and more deeply about the connections between characters, themes, concepts and context. Pupils comparing more examples is a next step.


Katie: ‘Comprehension questions gave pupils the opportunity to review what they had just read and practice writing in full sentences. They also allowed me to see the extent of their understanding of the basics before we moved on to deeper discussion and analysis. Hinge questions were also useful, but perhaps we could add more, to assess understanding more quickly.’
6. QUESTIONS

Katie: The options for the multiple-choice questions were not always as rigorous as they could be. We need to make sure that options reveal what pupils were thinking if they select an incorrect answer. 

Daisy Christodoulou suggests adding quotations to the repertoire of questions (mostly on knowledge of plot, characters, context and themes).

What is the best description of this quotation?

‘But even if he has been wicked, think how young he is, think that he may never have known a mother’s love, or the comfort of a home; and that ill-usage and blows, or the want of bread, may have driven him to herd with men who have forced him to guilt.’

a) If children are not looked after properly, it doesn’t matter that much.

b) Criminals should be severely punished.

c) The individual is always responsible for their own actions.

d) If children are not looked after properly, they might turn to crime.

e) All children have human rights.

 

I also think adding rigorous comparative questions can stretch pupils:

What is the most accurate similarity between Nancy and The Artful Dodger?

a) They are both enslaved to Bill Sikes.

b) They both remain loyal to Bill Sikes.

c) They both betray Fagin.

d) They were both trapped into a life of crime.

e) They were both transported for life.

 

Next steps

The evaluation has guided us to make these improvements:

  • 10+ concept explanations: with example series
  • 60+ hinge words: vital & extended vocabulary list
  • 15 summaries: a three-sentence summary of each chapter
  • 15 stretch exemplar models: ‘maximum quality expected’ to stretch top sets
  • 1 stretch exemplar essay: ‘maximum quality expected’ to stretch top sets

Evaluating units after teaching them, and acting instantly on the insights, is well worth the time it takes. It sows the seeds for a plentiful harvest next season: a bountiful crop of student success in the subject.

 Sowingseeds

About Joe Kirby

School leader, education writer, Director of Education and co-founder, Athena Learning Trust, Deputy head and co-founder, Michaela Community School, English teacher
This entry was posted in Education. Bookmark the permalink.

7 Responses to How to evaluate a knowledge unit

  1. Pingback: Pragmatic Education – Essays vs Mastery | docendo discimus

  2. dodiscimus says:

    I think those questions about interpreting the quotation and comparing Artful Dodger and Nancy are superb examples of just how effective MC questions can be in stretching pupils. Do you think that there might also be value in dissecting quotations further as an approach to dealing with misunderstandings of language? E.g. in that quotation “want of bread” strikes me as the sort of phrase a lot of pupils would find difficult so is it that (a) Oliver likes bread (b) Oliver was hungry (c) Oliver could not get bread? This is a much less demanding question than the examples you give – I think my idea is that you might have quite a lot of questions like this in a task and that pupils would get a lot of them right, partly because it would just make them focus on the meaning for long enough to realise that the phrase wasn’t actually about bread. This might speed up the ‘upfront teaching’ of difficult vocab by saving whole class focus for the bits that are causing them all trouble and allow differentiated intervention as you go round checking progress.
    I also want to draw a different line on your graph. I can’t do this in a blog reply so it’s on my blog http://wp.me/p44DHA-3l
    What I’m wondering from your data is whether mastery of the knowledge only affects the quality of essays once pupils have the essay-writing skills to make use of the extra knowledge. I’ve no doubt that having no knowledge would reduce essay quality but maybe basic knowledge is the first requirement, then improving essay-writing skills produces the next step up in performance, then after that better knowledge produces even better performance. This fits so nicely with my view that we shouldn’t get too carried away with the knowledge side of the knowledge vs skills debate that I need to consider bias – the sample size is obviously too small to draw any general conclusions so I’ll leave the thought there.
    By the way, if that’s the quality of work you’re getting from low ability Y7, us scientists have a lot of ground to make up to get low ability Y7 conclusions written to this standard – well done!

  3. Pingback: How to evaluate a knowledge unit | Pragmatic Education | Learning Curve

  4. Pingback: A guide to this blog | Pragmatic Education

  5. Pingback: Life after Slide Sprawl: 4 Renewable Resources | Pragmatic Education

  6. Pingback: Articles | Joe Kirby

Leave a Reply