Our doubts are traitors,

And make us lose the good we oft might win


By fearing to attempt.

Shakespeare, Measure for Measure

Since taking the plunge and adding multiple-choice questions to my assessment repertoire, I’ve found they have refreshing and unexpected advantages.

They make assessment more reliable, marking less labour-intensive, pupil understanding and misconceptions more visible, and allow a wider breadth of knowledge to be assessed across a unit than just using essays or complex, holistic end-of-unit assessments. They save countless hours of marking downstream, and get pupils thinking deeply about subject content.

Both Alex Quigley and Cristina Milos have written perceptively about how tricky they are to create. How can we ensure that the advantages outweigh the limitations?

Research from Little, Bjork, Bjork and Angello (that Alex cites) suggests not only that they are as effective as short-answer tests for retention, but they also have an important advantage over them – that pupils have to think through incorrect alternatives. The key insight is that these alternatives must be plausible enough to enable pupils to retrieve why correct alternatives are correct and incorrect options are incorrect.

I’ve found these seven principles helpful in multiple-choice design:

1. The proximity of options increases the rigour of the question

For instance, the question is, what year was the battle of Hastings? Options 1065, 1066, 1067, 1068 or 1069 are more rigorous than options 1066, 1166, 1266, 1366 or 1466. Of course, the question itself also determines the rigour: ‘80 is what percentage of 200?’ is much easier than ‘79 is what percentage of 316?’

2. The number of incorrect options increases rigour

Three options gives pupils a 33% chance of guessing the correct answer; five options reduces the chances of guessing to 20%; always create five rather than three or four options for multiple choice questions. A ‘don’t know’ option prevents pupils from blindly guessing, allowing them to flag up questions they’re unsure about rather than getting lucky with a correct guess.

3. Incorrect options should be plausible but unambiguously wrong

If options are too implausible, this reduces rigour as pupils can too quickly dismiss them. For instance, in the question: what do Charles Dickens and Oliver Twist have in common, an implausible option would be that they were both bank robbers. However, if answers are too ambiguously similar, this creates problems. For instance, in the question, ‘What happens in the plot of Oliver Twist?’, these options are too ambiguous:

a)     A young boy runs away to London

b)     An orphan falls in with a street gang of street urchins

c)     A poor orphan is adopted by a wealthy gentleman

d)     A criminal murders a young woman and is pursued by a mob

e)     A gang of pickpockets abduct a young boy

4. Incorrect options should be frequent misconceptions where possible

For example, if you know pupils often confuse how autobiographical ‘Oliver Twist’ is, create options as common confusions. These distractors flag up what pupils are thinking if they select an incorrect option:

a). Both were born in a workhouse

b). Both were separated from their parents and family

c). Both were put in prison for debt

d). Both had families who were put in prison for debt

e). Both were orphans

5. Multiple correct options make a question more rigorous.

Not stating how many correct options there are makes pupils think harder. For example:

Which characteristics of “Elegy Written in a Country Churchyard” can be seen as Romantic?

A. It celebrates the supernatural.

B. It is written in iambic pentameter.

C. It emphasises emotion over reason.

D. It deals with the lives of common people.

E. It aspires to nature and the sublime.

6. The occasional negative question encourages kids to read the questions more carefully.

Once they get a question like ‘Which of these is NOT a cause of World War 1?‘ wrong, and realise why, they’ll work out they need to read questions again to double-check on what it is they’re asking.

7. Stretch questions can be created with comparisons or connections between topics.

What was common to both the USA and Germany during the Great Depression?

a)     Jewish immigration increased

b)     Membership of Ku Klux Klan increased

c)     Public works projects were implemented

d)     Government social programs were reduced

Good, Better, Best

Here is an example of honing a question to take into account the key insight that multiple-choice options must be plausible, but unambiguously distinctive. Pupils aren’t able to work out the good, better, or best options from context alone. They’d need to carefully think through the nuances.

What does the word ‘resilient’ mean in this sentence?

Her resilient attitude toward life enabled her to overcome difficult situations.

Bad multiple-choice options are not plausible

A. depressing

B. dishonest

C. flexible

D. anxious

Good multiple-choice options are plausible and not too ambiguous

A. flexible

B. energetic

C. positive

D. enthusiastic

Better options are plausible and less ambiguous

Her resilient attitude toward life enabled her to overcome difficult situations.

A. flexible and durable

B. energetic and enthusiastic

C. positive and creative

D. logical and calm

Best multiple-choice options plausible yet distinctive

Her resilient attitude toward life enabled her to overcome difficult situations.

A. flexible and durable, with the ability to bounce back from setbacks

B. energetic and enthusiastic, with the ability to turn negatives into positives

C. positive and creative, with the ability to make something out of nothing

D. logical and calm, with the ability to solve complex problems

So, multiple-choice questions require us to design options carefully if they are to become a valuable part of our assessment mix.

Deep roots produce sweet fruits

If Heads of Department invest the up-front time to create them for their subject, their teachers stand to benefit from a strong return on that investment in terms of time saved and pupil thinking – for years to come.