The structure and purpose of…

…an MCQ.

Hello all. Welcome to the 140th edition of TEPS Weekly!

Multiple-choice questions or MCQs are the workhorse of the ‘assessments’ part of teaching and learning. MCQs are omnipresent – whether it is a teacher checking student understanding just after they have finished a topic or a school conducting unit tests or an educational board doing diagnostics of how well students are learning or even in high-pressure and high-stakes entrance tests and qualifying tests.

Given the number of different scenarios in which MCQs are used, it is reasonable to suggest that MCQs have different purposes in each of these scenarios. And this is an important consideration in creating an MCQ. The purpose of the MCQ drives what kind of question we ask, what the wrong answer options will be, and so on.

Purpose of the Question

Before writing the question stem or options, we must ask ourselves: Why am I asking this question? Different purposes demand different styles of MCQs.

To identify misconceptionsHere, it is important to keep in mind that the correct answer option matters, but the pattern of wrong options chosen can be equally informative. It is important to design the wrong options so that we can identify misconceptions held by students. An experienced educator knows some misconceptions that students may have, and can create wrong options based on this.

Example:

Which gas or gases do plants release into the atmosphere during the day time?

1. Oxygen 

2. Oxygen and Carbon Dioxide

3. Oxygen and Water Vapour

4. Oxygen, Carbon Dioxide and Water Vapour

Try answering the question before you read further.

What information do we get about students who may have chosen other options?

Option A: Students have a misconception that respiration does not occur in plants during the day and have not recalled transpiration as a physiological process. 

Option B: Students have not recalled transpiration as a physiological process. 

Option C: Students have a misconception that respiration does not occur in plants during the day.

Option D: Expected Correct Answer

So, we see that MCQs can give us fantastic insights into what students know, don’t know or may be confused about – not just through the correct option but through each wrong option.

To measure automaticity and fluencyAutomaticity is the ability to recall a fact or process without much effort, while fluency is the ability to use a fact or process efficiently. If our goal is to see how quickly and accurately students can recall facts or apply a simple procedure, we should use short, clear stems with one clear concept, one clear correct option. Also, we should step back and ask if MCQs are the best question type for this purpose. Open-ended questions would also work. For example: 38 + 15 = ________. 42 + 8 = ______. And so on. 

However, if we want to pick up where the student is making a mistake, we can quickly get this from MCQs. 

For example:

38 + 15 = ________.

1. 53

2. 413

3. 71

4. 43

Try answering the question before you read further.

The expected correct answer is option A.

Option B: Student has not carried over the ‘10’ from 13 (8+5) to the tens place. Most likely that it is a conceptual error of understanding place values.

Option C: Student has made a procedural error and put in 1 (from 13) in the ones place, and carried over 3 to the tens place.

Option D: Student is most likely not aware of what to do in the carry-over process, and has just added vertically.

So using a bunch of MCQs not only allows you to test automaticity and fluency, but it also gives you an insight into what may be coming in the way of the student achieving the needed automaticity and fluency.

For summative assessments and markingIn high-stakes tests, MCQs often need to sample a broad range of topics and cognitive levels like recall, understanding, application or reasoning. Here, reliability and fairness become critical. If we are conducting a benchmarking test across the state which includes urban and rural areas, the context of questions should be such that it doesn’t disadvantage one group of students. 

For example:

How many 10-litre beakers are needed to hold 106 litres of water? 

This question seems innocent enough. But a student in a rural school who has not seen a beaker or heard of it will react to the word very differently than a student from a city-based private school who has probably used a beaker in their science lab. The solution is simple: use the word ‘pot’ instead of ‘beaker’. By changing the word, we make the responses we get from the questions more reliable (as we are sure that rural students have comprehended the question same as the urban students), and we make it fair for all students.

We have explored how purpose plays a big part in creating the questions and more importantly the correct and wrong options. So let’s just focus on the options now.

Crafting the Options

Wrong options are often called distractors. Why? Are we tricking students by distracting them? No. There is a different lens through which we should look at them. A student who is conceptually sound will not be distracted from the correct option, but a student who is not completely conceptually sound will be pulled into thinking that one of the wrong options could be correct. So, the option distracts the student from the pathway toward the correct option, telling us that the student is not conceptually sound. And if we have designed the correct type of distractors, we will also know what the students need to work on.

Some properties of good distractors:

• Represent typical misconceptions, slips or misapplications we have seen in class

• Avoid giveaway answers that are obviously wrong

• Are not ambiguous or possibly correct in certain situations

When writing a question, it can help to start by listing the most common ways students get this wrong, and then turn those into distractors. The correct answer should be correct for the right reason, and the wrong answers should be wrong for predictable reasons.

The Right Kind of Cognitive Load

Another thing to consider before starting to create an MCQ is the cognitive load of the test taker. Cognitive load is the amount of information that the working memory can hold and process at the same time. When a student is reading an MCQ, they are comprehending the question and the choices given, they are recalling facts from their long-term memory, and trying to answer the question. All of this together constitutes the cognitive load on the student.

Let’s break down this cognitive load:

1. The load of the inherent difficulty of the concept

2. The load of processing information and concept from the question and options

3. The load of processing information around the concept in the question and options

It will be obvious that some combination of the first two are needed to check student understanding and the third one needs to be as low as possible. As we have written earlier, context only helps if it clarifies the question to the student.

Take this question for example: 

During an English club meeting, students were preparing a play. One student, Rohan, had written the line ‘She go to the market every day’. The group leader asked him to correct the sentence.

Which option identifies the tense of the sentence that Rohan will correctly use?

A. Simple Past

B. Simple Present

C. Present Continuous

D. Simple Future

You will see that the testing objective here is to check if students know the tense of the correct sentence, ‘She goes to the market every day’. But all the context of Rohan getting a sentence wrong, another person correcting him, and the correct sentence not even being presented to the test taker is a lot of context which is useless to answering the question. The question could just have been this:

Identify the tense of the sentence: ‘She goes to the market every day’.

A. Simple Past

B. Simple Present

C. Present Continuous

D. Simple Future

Tips to reduce unnecessary cognitive load

1. Cut the fluff – just as written above.

2. Keep the language simple.

3. If context is needed, set it up with 3-5 sentences at the maximum (and this changes with the grade as well).

4. Have one testing objective.

5. If a question needs extra explanation (orally) to be understood, it needs to be rewritten.

6. Avoid a negative question like, “Which is NOT the capital of India?”, but if you have to use it, highlight the negative word.

7. Options that have “None of the above” or “All of the above” also increase cognitive load. If these options do not inform about student misconceptions, avoid using them.

8. Don’t use trick questions like, “Which letter is the vowel in the word ‘RHYTHM’”? 

Remember! The aim should be to challenge and not to confuse. Even experienced teachers rarely write perfect MCQs on the first attempt. Treat your questions as if made of clay that can always be moulded and refined over time, and not set like a ceramic pot which once set cannot be changed. If you would like to know more about how to use MCQs in different types of assessments, please head on to TEPS Courses – Assessments!

---

If you found this newsletter useful, please share it.

If you received this newsletter from someone and you would like to subscribe to us, please click here.

Edition: 4.47